Real-Time Data Lies. Build Self-Correcting Systems Against a Source of Truth.
The same phone call can show up as both an incoming call and an outgoing one, and which answer you get depends on nothing more than which signal won a race to your server.
That is real-time data in one sentence. It lies more often than people think, not because it is broken, but because a live event is only ever a partial snapshot. The cheap move is to trust the first one that arrives. I build systems that treat that first signal as a draft and reconcile it against the record that actually owns the truth.
The short version: Real-time events are fast but incomplete. The same call can read as inbound or outbound depending on which snapshot lands first, and its duration often reads as zero because the call has not ended yet. The fix is not to correct one live event with the next, which is just as partial. Name the one source that owns each fact, let the live feed write to the screen for speed, then reconcile every row against the settled record and let that record win. Add one guard so a slow, stale update can never overwrite a fresher correction.
Most real-time systems make the same mistake. They take the first event that lands and write it down as fact. The event is fast. It is also incomplete. It knows one slice of what happened and nothing about the rest, and that gap is where the wrong answers come from.
Take a live operations dashboard that watches phone lines for a busy front desk. Who is calling, which queue they land in, who picks up, how long they talk, who gets missed. The team runs their whole day off that screen, so it has to be both instant and right.
Why a live event is a partial snapshot
A phone system does not hand you one clean fact when a call happens. It sends a stream of snapshots. Call starting. Call ringing. Call answered. Call ended. Each one is a partial picture of the same call, fired a fraction of a second apart, and they do not always arrive in order.
Every call also has two sides. When a staff member dials out, the system marks the staff device as the outbound side and the other phone as the inbound side. That is correct from the system’s point of view, because each leg only knows its own direction. So the first snapshot to arrive is a coin flip. Grab one leg first and the call reads as inbound. Grab the other first and it reads as outbound. Same call, two answers, decided by which signal won the race.
Duration has the same flaw. A snapshot that says the call was answered often lands before the call has an end time. Subtract a start from a missing end and you get zero. The call is real. The duration just has not happened yet.
| The live event | The settled record (source of truth) | |
|---|---|---|
| Speed | Instant | Arrives after the call ends |
| What it knows | One leg, one slice, maybe out of order | The whole call, every leg |
| Direction (in or out) | A coin flip, decided by a race | The direction the system stands behind |
| Duration | Often zero, the end has not happened | The real talk time |
| Its job | Tell the story as it happens | Hold the final word |
This is the shape of almost every real-time feed: live, fast, and only half there. A single event cannot tell you whether it is wrong or simply early. It only knows itself, so asking it to judge the whole story is asking the wrong source.
Why a source of truth beats the first event
The same phone system keeps a full call record. Not the live snapshots, the settled history. After a call ends, that record holds every leg in one place, with the direction the system itself stands behind. It is slower to arrive. It is also authoritative. One source that sees the whole call beats many sources that each see a sliver.
So I decide up front which source owns each fact, and let everything else defer to it. The live events still write to the screen the second a call rings, because the team needs to see it happen. They no longer get the final say. A separate step pulls the settled record and reconciles the row against it. If the live guess said inbound and the record says outbound, the record wins. If the call is stuck at zero and the record holds the real talk time, the record fills it in.
The order matters here. Correcting a live event with the next live event does not work, because the next one is just as partial as the first. A later snapshot can land thin, missing the queue or the caller name, and overwrite values an earlier one already got right. The fix is to stop treating “this is wrong” and “this is incomplete” as the same problem. You cannot tell them apart from inside the stream. You can tell them apart against the record.
How a self-correcting system holds the line
A self-correcting system does not wait for a person to spot the mistake. It checks itself on a schedule and against the source that owns the answer.
I wire the reconciliation to fire from several places. The moment a call ends. A sweep that targets known mislabels. A scheduled pass that re-checks everything. The instant someone opens a record for detail. Different triggers, one rule: whatever the authoritative source says, the row matches it. If a new bug ever creeps into the live mapping, this layer heals it on the next pass, and the system does not need me watching it.
One guard makes the whole thing safe. Several correctors can target the same row at the same second, and a slow one holding a stale copy could overwrite a fresh fix with an old answer. So each correction claims the row first, does its work, then writes only if nothing newer landed while it was working. No corrector gets to undo a better answer with a worse one.
That single rule turns a fragile feed into one that repairs itself. The live layer stays fast for the people who need speed. The settled layer keeps it honest. Neither one has to be perfect on its own.
Trust the record, doubt the rumor
This is not really about phones. It is about every system that takes inputs that look like facts and are actually first drafts. Sensor readings. Status updates from another service. Partial writes. A user action mid-flight. They all arrive early and confident and half-formed.
The discipline is the same in each case. Name the one source that owns the truth for each fact. Let the live signal move fast and tell the story as it happens. Then reconcile against the record that sees the whole thing, and let it win every time the two disagree.
Real-time data earns its speed by being early. It does not earn the final word. Give that to the source built to hold it, and the system stops needing you to catch its mistakes.
Trust the record. Doubt the rumor.
Frequently asked questions
Why is real-time data often wrong?
Because a live event is a partial snapshot. It captures one slice of what happened, sometimes out of order, before the full picture exists. It is not broken, it is early, and treating that early signal as final is what produces wrong answers.
What is a source of truth in a data system?
It is the one record designated as authoritative for a given fact. In a phone system it is the settled call record that arrives after the call ends, holding every leg and the direction the system stands behind, rather than the fast live snapshots.
Why not just correct a live event with the next live event?
Because the next event is just as partial as the first. A later snapshot can arrive missing fields and overwrite good values an earlier one already captured. You cannot reliably tell “wrong” from “incomplete” from inside the stream, only against the authoritative record.
What makes a system self-correcting?
It reconciles its data against the source of truth automatically, on multiple triggers such as when an event completes, on a scheduled sweep, and when a record is opened, so any error is healed on the next pass without a person noticing or intervening.
How do you stop a stale update from overwriting a newer correction?
Have each corrector claim the row, do its work, and write only if nothing newer landed in the meantime. That way a slow process holding an old copy cannot replace a fresher, more correct value with a worse one.