Somebody Has to Decide What Matches
The first two pieces treated data movement as an event: a file arrives, a file leaves. The version that eats the most engineering time is neither — it’s the ongoing one, where the product and some other system are both supposed to hold the same information, forever, while both of them keep changing. And the difficulty there isn’t moving the bytes. It’s deciding when a record here and a record there are the same thing.
Every integration eventually runs into this. Their system calls it a company, yours calls it an account, and the only thing connecting them is a name somebody typed. “Acme Inc” and “ACME, Inc.” and “Acme” are three rows and one customer, and no amount of careful field mapping fixes that, because the problem isn’t the fields — it’s that neither side has an identifier the other side agreed to use. Matching on the closest available proxy works until it doesn’t, and when it fails it fails in the expensive direction: two customer records merged that shouldn’t have been, or one customer silently split in half.
Which is why the durable answer is to stop inferring identity and start storing it. When a record is first matched — however that happens, including a person deciding — write down the correspondence: this local id maps to that remote id. From then on the sync doesn’t guess, it looks up. That mapping table is unglamorous and it’s the whole game; without it, every run re-derives its conclusions from data that keeps drifting, and the conclusions drift with it.
The other half is what happens when both sides changed. Any sync that runs in two directions has to answer this, and most answer it by accident: last write wins, where “last” means whichever job ran most recently, which is not the same as whichever human edited most recently. That produces the worst kind of data loss — quiet, plausible, discovered weeks later by someone who’s certain they fixed that field. Better to decide deliberately. One side owns each field and the other side follows. Or conflicts are detected and surfaced rather than resolved silently. Both are defensible; having no answer is not, and “no answer” is the default state of most integrations that describe themselves as two-way.
Then there’s the failure everyone plans for and nobody instruments: the sync that stops working. Not loudly — the credential expired, the remote schema changed, the job’s been erroring for eleven days — and because nothing is visibly broken, both systems keep serving stale answers with total confidence. This is the logging thread’s point about the reader and the alerting thread’s about ownership arriving together. Somebody needs to be able to answer “is this in sync right now, and if not, since when,” and that requires the sync to report its own state rather than being presumed healthy because no one complained.
So the thread closes where it started, in a different register. The import is a first impression, the export is a promise about the exit, and the sync is what you live with in between — and all three are really the same question about whether the product treats a customer’s data as something it holds on their behalf, or something it has captured. The first is a design stance that costs real work. The second is what people assume by default, which is why the work is worth doing visibly.