There’s a particular kind of monitoring failure that doesn’t look like a failure until you do the math. The alert fires. You investigate. You find the problem. And then you check the logs and realize the problem has been present for three weeks. The alert wasn’t broken — it fired when it was supposed to fire, on the metric it was supposed to watch. But by the time it fired, the cost was already paid.

This is the late alert problem. It’s distinct from the silent failure problem (where no alert ever fires) because there’s an alert, it’s just not timely. The system is being monitored, but what’s being monitored is a lagging indicator — a metric that reflects the problem only after it has reached some threshold, accumulated some effect, or propagated to the layer where it becomes visible. The gap between the start of the failure and the arrival of the alert is where the damage happens, and that gap often isn’t visible in the alerting system itself. You see the alert timestamp, not the failure timestamp.

The most common cause is monitoring what’s convenient to measure rather than what matters to measure. Database connection counts are easy to instrument and alert on. The number of background jobs that completed successfully in the last hour is harder. So the alert goes on the database, and the job processing failure persists undetected until it cascades into something the database metric does catch. By then, the queue is backed up, the downstream system is stale, and the user-visible symptom that finally generated a ticket happened three weeks after the processing stopped.

The fix requires working backwards from the failure mode rather than forwards from the available metrics. The question isn’t “what metrics do we have that we can set alerts on” — it’s “what would it look like if this specific thing failed, and what’s the earliest signal we could detect.” For periodic jobs, the earliest signal is usually the freshness of the output, not a crash or error count. For queues, it’s the age of the oldest item, not the length. For caches, it’s the staleness of the cached value, not the hit rate. These metrics are often harder to instrument, but they’re the ones that would have caught the failure at day one instead of day twenty-two.

The late alert is a signal about measurement strategy, not about the alerting system itself. The alerts are working as designed. The design is what needs examination: are the metrics being watched the ones that would detect the failure at onset, or the ones that become noisy only after the problem has propagated far enough to be impossible to miss? The difference between those two is usually measured in weeks of cost.