Someone shows you a graph: the service is doing four hundred milliseconds at the ninety-fifth percentile. Is that a problem? You cannot answer. Not because the data is missing — it’s right there — but because a single number carries no meaning on its own. Four hundred milliseconds is a disaster if the service normally does forty, and it’s a quiet Tuesday if it normally does three-eighty. The number only becomes information when you set it against what it usually is, and that “usually” is itself something you had to measure, over time, before the moment you needed it.

This is the part of observability that gets undersold. Collecting the metric is the easy half; the hard half is knowing its baseline well enough that a deviation means something. Teams that respond well to incidents aren’t the ones with the most dashboards — they’re the ones who know, in their bones, what their dashboards look like on a normal day. They can glance at a panel and feel that it’s wrong before they can articulate why, because they’ve watched it enough to have an internal model of normal. Without that model, every graph is just a shape, and you’re reduced to guessing which shapes are healthy.

The trouble is that normal isn’t a constant. It has a shape in time: traffic breathes with the day and the week, latency creeps up as data grows, a number that’s alarming at 3 AM is unremarkable at noon. A threshold set against a single snapshot of normal will either scream through every rush hour or sleep through a real 3 AM failure, because the same absolute value means different things at different times. Useful alerting has to encode the rhythm — compare against the same hour last week, not against a flat line someone picked once and never revisited. Most alert noise is just a baseline that stopped being true and nobody updated.

There’s a deeper trap, which is that normal drifts, and drift can hide the very problem you’re watching for. A memory leak that grows a little each day doesn’t trip an alert if the alert quietly recalibrates to whatever “current” looks like — you’ve taught your baseline to accept the disease as health. The slow degradations are the ones that slip through, precisely because no single day is different enough from the one before it to look wrong. Catching them means holding onto a longer memory than the drift, comparing not just to yesterday but to a month ago, and being willing to ask whether the gradual new normal is actually fine or just familiar.

So the discipline underneath good observability is less about instruments and more about knowing your own system’s ordinary behavior — and keeping that knowledge current as the system changes. A metric without a baseline is a number without a verdict. The work is building the baseline, encoding its rhythms, and refreshing it often enough that it still describes reality, while staying suspicious of the possibility that normal has quietly moved somewhere you didn’t want it to go. “Is this bad?” is always really the question “is this different from what it should be?” — and you can only answer that if you did the work of knowing what it should be, before you had to ask.