Tested Is Not the Same as Verified
Writing a test and verifying behavior feel like the same activity, but they’re two different claims stacked on top of each other, and it’s worth separating them because only one gets checked automatically. The first claim is “the code does what this assertion says” — that’s what the test runner confirms every time it goes green. The second claim is “this assertion says the right thing” — that the test actually encodes the behavior you meant to guarantee. Nothing checks the second claim for you. A test can pass perfectly, forever, while asserting something subtly wrong, and the passing will never tell you that, because passing is defined relative to the assertion, not relative to reality.
This gap is where an entire category of bug hides: the test that’s wrong in a way that makes it agree with broken code. It happens more often than it should, because writing a test and writing the code it tests often come from the same mental model, at the same time, by the same person. If that model has a mistaken assumption baked in, the test and the code share the mistake, and they confirm each other instead of catching it — the test isn’t checking the code against reality, it’s checking the code against the same misunderstanding that produced it. A green suite in this state looks identical to a green suite that’s actually correct, which is exactly what makes it dangerous; there’s no signal from inside the test run that tells you which one you have.
A related version of the same gap shows up in tests that assert something true but not the thing that matters. A test can correctly verify that a function returns a value without empty-string-checking whether that value is the right value, that a request returns 200 without checking the body reflects reality, that an operation completes without checking it did the correct thing rather than merely finishing. These tests aren’t wrong, exactly — everything they assert is true — but they’re aimed at a weaker property than the one anyone reading “test passed” assumes was verified. The test is honest about what it checked; the reader’s assumption about what “passed” implies is where the gap opens up.
The corrective isn’t more tests, it’s a different question asked at the moment a test is written, before it has a chance to calcify into false confidence: if this assertion is wrong about what correct behavior looks like, would this test still pass? If the answer is yes — if the test would pass regardless of whether the underlying assumption is true — the test is verifying that the code ran, not that it’s correct, and that distinction matters enormously even though both states look identical on a dashboard. Good tests are ones that would fail if your understanding of the requirement were wrong, not just ones that fail if your implementation diverges from your understanding of the requirement — the first catches misunderstanding, the second only catches slip-ups within an unexamined misunderstanding.
This is also the argument for a second pair of eyes on tests specifically, not just on code: someone who didn’t write the implementation is far more likely to notice that the test is checking the wrong thing, because they don’t share the blind spot that produced both the test and the code. And it’s the argument for occasionally testing against an independent source of truth — a spec, a domain expert, a real-world example — rather than only against your own mental model, however carefully you think you’ve applied it. Tested and verified look the same from the outside: green, passing, confident. The difference only shows up when someone asks not “did this run correctly” but “was this ever checking the right thing” — and that question has to be asked on purpose, because nothing in the test suite will ask it for you.