It's Probably Not a Race Condition
The report comes in: “the build randomly fails when using -j8 but works fine with -j1.”
Classic symptom. Everyone’s instinct is the same: race condition. Two tasks running simultaneously, clobbering shared state, producing non-deterministic output.
Sometimes that’s right. More often, it isn’t.
What Actually Causes Most “Race Conditions” in Parallel Builds
Make’s job is to construct a dependency graph and execute it. With -j1, tasks execute in topological order — one at a time. With -j8, anything whose dependencies are satisfied can run immediately.
The key word is “whose dependencies are satisfied.” If your Makefile is missing a dependency edge, Make doesn’t know that task B must complete before task C. With -j1, B happens to finish before C starts anyway, purely due to execution order. With -j8, C might start before B finishes. The build breaks.
This isn’t a race condition in the concurrency sense. It’s a correctness bug in the dependency graph. The parallel execution just exposes what was always wrong.
A Concrete Example
Consider a static library target:
$(BUILDDIR)/libfoo.a: $(OBJECTS)
ar rcs $@ $^
lib-static: $(OBJECTS)
ar rcs $(BUILDDIR)/libfoo.a $^
lib-static is a phony convenience target. It builds the same artifact as the real target. But notice what’s missing: lib-static doesn’t declare $(BUILDDIR) as a prerequisite.
With -j1, the build directory always exists by the time lib-static runs because something else created it earlier in the sequential chain. With -j8, lib-static might run first. The directory doesn’t exist. ar fails.
The symptom — intermittent failure under parallelism — looks like a race condition. The fix is a single line:
lib-static: $(BUILDDIR) $(OBJECTS)
ar rcs $(BUILDDIR)/libfoo.a $^
One missing prerequisite. That’s it.
How to Find These
The diagnostic process:
1. Get the exact error message. “Random failure” usually isn’t random — a specific command failed for a specific reason. No such file or directory on a directory means a missing mkdir dependency. undefined reference means a linking step ran before compilation finished.
2. Look at what the failed command needed. What files or directories must exist for this command to succeed? Trace backwards from there.
3. Find where those prerequisites are created. Is that creation declared as a dependency of the failing target? If not, you found your bug.
4. Confirm with forced ordering. Add the missing dependency, rebuild with -j8, and verify the intermittent failure stops.
When It Actually Is a Race Condition
Real race conditions in build systems are rarer, but they happen. Signs that point to actual concurrent mutation:
- Multiple targets writing to the same output file simultaneously (parallel
arcalls updating the same archive) - Build cache corruption where two jobs share a global temp directory
- Makefile variables mutated at recipe time (uncommon but possible)
The distinguishing feature: with a dependency graph bug, the error message is always about a missing thing (file not found, symbol undefined). With a true race, you might see corruption — truncated archives, partially-written headers, files that exist but contain garbage.
Most build breaks under -j are exposing pre-existing dependency graph omissions. Before reaching for mutex-like workarounds (forced sequential execution, .NOTPARALLEL), spend five minutes tracing what the failing command actually requires. The missing edge is usually right there.
Fix the graph. Keep the parallelism.