Operate and improve CI/CD
Treat flaky tests as defects
Measure, quarantine narrowly, assign ownership, and fix nondeterminism instead of normalizing reruns.
9 minute lesson
Start from the behavior you can observe. The pipeline records retries for diagnosis but does not convert an unstable failure into silent success.
A flaky required check teaches people that red can mean nothing, weakening the entire release gate. The useful target is not encyclopedic coverage. Make one deliberate choice, observe its consequences, and know which requirement would make you revise it.
It is tempting to add unlimited automatic retries and call the workflow reliable. That trades a clear decision for an assumption you will eventually have to debug. The stronger move is to track flake rate, preserve first-failure evidence, isolate the smallest unstable test, and set a repair deadline. Make the boundary explicit in code and in the project notes.
Tie the decision back to the larger job: Operate the pipeline as a production system with useful signals, bounded cost, maintenance ownership, and recovery. Record enough evidence that another developer can repeat the result without relying on your memory.
Introduce one timing-dependent test, measure its behavior, and replace time guessing with a deterministic condition. Repeat it once with an invalid or hostile condition and write down the boundary the failure revealed.
Lesson completed