the test that fails only on tuesdays
FW-003running
a flaky suite in a big pipeline. every fix reveals another flake wearing a trench coat.
hypothesis
flakes come from shared state. find the state, kill the flake.
apparatus
- buildkite
- retry logs
- a spreadsheet of shame
procedure
- 1.collect two weeks of retries. group by failure signature.
- 2.the top 20% of signatures explain 80% of pain. start there.
- 3.TODO: finish this sentence with what you found.
measurements
retry rate, beforeSAMPLE VALUE
retry rate, afterTODO
failure notes
- quarantined tests breed. nobody owns a quarantined test.
conclusion
unwritten. the test continues.
next experiment
ownership labels on every quarantined test. watch what happens.