Find and fix flaky tests in a broken CI pipeline overnight
With CI logs in reach, Claude Code tracked a rolling 30-day pass rate, chased the noisiest flakes, built minimal repros, and tried fixes overnight. Over a few weeks the pipeline shrank by about 20 minutes and master pass rate climbed ~35% from roughly 40%, though some suggested patches were thrown out on review.
- Agent or tool
- Claude Code
- Topic
- Software Development
- Reported on
- Hacker News
- Added
- 2026-10-15
Why it matters
Coding agents are moving from autocomplete to doing whole tasks: reading a codebase, making a plan and changing many files. This case is a concrete example of where that works in practice. It is one of 106 sightings filed under Software Development on Agent Sightings (number 101 by interest), and one of 164 involving Claude Code.
How you could try it
Here is a reusable prompt, written by us in our own words, that captures the task. Paste it into Claude Code or any similar agent and adjust the details to your situation.
In this repo, measure flaky tests from CI history, prioritize the worst offenders, attempt minimal repros and fixes overnight, and open draft PRs. I will review before anything merges.
Tip: Work on a fresh branch, ask for a plan before edits, and review the diff and test results before merging.
Source
Read the original first-hand account on Hacker News: https://news.ycombinator.com/item?id=47970174. Our summary is a paraphrase; details are in the source.