Test how well coding agents can onboard to your product
Case study takeaway: you enter your product's site on AX Check, which first checks the site itself. Three coding agents then try to onboard end to end in sandboxes. Results appear as scores out of 100 with a letter grade and a leaderboard.
- Agent or tool
- Custom agent
- Topic
- Software Development
- Reported on
- Hacker News
- Added
- 2026-10-15
Why it matters
Coding agents are moving from autocomplete to doing whole tasks: reading a codebase, making a plan and changing many files. This case is a concrete example of where that works in practice. It is one of 106 sightings filed under Software Development on Agent Sightings (number 19 by interest), and one of 182 involving Custom agent.
How you could try it
Here is a reusable prompt, written by us in our own words, that captures the task. Paste it into your own agent or any similar agent and adjust the details to your situation.
In this repo, test how well coding agents can onboard to your product. Read relevant code, apply a minimal patch, run available checks, and summarize the diff. Stop before merge or deploy.
The prompt breaks down into 3 steps you can check as the agent works:
- In this repo, test how well coding agents can onboard to your product.
- Read relevant code, apply a minimal patch, run available checks, and summarize the diff.
- Stop before merge or deploy.
Tip: Work on a fresh branch, ask for a plan before edits, and review the diff and test results before merging.
Source
Read the original first-hand account on Hacker News: https://news.ycombinator.com/item?id=49744416. Our summary is a paraphrase; details are in the source.