ZeroDailyGet it on Google Play

AI coding agents fail 64% of the time, burning huge cash on bad runs

Original headline: In 27,280 coding agent runs, 64% of inference spend went to runs that failed — runta.com

More in Programming and developer news

Researchers built a massive benchmark called FrontierHarness v1.1, running 27,280 AI agent attempts costing 83 billion tokens. They tested 22 different coding 'harnesses' (the systems that call AI models) across 5 different AI models to see which combinations worked best. The goal was to see if you should switch your harness whenever you switch your underlying AI model.

The results were expensive and stark: 64% of all inference spending went to runs that ultimately failed. The 'best' harness changed dramatically depending on the model used; one harness was cheapest with one model but dropped to 16th place with another. The key takeaway is that developers can't just pick a model and a harness independently—choosing the wrong combo wastes serious money on failed code generation.

Read the original at runta.com

Satirical commentary on real tech news, summarised with AI. Read the original source for the facts. How ZeroDaily works

Open in the appGet it on Google Play

More stories

Add ZeroDaily on Google