How AI Agents Generate Technical Debt at Industrial Scale 😬

Everyone is obsessed with benchmarks like SWE-bench and HumanEval, proving that neural networks are about to replace developers. The problem is that these metrics evaluate exclusively snapshot programming. Give a task -> AI spits out a PR -> tests pass -> triumph, we're done.
But in real production, writing code that passes a pipeline once is not even half the battle. Real engineering is when your code can be modified without pain a year later. And that's where AI starts to fall apart.

Folks from Alibaba and Sun Yat-sen University released a paper describing how they made agents not just close single isolated issues, but maintain real codebases over a long distance (the benchmark averages 233 days of history and 71 commits per project).

They created SWE-CI — the first benchmark that evaluates not the ability of AI to spew out a piece of code to close one issue, but its ability to actually maintain a project.
Instead of a dumb "pass/fail", they introduced the EvoScore metric. It penalizes technical debt. If your fix on the 3rd iteration makes it impossible to add a new feature on the 10th iteration — your score plummets.

To do this, they set up a CI-loop with two agents:
1️⃣ Architect: looks at failing tests (test gaps) and writes a high-level spec.
2️⃣ Programmer: tries to write code to fulfill the spec.
And so on in a loop.

What are the results?
🟠Zero-regression rate at rock bottom. The main stability indicator. If a test passed before and fails after your commit, it's a regression. So, most hyped LLMs have a zero-regression rate below 25%. In other words, trying to fix one thing, they break something that already worked in 3 out of 4 cases. Only Claude Opus managed to cross the 50% threshold.
🟠Strategic planning is absent. Models like Kimi and GLM (yes, the Chinese are actively pushing their LLMs) are geared towards quick results — they produce excellent code in the moment, but quickly drown in their own legacy over the long haul. DeepSeek and GPT models try to play the long game, but still stumble.

In short, neural networks still don't understand "conscious effort." They operate on probabilities, not architectural vision. Give an agent a production project to maintain for six months, and it will turn it into a spaghetti monster that no one dares to touch.