Back to the series

Act 1 · A person refuses · 03

The AI does not know when to stop, in either direction

2645 characters#AIEngineering#LLM#EngineeringManagement#SoftwareEngineering
Evidence card: three rulings, quoted verbatim in Chinese with English glosses labelled as glosses.
Evidence card

Below is the post exactly as published on LinkedIn, unchanged.

My AI implementer twice tried to narrow what a review required. Then it ran nineteen rounds of review on a number it never measured twice.

Those are the same failure, and not one the model can fix for me.

TL;DR — knowing when something is finished is the judgment I have not managed to hand over. It runs in both directions, and each direction needed its own ruling.

The first time was a code review that came back with ten findings. Another model runs those, never the one that wrote the code. That is the red team. The implementer proposed a plan: fix most of them now, take a smaller guard for the expensive one, and push the full version to the next day. It cited the slippage rule, the project's own timebox on a daily unit of work, as the reason.

That rule is real. It caps a day's slice of work at two calendar days, and at the deadline you cut scope, close it, tag it, and move what you cut to the roadmap. It is a rule about the calendar. I do not read it as a licence to under-fix a review finding, and reading it that way is how a deadline rule quietly turns into a quality rule.

I ruled: fix everything the red team flagged.

Two days later it happened again, quieter. Every P1 and P2 was fixed. Two P3s were left with a note — "no action" on one, "residual gap" on the other. Nobody was overruled. The work just stopped slightly early, at the point where what remained was small enough to sound optional.

Same answer. All of them, P3s included.

Then it went the other direction, and that one I did not see coming.

A probe script had been through nineteen rounds of that review. Round after round found something. Round after round got fixed. And the number the whole script exists to produce was still n=1, a single run, never repeated. Nineteen rounds of rigour had bought nothing measurable, and from the inside the process looked perfectly healthy the entire time.

So I called it the other way: this probe stops here. Converge. Stop piling on content that buys nothing.

What three rulings taught me is that the model is genuinely good at the work and has no stable sense of when the work is done. It stops early when what is left looks small. It runs too long when round after round still produces a finding. Both of those feel like diligence from the inside, which is exactly why it cannot catch either one.

When this project began, neither completion rule was written down. Both are written down now as decisions I made later, not as constitutional rules, because I had to make the same call twice before I noticed it was the same call.

Enough, and not enough. That is the judgment I have not been able to delegate.

What a reader takes away

Stop-judgment is bidirectional, and each direction needs its own written rule — one to stop under-delivery, one to stop rigour that has stopped buying anything.

Check it yourself

Each pointer opens a real file on the LearnArken repository’s main branch.