Tests written by a model in the same context as the change are what find the bugs; the automated tests left behind are the lesser product.

Added to korrents 2 Sept 2026 · How quotes work · Something wrong? Tell us

Where this collides

• Cannot both be true

  In an agent-written codebase the agent-written tests are equally untrustworthy; manually using the product is the only reliable measure of whether it works.

  Steinberger has the model write tests in the same context as the change because that is what finds the bugs. Zechner says the tests an agent writes for its own code are as untrustworthy as the code, and that manually using the product is the only reliable measure left. They cannot both be right about whether a model can check its own work.

Shared subject first, then similar wording. They may agree or disagree.

• Modern AI models tend to write tests for every tiny change, even when those tests aren't useful. PSPeter Steinberger24 Sept 2026
• When debugging with AI it is easy to get stuck inside the model's search space, chasing its candidate fixes while the real answer sits in context only you have. BVBen Vinegar2 Jun 2026
• Agentic code review raises the floor but cannot be trusted, because the model reading the code is the same model that wrote it, and it will tell you the code is great. DHDex Horthy15 Jul 2026
• Automatic context given to coding models should clear a high bar for accuracy and relevance, since more context makes them slower, costlier and more confused. MTMichael Truell6 Oct 2024
• In the future, humans will not find bugs in model-produced code in reasonable time and will only review system composition, not via PRs or line-by-line review. TBThorsten Ball19 Sept 2026

On the map

Open the map on its own page