Read the piece on normaltech.ai ↗
Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.
Their wordsOur view is that reliability lags capability, and that reliability will remain a barrier to deployment unless researchers and developers focus effort on improving reliability as a separate dimension from accuracy.
Their wordsOur view is that for autonomous operation in high-stakes contexts, we need 3-5 "nines" of performance - 99.9% to 99.999% accuracy - in order for reliability to become a non-issue, and we don't think LLM-based agents are on track to reach such a threshold.
Their wordswe expect each order of magnitude decrease in unreliability (1-reliability) to be as hard as the previous one.
Keep reading