What can be checked cheaply can be automated, delegated or optimised; what cannot be checked stays hard, and is where judgment lives.
Their wordsThe more a task/job is verifiable, the more amenable it is to automation in the new programming paradigm.
Their wordsIf a task/job is verifiable, then it is optimizable directly or via reinforcement learning, and a neural net can be trained to work extremely well.
Their wordsAnd so you're kind of like you're either on rails and you're part of the super intelligence circuits or you're not on rails and you're outside of the verifiable domains and suddenly everything kind of just like meanders.
Their wordsAnd what you can do if you've got one of these conformance suites is you can give it to a a good agent and say, "Write code until this test suite passes." And it kind of will.
Their wordsit turns out you can do this if you think really hard about okay, how do I have agents prove to me that the stuff they've written works?
Their wordsAI is still in a state where it is much better at verifiable tasks than non-verifiable tasks.
Mental modelFeedback loops
Their wordsBut really the software engineering agents I think can be done faster sooner than any other agent because it is a verifiable domain. You can always unit test or compile, and there's many different regions of it can inspect the whole code base at once, which no engineer really can.
Their wordsAnd I think the lesson in loops engineering is like if you can make a problem very verifiable, you can kind of like treat it like a black box
Their wordsI think the skill nowadays is less about prompt engineering and more about figuring out how do you give Claude a hard task that seems a little bit too hard. And then how do you make it possible for Claude to verify its work along the way? And the verification I think is probably the single most important thing that people do not get right
Mental modelValue moves to what stays scarce
Their wordsSo, we're now in a situation where suddenly people can generate thousands of theories for a given scientific problem. And now we have to to verify them, evaluate them and this is something which we we have to to change our structures of science to actually sort this out.
Their wordsMy one little bit, the one little bit of of fundamental uncertainty even on long time scales is this thing about tasks that aren't verifiable. Like, planning a mission to Mars, like, uh you know, doing some fundamental scientific discovery like like CRISPR, like, you know, writing a writing a novel. Hard to hard to verify those tasks.
Their wordsWe already see substantial generalization from things that that verify to things that don't verify. We're already seeing that.
Their wordsa cybernetic machine can nail the task and completely miss the job. coding makes this very very easy to see because the tasks are so legible (and verifiable!): write the function, migrate the API, fix the tests.