Jakub Pachocki
Chief Scientist at OpenAI. Worked on the research that turned scaled reinforcement learning into reasoning models, and writes about where he thinks that path leads.
Jakub Pachocki did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
Filter All topics · Newest first
-
Their wordsThe idea of racing forward at all costs seems absurd once one internalizes the seriousness of the stakes.
-
Their wordsAI is grown more than designed - it is, to first degree, the product of repeating a straightforward optimization step many times on a hard-to-imagine amount of compute.
-
Our reading
How far AI systems are scaled must be constrained by how confident we are in their safety.
Their wordsScaling AI systems has to be constrained by our confidence in safety.
-
Their wordsWe may be used to thinking of AI as tools, but some agents will be pursuing their own objectives.
-
Their wordsI expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established.
-
Our reading
No lab has solved alignment and monitoring well enough to keep scaling at maximum speed much longer.
Their wordsCurrently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.
-
Their wordsThe strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI.
-
Our reading
The current speed of AI progress could be sustained all the way into recursive self-improvement.
Their wordsBased on internal results, I have a strong expectation that this speed of progress could be sustained into recursive self-improvement.
-
Our reading
Chain-of-thought monitoring is getting less reliable, not more, as models grow more capable.
Their wordsHowever, unfortunately our evaluations indicate our ability to rely on CoT monitoring is progressively diminishing.
-
Their wordsAnd I believe that international coordination on future AI development needs to become a top priority for governments around the world.
-
Their wordsStill, I expect general AI progress to increasingly be bottlenecked by confidence in monitoring.
-
Our reading
Nobody is prepared for the consequences of machine intelligence continuing to rise this fast.
Their wordsI am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence.
-
Their wordsThe fundamental challenge of AI alignment is generalization.
-
Their wordsA clear risk discussed throughout this year is to cybersecurity: the models are becoming superhuman in their ability to break in and out of computer systems.