The least verifiable part of AI research is the judgement call about what goes into the one big training run.

Like the thing that I think is most likely to be sort of the bottleneck in terms of like the AI are really good at verifiable domains but not not at doing the actual thing is just like big experiments. You only get a few tries um well a few is maybe a bit understated but like basically like historically R&D has been driven by doing near frontier scale experiments and that has been pretty important and like actually doing the one big training run where you decide exactly what to include in that.