22 korrents from this recording
Every claim below is a statement made in this recording, quoted word for word and linked to the second it was said, so you can hear it rather than take our word for it. The wording comes from the transcript published alongside the recording; the sentence above each quote is our reading of the claim, not their wording.
Their wordsAnd in all of these applications, the customer is making a decision based off of the recommendation of the AI model more or less. Uh, and this means that if the customer is ultimately like kind of making the decision, this means that if the system makes a mistake, um, that's okay because usually the person can kind of recognize that or or decide what to do even despite that mistake.
Their wordsUh and this means that they're going to be far more useful when they're operating fully autonomously. And as a result, this requires us to develop physical AI systems that make far fewer mistakes than the machine learning systems that have been deployed thus far.
Their wordsUm, and while this generally improves the reliability of the model, uh, people eventually get tired and it's hard to get really, really high reliability with a person that's manually tuning this. And so what would be even better is if the AI system itself can iterate on the scenario in which you want it to have higher reliability where it on its own automatically seeks out places where it needs more data, where it needs more supervision.
Their wordsNow maybe this isn't completely out of the question but this would be quite challenging uh to do and that's because the calculus is a little bit different. We're not just running compute to optimize for a use case. We're actually running the robot in the real world and using the hardware and attempting the task in the real world.
Their wordsand trying to fold two boxes together isn't useful data that will teach the model how to get better at the task. And so that would be kind of wasting a lot of time on the robot attempting to go down the wrong path for solving the problem. And so instead of spending a lot of time trying to do that task, what we'll do is we'll actually have a human intervene and show the robot what to do and how to recover from that situation.
Their wordskind of going back to this reliability question, we took this policy and we ran it not just once, but we ran it for 13 hours straight. Uh and we basically wanted to evaluate is this policy not only good at making a latte once, but can it do so reliably to the extent that it would be needed to be useful in the real world?
Their wordsSo, you might be surprised to hear that most state-of-the-art foundation models for robotics have no memory or no context. They're just operating on the current sensor observations, the current camera readings, uh, and predicting actions based off of that.
Their wordsAnd then for longer memory, for memory that spans multiple minutes or multiple hours, we don't necessarily need video of exactly what happened in that past uh in that past history.
Our reading
A model you have to fine-tune for each thing you want it to do is not a general-purpose model.
Their wordsBut if you have to fine-tune a model, you actually aren't getting a general purpose model um for the things that you want it to do because you have to fine-tune it for each individual thing.
Their wordswhen you can basically bridge the concept of an avocado and a chair and show that you kind of can combine those two, it means that the model has at least some kind of conceptual understanding of what an avocado is and what a chair is to the point that it can combine them into something that exhibits both concepts at the same time.
Their wordswe see that the across the board the single PIO like pre-trained PIO7 model matches or outperforms the fine-tuned specialists that were developed with reinforcement learning post-training for those downstream tasks.
Their wordswe find that the performance on held out tasks decreases dramatically. Whereas if we um just take out a random 20% of the data that's less diverse than the most diverse subset, the performance um only decreases a little bit. And so this suggests that actually having really diverse data plays an important role in enabling it to generalize to new tasks.
Their wordswithout metadata prompting when you add lower quality data from 80% data to 100% data the performance actually decreases which is perhaps not too surprising because you're adding lowquality data to your data mixture whereas with the metadata prompting the performance actually increases when you add that lowquality data
Their wordsI think that the distribution channel for physical models is going to be slower uh unfortunately because you actually need a physical robot there
Their wordsat the same time in terms of the capabilities of these models I think that we are really starting to get to the point where these models are actually useful in the real world and I think that getting to the kind of the capabilities of chat GBT I think is um yeah very much on the horizon in the next few years.
Their wordsI mean at the very least I actually think that just starting with a generalist policy and then fine-tuning it even like right off the bat uh can be really effective.
Their wordsBut I think the PhD is an incredible opportunity to first learn a lot about how to handle uncertainty, how to pick good problems to work on.
Their wordsUh and just like how we see in language models how now a lot of time is spent actually generating data, generating synthetic data by actually running the model and having it think through things. I think a lot of the data in the future in robotics is going to be the robot attempting to do lots of tasks in lots of real world circumstances.
Their wordsUh and likewise um robots can't like watch a person doing something and then figure out how to do it themselves directly. They really need their experience on their own platform um to learn effectively.
Their wordsuh the we found that that leads to improvement and we saw in the shirt folding example we saw like a quantitative bump from using that sort of imagination compared to not using it. At the same time I think that the model actually performed surprisingly well without that as well.
Their wordsand yeah, I think it's either you need to figure out how to make the data faster or you need to figure out how to be faster than the data. We've seen the evidence of being able to be a little bit faster than the data.
Their wordsUm the robot essentially had learned this sort of equivariance between his left hand and his right hand so that it could actually transfer uh behaviors from one hand to another. uh despite the fact that that was never in the data.