-
5:06 · watch on youtube.com↗
Their wordsa separate AI model that's checking all of the commands that get run. So, you know, kind of funnily enough, that's actually more secure than the fatigue of having to watch hundreds of shell commands and approve each one.
-
5:20 · watch on youtube.com↗
Their wordsWe've kind of reached the point where if you did a blind test with what the AI model's checking on the commands and what a human's checking on the commands, most of the time the the model's actually doing a better job.
- 1 min later
-
5:50 · watch on youtube.com↗
Their wordsAnd I think the main reason is because we all know how to text, you know, everyone knows how to text, everyone is pretty proficient with using their phones, and we've kind of distilled down this way of working with AI that feels like you're just texting a friend
- 2 min later
-
7:25 · watch on youtube.com↗
Their wordsIt's funny, you don't really hear people talk about hallucinations anymore, because it's kind of a solved problem, mostly.
-
7:54 · watch on youtube.com↗
Their wordsotherwise known as computer use, which basically means that any SaaS product or any website that didn't expose an API or an MCP server can be used headlessly, basically.
- 7 min later
-
14:50 · watch on youtube.com↗
Their wordsUm but the most critical thing here is that if you put the agent loop on the virtual machine, then you can't shut down the virtual machine independently from the uh from the agent server.
- 2 min later
-
16:50 · watch on youtube.com↗
Their wordsSo, what you really want here is you want a durable workflow, which means that it's safe for you to actually retry that, and when you retry after a failure, it picks right up on step three.
- 5 min later
-
21:31 · watch on youtube.com↗
Their wordsIn general, the principle here really is to try the cheapest option or the most reliable option first and kind of work your way up the ladder here.
- 4 min later
-
25:57 · watch on youtube.com↗
Their wordsBut, if you see the model doing, you know, 90% of its time doing shell commands for this one thing, it actually can be more efficient to make a dedicated tool for some of that stuff.
-
26:17 · watch on youtube.com↗
Their wordsLike, okay, this thing failed. Like, click on this link or here's what you need to do next. Turns out that's also very helpful for agents.
- 1 min later
-
26:51 · watch on youtube.com↗
Their wordsIt would be great if, you know, when these untrusted inputs like a email in your inbox could be a phishing email or some webhook, you don't want to send that to the model and act like it was a user instruction.
- 1 min later
-
27:26 · watch on youtube.com↗
Their wordsOkay. Context engineering is al- also known as harness engineering. Like, I feel like it's kind of the same thing. They've evolved over time.
- 4 min later
-
31:45 · watch on youtube.com↗
Their wordseventually it's going to hit a point where it needs to summarize and all compression is lossy. So you have to figure out the best algorithm to do the compression and not lose important details
- 6 min later
-
37:51 · watch on youtube.com↗
Their wordsAnd what we mean by this is you kind of have to internalize and assume that the models today are the worst they'll ever be, which means that in 6 months uh you might need to completely rebuild the UI.
- 1 min later
-
38:50 · watch on youtube.com↗
Their wordsI think it's very possible that pretty soon your agents will be working for weeks or months. And it does bring up some kind of interesting questions in terms of how we train and evaluate those models.
- 1 min later
-
39:21 · watch on youtube.com↗
Their wordsI think ideally if you have this digital colleague, you give it literally all the tools that you have if you were onboarding a real human. And so I think organizations are still warming up to that idea, of course.
- 1 min later
-
40:30 · watch on youtube.com↗
Their wordswhat do you do when you need to evaluate an agent that runs for a month, but the models ship every month?
- 1 min later
-
41:18 · watch on youtube.com↗
Their wordsuh, I don't think that we have it perfect yet and I think trying to thread the needle between being proactive and suggesting
-
41:38 · watch on youtube.com↗
Their wordsI also think that it's a pretty safe bet to assume that at this time next year there will be significantly, significantly more tokens flowing in the world
- 1 min later
-
42:31 · watch on youtube.com↗
Their wordsAnd the reality is, this is just kind of the new normal now where every new product is going to have to have some kind of agentic side to it, either integrating or it will become a you know, a headless piece of data for another agent to use.
- 1 min later
-
43:38 · watch on youtube.com↗
Their wordsUm so I'm still I'm still pro looking at code. You know, I still look at the code that I'm writing. Some people not looking at code. I still think it's helpful, especially to understand the architecture that you're shipping.
- 1 min later
-
44:48 · watch on youtube.com↗
Their wordsBut I think it's helpful to understand just like when you're building a web app, it's helpful to know how databases work because increasingly the there are so many decisions you make at the product and the harness level that are influenced by the models themselves.