korrents

korrents · The Pragmatic Engineer Podcast

Simon Willison: Engineering practices that make coding agents work - The Pragmatic Summit

Simon Willison · 28m · youtube.com

18 korrents from this recording

Simon Willison did not write this page.

Every claim below is a statement made in this recording, quoted word for word and linked to the second it was said, so you can hear it rather than take our word for it. The wording comes from the transcript published alongside the recording; the sentence above each quote is our reading of the claim, not their wording.

  1. 4:07 · watch on youtube.com

    it turns out you can do this if you think really hard about okay, how do I have agents prove to me that the stuff they've written works?
  2. 1 min later
  3. 4:43 · watch on youtube.com

    I think Opus 4.5 was the first one that earned my trust. Like I'm very confident now that for classes of problems that I've seen it tackle before, it's not going to do anything stupid.
  4. 1 min later
  5. 5:55 · watch on youtube.com

    the key thing about TDD is that it means that the agents won't write more than they need to.
  6. 1 min later
  7. 6:41 · watch on youtube.com

    I think I see people who are writing code with coding agents and they're not writing any tests at all. That's a terrible idea.
  8. 1 min later
  9. 7:16 · watch on youtube.com

    just cuz the test suite passes doesn't mean that the web server will boot. You know, there's there's always a chance that when you actually try in the real world, something's not going to work.
  10. 1 min later
  11. 8:37 · watch on youtube.com

    And what you can do if you've got one of these conformance suites is you can give it to a a good agent and say, "Write code until this test suite passes." And it kind of will.
  12. 2 min later
  13. 10:19 · watch on youtube.com

    Anything that you're maintaining over the longer term, the code quality does start really really mattering.
  14. 1 min later
  15. 11:31 · watch on youtube.com

    one of the magic tricks about these things is that they they're incredibly consistent. If you've got a code base with a bunch of patterns in, they will follow those patterns almost to a T.
  16. 2 min later
  17. 13:12 · watch on youtube.com

    The problem with language models is they're incredibly gullible by design. Like, language models do exactly what you tell them to do, and they will believe almost anything that you say to them.
  18. 1 min later
  19. 14:19 · watch on youtube.com

    Problem is, you can solve SQL injection by parameterizing your queries. You can't do that with LLMs.
  20. 2 min later
  21. 15:49 · watch on youtube.com

    I call it the lethal trifecta because the only guaranteed solution is to cut off one of the legs.
  22. 16:18 · watch on youtube.com

    So, I think the most important thing is sandboxing. You want your coding agent running in an environment where if something goes completely wrong, if somebody gets malicious instructions to it, that the damage is greatly limited.
  23. 4 min later
  24. 20:24 · watch on youtube.com

    OpenAI and Anthropic have both realized that code is the most important thing to optimize the models for, cuz that's where the money is. Like coders will spend $200 a month on a plan if it's good enough, it turns out.
  25. 1 min later
  26. 21:35 · watch on youtube.com

    I think the most interesting question is what can the models we have do right now? And so the only thing I care about today is what can Claude Opus 4.6 do that we haven't figured out yet. And I think it will take us 6 months to even start exploring the boundaries of that.
  27. 2 min later
  28. 23:59 · watch on youtube.com

    I think that might be what saves us. I think the fact that no, you can't have one engineer and have him do a thousand projects because after three hours of that he's going to literally pass out in a corner.
  29. 24:22 · watch on youtube.com

    if you've always stuck to two programming languages because of the overhead of learning a third, go and learn a third right now. And don't learn it, just start writing code in it.
  30. 2 min later
  31. 26:46 · watch on youtube.com

    Why would I use a date picker library where I'd have to customize it when I could have Claude write me the exact date picker that I want.
  32. 1 min later
  33. 27:53 · watch on youtube.com

    projects are flooded with junk contributions at the moment to the point that people are trying to convince GitHub to disable pull requests, which is something GitHub have never done.