korrents

Charity Majors

@charity-majors · 50 positions · 1 change of mind

Co-founder and CTO of Honeycomb, previously an infrastructure engineer at Parse, Facebook and Linden Lab. Co-author of "Database Reliability Engineering" and "Observability Engineering". Writes at charity.wtf about on-call, hierarchy, hiring and the engineer/manager pendulum.

Charity Majors did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

  1. I also believe that, you know, there's been this whole push towards individual output, but teams are still what matter. Teams output.
    spoken · machine transcript hear it at 0:07:24 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 1st of 26 in this recording

  2. We can go fast. Let's do same thing faster, you know, just boom. And I've come to feel like that is a very immature description of what better is.
    spoken · machine transcript hear it at 0:08:01 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 2nd of 26 in this recording

  3. And it wasn't actually the models. It was the harnesses. It was all the tooling.
    spoken · machine transcript hear it at 0:10:31 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 3rd of 26 in this recording

  4. What I was saying in that piece though was, I think we were right to be skeptical that time. But now, I see the same thing playing out with would you be willing to ship it some code you didn't read? There's no point in arguing about if it will happen or when it will happen. Talk about what it would take.
    spoken · machine transcript hear it at 0:12:04 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 4th of 26 in this recording

  5. I mean, if you if you think about it, you could generate 10,000 variants of a function faster than you could write it once.
    spoken · machine transcript hear it at 0:14:55 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 5th of 26 in this recording

  6. Like that just does not seem like the ideal artifact. We should be able to store them somewhere. We should be able to have architecture diagrams that we can review and discuss that generate that code to spec.
    spoken · machine transcript hear it at 0:17:03 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 6th of 26 in this recording

  7. it's always weird to me just how much software engineers really seem to believe that the world exists in the repo. It doesn't. It's production, you know?
    spoken · machine transcript hear it at 0:20:22 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 7th of 26 in this recording

  8. Production is not what happens after development. It is a stage of development.
    spoken · machine transcript hear it at 0:21:02 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 8th of 26 in this recording

  9. As soon as you merge, it should be going out. You like you should have to stop the train to make your code not go into production as soon as you've merged.
    spoken · machine transcript hear it at 0:22:03 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 9th of 26 in this recording

  10. It's like they have all the wisdom of their most senior engineers looking at every single diff and that is fantastic. Which means that you don't have to worry about remembering and looking and nitpicking and all the things that we're not good at anyway
    spoken · machine transcript hear it at 0:26:36 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 10th of 26 in this recording

  11. Okay, if I'm not going to read this code, how do I know it's going to perform within boundaries of the last code that I generated? That is conformance testing.
    spoken · machine transcript hear it at 0:28:02 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 11th of 26 in this recording

    code generation

  12. If you're debiting from this trust account in the creation of the code, it has to get built up somewhere else.
    spoken · machine transcript hear it at 0:28:26 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 12th of 26 in this recording

  13. Another another thing we point out is just there is no human in the loop. You own the loop. The loop is yours. The loop is mine. It would not exist if it was not for me. So I am the owner, right?
    spoken · machine transcript hear it at 0:32:00 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 13th of 26 in this recording

  14. Here's a baseline. Uh you cannot send anyone something you haven't read. And in fact, if it would take them longer to read it than it took you to make it, it's probably slop.
    spoken · machine transcript hear it at 0:34:55 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 14th of 26 in this recording

  15. You can use AI as a shortcut to help you not have to think too much. And you can use AI to help you think more deeply and more rigorously. And both of those use cases have their place. But when it comes to your core job function, we primarily want the second one, right?
    spoken · machine transcript hear it at 0:36:42 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 15th of 26 in this recording

  16. See, the problem is that neither side is making it up. Like they are seeing really scary trends.
    spoken · machine transcript hear it at 0:38:17 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 16th of 26 in this recording

  17. One of the things that's ironic though is that both of these sides feel like they are the tiny minority and they're outmanned and they're being suppressed and they are standing up for what is truth and valor in the face of the big AI folks or the big skeptics.
    spoken · machine transcript hear it at 0:39:28 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 17th of 26 in this recording

  18. It's like we need to hear the wins. We need to hear what's We need to hear about what's possible. We need to hear what's exciting. But you got to couple it with the costs.
    spoken · machine transcript hear it at 0:42:30 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 18th of 26 in this recording

  19. Software is made of logic and language. AI is made of logic and language. And because of that, we can bake in guardrails.
    spoken · machine transcript hear it at 0:44:49 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 19th of 26 in this recording

  20. And I have decided not to sink anymore because writing is thinking on paper. And there's no shortcut for doing that thinking.
    spoken · machine transcript hear it at 0:47:20 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 20th of 26 in this recording

  21. That was always a bad idea because it's split brain. Half of you are writing the code and the other half are understanding it. I would argue that you can't really understand the code you write unless you're operating it.
    spoken · machine transcript hear it at 0:50:27 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 21st of 26 in this recording

  22. But I also think that in my mind, 20 years of DevOps was really about one thing. Trying to create one feedback loop that connected people writing code to that code in production. And it failed.
    spoken · machine transcript hear it at 0:51:07 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 22nd of 26 in this recording

  23. Auto instrumentation has gotten so good in recent years. If you're using Open Telemetry and everyone should be using Open Telemetry. All of the common patterns like all of the models are trained on them. So, it is literally faster and easier to build with instrumentation than to than not to.
    spoken · machine transcript hear it at 0:56:00 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 23rd of 26 in this recording

  24. If we don't do it ourselves, meaning hold ourselves to a high standard, build with efficiency, constantly be like trying to get better, if we don't do it ourselves, someone will come and do it to us.
    spoken · machine transcript hear it at 1:09:15 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 24th of 26 in this recording

  25. it's it's not enough to be a good person. I believe that people who are kind and care about people can and and usually do do better than sociopaths in the same roles, but only if they're good at business.
    spoken · machine transcript hear it at 1:10:10 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 25th of 26 in this recording

  26. The hardest thing about quantifying the value of junior engineers is that we don't know how to quantify the value of any engineer. So it's all vibes.
    spoken · machine transcript hear it at 1:16:34 · all korrents from this recording

    Stop being skeptical about AI for development with Charity Majorsyoutube.com 26th of 26 in this recording

    AI and jobs

  27. 20 months earlier
  28. Earlier this year I started writing a piece on why “hire great people and get out of their way” is such terrible, dangerous, counterproductive advice to give anyone in a leadership role.

    “Founder Mode” and the Art of Mythmakingcharity.wtf 1st of 8 in this piece

  29. You should ALWAYS have as few employees as possible. Always. Hiring more people should never be the first lever you reach for, it’s what you do after exhausting your other options.

    “Founder Mode” and the Art of Mythmakingcharity.wtf 2nd of 8 in this piece

    hiring

  30. Either can work. Both have tradeoffs and implications. If you try to import either philosophy wholesale, it will break in unexpected ways; if you try to mix and match, it will probably be an unfettered nightmare.

    “Founder Mode” and the Art of Mythmakingcharity.wtf 3rd of 8 in this piece

  31. Can you be calibrated as an interviewer on every single opening, for every role? My God, no, not even close.

    “Founder Mode” and the Art of Mythmakingcharity.wtf 4th of 8 in this piece

    hiring

  32. I’ve said many times that if I had to choose between interviews or references, I would pick references every time.

    “Founder Mode” and the Art of Mythmakingcharity.wtf 5th of 8 in this piece

  33. you want to hire people for their unique strengths, not their lack of weaknesses. If they’re strong where you need them to be strong, it’s okay if they aren’t equally superpowered at everything — that’s why we build teams, to supplement and balance each other out.

    “Founder Mode” and the Art of Mythmakingcharity.wtf 7th of 8 in this piece

  34. Here’s one small mental hack that makes a world of difference: remember that you are trying to hire the right people to join your team/org/company. Not the “best” people. The right people.

    “Founder Mode” and the Art of Mythmakingcharity.wtf 8th of 8 in this piece

  35. 11 months earlier
  36. But hierarchy is not intrinsically authoritarian. Hierarchy did not originate as a political structure that humans invented for controlling and dominating one another, it is in fact a property of self-organizing systems, and it emerges for the benefit of the subsystems.

    Questionable Advice: "My boss says we don’t need any engineering managers. Is he right?"charity.wtf 1st of 5 in this piece

  37. If you aren’t making building the organization someone’s number one job, it won’t be anyone’s number one job, which means it probably won’t get done very well.

    Questionable Advice: "My boss says we don’t need any engineering managers. Is he right?"charity.wtf 2nd of 5 in this piece

  38. On the other hand, the manager-free experiments I’m aware of (e.g. holacracy at Medium and GitHub, or “Choose Your Own Work” at Linden Lab) have all been quietly abandoned or outgrown.

    Questionable Advice: "My boss says we don’t need any engineering managers. Is he right?"charity.wtf 3rd of 5 in this piece

  39. But everything I have ever experienced leads me to believe that a fewer number of larger teams, each helmed by an experienced engineering manager, should way outperform this gaggle of tiny groups.

    Questionable Advice: "My boss says we don’t need any engineering managers. Is he right?"charity.wtf 4th of 5 in this piece

    management

  40. What matters is moving the business forward, not churning out code.

    Questionable Advice: "My boss says we don’t need any engineering managers. Is he right?"charity.wtf 5th of 5 in this piece

  41. 15 months earlier
  42. The “Big Lie” of hierarchy is that your organizational structure is a vertical tree from the CEO on down, where higher up is always better.

    The Hierarchy Is Bullshit (And Bad For Business)charity.wtf 1st of 3 in this piece

  43. The times I have come closest to burnout or flaming out have never been when I was working the hardest, but when I cared the least.

    The Hierarchy Is Bullshit (And Bad For Business)charity.wtf 2nd of 3 in this piece

  44. But if there’s one thing we know, it’s that for industries that are fueled by creativity and innovation, command-and-control leadership is poison.

    The Hierarchy Is Bullshit (And Bad For Business)charity.wtf 3rd of 3 in this piece

  45. 3 months earlier
  46. Being on call should not be a constant cycle of things breaking down and firefighting, or alerts going off at all hours. This is not ‘normal.’ These are telltale signs of a fragile system and lack of alert discipline.

    Why On-Call Pain Is A Sociotechnical Problemcharity.wtf 1st of 5 in this piece

  47. It is engineering’s job to own their code in production. It is management’s job to make sure it doesn’t suck.

    Why On-Call Pain Is A Sociotechnical Problemcharity.wtf 2nd of 5 in this piece

  48. It’s reasonable to be woken up two to three times a year when on call. But more than that is not okay.

    Why On-Call Pain Is A Sociotechnical Problemcharity.wtf 3rd of 5 in this piece

  49. The ideal on-call rotation has seven to eight people; five people is a bare minimum.

    Why On-Call Pain Is A Sociotechnical Problemcharity.wtf 4th of 5 in this piece

  50. I generally come down on the side of ‘no, it’s part of the job,’ just like it is for doctors.

    Why On-Call Pain Is A Sociotechnical Problemcharity.wtf 5th of 5 in this piece

  51. 5 years earlier
  52. Fuck the whole idea that only managers get career progression.

    The Engineer/Manager Pendulumcharity.wtf 1st of 4 in this piece

    management

  53. The best frontline eng managers in the world are the ones that are never more than 2-3 years removed from hands-on work, full time down in the trenches. The best individual contributors are the ones who have done time in management.

    The Engineer/Manager Pendulumcharity.wtf 2nd of 4 in this piece

    management

  54. You can only really improve at one of these things at a time: engineering or management.

    The Engineer/Manager Pendulumcharity.wtf 3rd of 4 in this piece

  55. Management is highly interruptive, and great engineering — where you’re learning things — requires blocking out interruptions. You can’t do these two opposite things at once.

    The Engineer/Manager Pendulumcharity.wtf 4th of 4 in this piece