Top voices on software quality: Kent Beck, Martin Fowler, Simon Willison, Kent C. Dodds, Hillel Wayne and 11 others.
They agree tests buy confidence and split on which tests, how many, and whether agents change the answer.
Our reading of what a group of claims have in common, one level above a korrent: a korrent is our reading of what somebody said, a view is our reading of what several korrents share. The claims are printed under it, so the sentence can be checked against them.
Nobody is filed under a view — claims are. A person is named here because a dated claim of theirs sits under it, and somebody whose claims sit under two views appears under both.
Salvatore SanfilippoMarc LittlemoreThorsten BallMartin Fowler
A viewour reading
Their wordsunit tests are just source code at this point e2e test is the only real kind of test we can trust to guard actual behavior
Their wordsunless the code is completely out of your control, the need for mocking often indicates a design problem-consider refactoring to make the code easier to test directly.
Their wordsDid you know, the tests you write with react-testing-library look almost identical to the tests you write with vue-testing-library. Magic things happen when you don't test implementation details. :)
Their wordsThe more your tests resemble the way your software is used, the more confidence they can give you.
Their wordsBut unit tests don't give you much confidence in your code.
A viewour reading
Mental modelThe map is not the territory
Their wordsIt is a known fact that covering all the lines of the code does not mean covering all the possible states.
Their wordsThe goal is to write tests that are easy to understand and maintain, not to test everything.
Their wordsA lot of complex, "look at this clever trick", overly-abstracted, unreadable code works and makes the tests pass.
Their wordsTest coverage is of little use as a numeric statement of how good your tests are.
Mental modelWhen a measure becomes a target
Their wordsI would be suspicious of anything like 100% - it would smell of someone writing tests to make the coverage numbers happy, but not thinking about what they are doing.
A viewour reading
Their wordsAnd then there were people who used it as a moral cudgel. Like you should be if you're not using TDD, you're not professional. And that's just such People can write very good software with a wide variety of workflows.
Mental modelFeedback loops
Their wordsIt's when you have this rapid alternation between I do a thing, I learn a thing, I do a thing, I learn a thing, I do a thing, I learn a thing. That's where TDD is really powerful. But it's not a moral decision. It's a practical decision.
Mental modelFeedback loops
Their wordsThen you had code review which gave you another level of feedback. You could roll out internally more frequently. And everybody was using Facebook for all kinds of stuff, personal and internal business stuff. So whatever feature you developed, people would start using it immediately. So you get another round of feedback. Then we had this phased roll out process where you'd start rolling your stuff out. If there was a problem, the blast radius would be limited to a a few million people.
Their wordsUncle Bob gives terrible advice. Following it will make your code worse.
Their wordsThere are too many private functions, writing unit tests for these is a waste of time as they are both simple and using assertions.
A viewour reading
Mental modelSecond-order thinking
Their wordsIt costs tokens in the short run, but it can save them in the long run because one of the classic genie mistakes is stuff doesn't work.
Their words
Their wordsthe key thing about TDD is that it means that the agents won't write more than they need to.
Their wordsI think I see people who are writing code with coding agents and they're not writing any tests at all. That's a terrible idea.
Their wordsAnd what you can do if you've got one of these conformance suites is you can give it to a a good agent and say, "Write code until this test suite passes." And it kind of will.
A viewour reading
Their wordsModern models just love writing tests for every tiny change, even if they aren't useful.
Mental modelFeedback loops
Their wordsthe problem with training models on maintainability is like the cost function of bad architecture and bad program design can't be evaluated by running the unit test because it hits you 3 to 6 months later
Mental modelMargin of safety
Their wordsyou can have an agent make code for you and it could work and it could test, but is it secure? Is it maintainable? If there's something going wrong, can you roll things back and understand what's going on?
Their wordsWorse, you realize that the gazillions of unit, snapshot, and e2e tests you had your clankers write are equally untrustworthy. The only thing that's still a reliable measure of "does this work" is manually testing the product.
Their wordsBigger changes always get tests. Automated ones usually aren’t great, but the model almost always finds issues when you ask it to write tests IN THE SAME CONTEXT. Context is precious, don’t waste it.
Asserted relations, each with the curator’s reason — never computed, because similarity cannot tell agreement from opposition.
These cannot both be true
Steinberger has the model write tests in the same context as the change because that is what finds the bugs. Zechner says the tests an agent writes for its own code are as untrustworthy as the code, and that manually using the product is the only reliable measure left. They cannot both be right about whether a model can check its own work.
Matched by wording, not yet read into a view. The archive grows on its own, so this is where new claims wait for somebody to read them.
The full chronology, with every quote: software quality as a timeline · person by person, side by side · all topics · Something wrong? Tell us