I love Will Larson’s texts. Make No Assumptions is another example of such a text:

It’s true that poor reasoning has always existed, long before harnesses, but my experience is that poor reasoning wearing well-formatted clothing is proliferating more widely than I’ve previously seen, and it is increasingly difficult to combat because certain social norms are – at least temporarily – collapsing around folks actually thinking. That collapse is largely driven by unprincipled adoption of AI techniques without paying attention to whether they work.

In software, the part “whether they work” can be vague. Coding agents can confirm a piece of software works by various means. Automated testing is one of them. They can work in a loop until the software covers everything written in the tests. Of course for that loop to succeed, one needs to provide the tests, and that means one needs to reason how the software should behave. Humans and agents alike can miss edge cases, of course, but reviewing what’s written can solve this problem to a degree.

There is also another part to this equation: “How they work”. Coding agents can provide different ways to solve a particular problem. I saw more than one occassion where the author accepted the work the agent produced and showed the automated tests as proof that it works. I wouldn’t say these were “blind acceptance”. These authors review the agent’s code before deciding to ship them. But when I ask about a certain part of the code they’d like to ship, they would go and ask another coding agent to explain what exactly it does. They initially had assumed that it would work in the way they expected, but they did not understand it. My question helped them question the reasoning behind it. In the era of coding agents, questioning “how” a certain piece of software works should be the default mode.

Perhaps with everything artificial intelligence related these days, another quote from Dune (Chapterhouse) fits well here:

Ready comprehension is often a knee-jerk response and the most dangerous form of understanding. It blinks an opaque screen over your ability to learn. The judgmental precedents of law function that way, littering your path with dead ends. Be warned. Understand nothing. All comprehension is temporary.