So, I've now come full circle, back to software engineering, after detouring through studying rationalism in economics, politics, philosophy and urban planning. And I have realized that long ago I had recognized the rationalist mistake in my own field of software engineering. It was present in the words of the critics of UNIX for not being designed according to some grand, theoretical blueprint, and instead being "hacked" together to fit the needs of the Bell Labs researchers. But even more so, it was present in the waterfall modelers and software managers requiring "complete specifications" before any coding starts.
One goal of this effort was to be able to hire really dumb programmers, whom one could pay very little. As T.S. Eliot might have put it, "They were dreaming of systems so perfect that no one had to be intelligent." But the dream is generally impossible to achieve: it was like the Soviet Union's five-year plans that would envision all economic contingencies in advance, just that waterfall development was on a smaller scale. (Note: if the scale is small enough, or if one gets really lucky, or, most especially, if there is really much greater flexibility down the road than the method strictly allows, waterfall development can occasionally work. And that gave the method its plausibility.) The massive specification documents produced rapidly became worthless, because business conditions continually change, and people building and using software only learn what they really should be building or what they really want in the process of the building. (This is very similar to how market participants only discover what consumers really want through the market process itself.)
As Dwight Eisenhower said about battle plans:
"I tell this story to illustrate the truth of the statement I heard long ago in the Army: Plans are worthless, but planning is everything. There is a very great distinction because when you are planning for an emergency you must start with this one thing: the very definition of 'emergency' is that it is unexpected, therefore it is not going to happen the way you are planning."
In other words, when we adopt Lean, Agile, and DevOps, we don't stop thinking about what we are going to build: we keep planning. We simply don't become attached to our plans: we recognize that we should be engaged in continuous learning at every point in any software project. And that learning might entail modifying almost any aspect of the project, right up to the highest level specification document. After all, if our specification document assumed "the enemy" would be in front of our forces, but we suddenly find that the enemy is behind us, it would be sheer madness to continue on with our originally specified plan as if nothing had changed.
The vast and growing suite of DevOps tools exists first and foremost to facilitate this way of working. And we should always keep that in mind as we implement various technical practices.