Published in · 6 min read · May 12
How about “software development”?
There’s a growing consensus among people who’ve been involved with Agile Software Development since the early days that something is rotten in the state of Agile.
Having slowly backed out of the Agile movement over the last decade or more, I approach the movement as a fairly skeptical observer.
Talking with folk both inside and outside the Agile movement — and many with one foot in and one foot out — has highlighted for me where the wheels came off, so to speak. And it’s a story that’s by no means unique to Agile Software Development. Like all good ideas in software, it’s never long before the money starts taking an interest and the pure ideas that it was founded on get corrupted.
But, arguably, Agile Software Development was fundamentally flawed straight out of the gate (or straight out of the ski resort, more accurately). If I look for a foundation for Agile, it clearly has its roots in the concept of evolutionary software development. Evolution is a goal-seeking algorithm that searches for an optimum solution by iterating designs rapidly — the more rapidly the better — and feeding back on what we learn with each iteration to improve our solution.
There are two keywords in that description: iterating and goal-seeking. There is no mention of goals in the original Agile Manifesto. The manifesto stipulates that the measure of progress is “working software”. It does not address the question of why we should build that software in the first place.
And so, many Agile teams — back in the days when Extreme Programming was still a thing — focused on iterating software designs to solve poorly defined — or not defined at all, let’s face it — business problems. This is pretty much guaranteed to fail. But, bless our little cotton socks, because we set ourselves the goal of delivering “working software”, we tended to walk away thinking we’d succeeded. Our customers… not so much.
This was the crack in Agile through which the project office snuck back in.
As Agile evolved as a brand, more and more of us tried to paint ourselves in the colors of management consultants. Because let’s be frank, that’s where the big bucks are. People who would once have been helping you to fix your build script were now suddenly self-professed McKinsey-style business gurus telling you how to “maximize the flow of value” in your enterprise, often to comic effect because nobody outside of the IT department took us seriously.
And then, one day — to everyone’s horror — somebody outside the IT department did start taking us seriously, and suddenly it wasn’t funny anymore. Agile “crossed the chasm”, and now people were talking about “going Agile” in the boardroom. Management and business magazines now routinely run articles about Agile, typically seeking input from people I’ve certainly never heard of who are now apparently world-leading experts. None of these people has heard of Kent Beck or Ward Cunningham or Brian Marick or any other signatory of the original Agile Manifesto. Agile today is very much in the hands of the McKinseys of this world. A classic “be careful what you wish for” moment for those from the IT department who aspired to be dining at the top table of consulting.
Agile’s now Big Business. And the business of Agile is going BIG. Like every good and pure thing that falls into the hands of management consultants, Agile has mutated from a small, beautiful bird singing a twinkly tune to a bloated enterprise albatross with a foghorn.
I’m often found hanging around on street corners muttering to myself incoherently about the leadership class. Well, it’s good to have a hobby.
Across the world — and especially in the UK — we have a class of people who have no actual practical skills or specific expertise to speak of, but a compelling sense of entitlement that they should be in charge, often of things they barely understand.
In the pre-Agile Manifesto world, IT was ruled by the leadership class. There was a huge emphasis on processes, driven by the creation of documents, for the benefit of people who were neither using the software nor writing it. This was a non-programmer’s idea of what programming should be. In the late 1990s, the project office was the Alpha and the Omega of software and systems development. People who’d never written a line of code in their lives telling people who do it day-in and day-out how it should be done.
Because, if they let programmers make the decisions, they’ll do it wrong!!! And, to be fair, we often did do it wrong. We built the wrong thing, and we built it wrong. It was our fault. We let the project office in by frequently disappointing our customers. But their solution just meant that we still did it wrong, only now we did it wrong on a much grander scale.
And just as we developers kidded ourselves that, because we delivered working software, that meant we had succeeded, managers deluded themselves that — because the team followed the prescribed processes — the customer’s needs had been met.
Well, nope. We ticked the boxes while the customer got ticked off.
It turns out that the working relationship between software developers and their customers is, and always has been, the crux of the problem. Teams that work closely and communicate effectively with customers tend to build the right thing, at least. There’s no process, standard or boxes-and-arrows diagram that can fix a dysfunctional developer-customer relationship. CMMi all you like. It doesn’t help in the end. And, as someone who specialized in software process engineering and wore the robes and pointy hats of a Chief Architect, I would know.
The Agile Manifesto was a reaction to the Big Process top-heavy approach that had failed us so badly in the previous decades. Self-organizing teams should work directly with customers and do the simplest things to deliver value. Why write a big requirements specification when we can have a face-to-face conversation with the customer? Why create a 200-page architecture document when developers can just gather around a whiteboard when they need to talk about design?
XP in particular seemed to be a welcome death knell for value-sucking Plan-Driven, Big Architecture, Big Process roles. It was the end for those projects like the one where I was the only developer but for some reason reported to three project managers, spending a full day every week traveling the country helping them to revise their constantly out-of-date Gantt charts.
And, for a while, it was working. The early noughties were a Golden Age for me of working on small teams, communicating directly with customers, making the technical decisions that needed to be made, and doing it our way.
But the project office wasn’t going to just slink away and die in a corner. People with power rarely relinquish it voluntarily. And they have the power to make sure they don’t need to.
Just as before, we let them back in by disappointing our customers. A lack of focus on end business goals — real customer needs — and too much focus initially on the mechanics of delivering working software created the opportunity for people who don’t write code to proclaim “Look, the people writing the code are doing Agile wrong!”
And, again, their solution is more processes, more management, more control. And, hey presto, our 6-person XP projects transformed into beautiful multi-team Enterprise Agile butterflies. Money. That’s what I want.
Agile today is completely dominated by management. It’s no longer about software development, or about helping customers achieve real goals. It’s just as top-heavy, process-oriented, and box-ticky as it ever was in the 1990s. And it’s therefore not for me.
Working closely with customers to solve real problems by rapidly iterating working software on small self-organizing teams very much is, still. But I fear the word for that has had its meaning so deeply corrupted that I need to start calling it something else.
How about “software development”?