Prompted partially by some comments yesterday on my post on How to Make Late Software, Even Later, and partially by a discussion on the altdotnet Yahoo list, I wrote this long email. As it became an epic in its own right, I thought it deserved a blog.
We Are Doing Agile!
I can claim to be doing Agile, and following with some degree of conformance, a known and understood Agile methodology. If my project fails, we must blame the Agile process, and our practices in some way for this
I can claim to be doing Agile, but make up my own variation of a known methodology, in which case a failure of the project is largely encumbant upon me for deciding to sacrifice the practices we chose to ignore, and the new ones we added.
I can claim to be doing Agile while just coding hell for leather and following no methodology, just cherry picking a couple of practices here and there, and picking the principles I like - in which case any failure is mine to bear.
All of these approaches should be highlighted in a risk analysis to management to make the decisions *before* you decide to pursue any of these paths. It is the decision of management to weigh risk vs reward and to make that decision. If I didn't write that report, or I wrote it badly, then again the fault is mine to bear. If I wrote it clearly expressing the dangers of a hack and see approach, and management chose that path, then it is their cross to bear.
You Cannot Blame A Process For Producing Bad Results
If all participants in a waterfall methodology like RUP followed the practice rigourously, then I have little doubt the project would have a high chance of success (as defined by the process, a close match to the intial requirements)
If all participants in an Agile process like XP followed the practice rigourously, then I have little doubt the project would have a high chance of success (as defined by by the process, a close match to the evolving requirements)
Bad people on any project can stuff any methodology, and bad application of good practices will always result in failure. Agile just gives you much earlier feedback that you are not getting it right - how you choose to interpret that feedback, or how management choose to, is the fault of those communicating or interpreting the information - not of the process.
How To Fail Under Agile
The worst, and I mean absolute worst project failure I ever worked on was done using Agile, using Extreme Programming, and the practices were fairly closely followed. It should have been a success. And for a few small but critical mistakes it would have been.
The first big mistake was not evolving the architecture first. The application had been developed under XP for over 9 months when I was brought in the provide the "Enterprise component" of the application. It became rapidly apparent that the application had been written as a desktop applciation, and was too tightly coupled to allow any extension points for an evolution of the functionality into a different UI and service layer.
The second big mistake was a large amount of code that had been written to allow the client applciation to operate on domain objects, using an ORM, persisted across a web service layer, relying on massive amounts of code generation. This was a technical mistake that should have been picked up by prototyping and getting the architecture right first.
The third big mistake was a very poor application of the XP process. The bug backlog for each week was continual, and continally increasing. It was seen as acceptable that stuff didn;t work, and in each stand up it was just refered to as a bug and would go on the backlog. Nobody ever stopped to take a grip of this, and realise that this was a warning sign. As complexity was increaing, the number of interactions in the software was going up rapidly, and people spent more time bug fixing. Out of around 5000 unit tests (afair) that existed when I got there, around 2000+ were failing - this was explained as changes to the code and hence they couldn't get them working. This invalidated pretty much the entire test suite.
These were not problems with the process (XP), but clear problems with no up front design or prototyping (point one), no technical understanding of the challenges involved (point two), and people badly applying the process (point three).
Was Agile or Extreme Programming to Blame?
Yeah, we know loads of it is in flux, and it is a bit 'out there' but this is Agile man, we live by the seat of our pants, we ride fast and hard, this is EXTREME!!!!!
In the respect that it allowed developers to go to senior management and say this, hell yes Agile and XP were to blame. It gave the excuse to developers that they could excuse stuff being chaotic, as being all part of the way this stuff works.
The reality was the project failed due to a manager that for all his ability, had never done Agile or XP before, and had taken to quoting Martin Fowler PoEAA so often when we tried to raise problems, that his copy was starting to fall apart. And the business owner took the Agile/XP no requirements thing to the Nth degree and would change his mind on features on a daily basis, so the team could never get a handle on the problem. And lastly the team, that had gone into a terminal march towards inevitable failure, but had never spoken up loudly enough.
Oddly enough, that was the single team I have worked on that had by far the highest technical ability of any team as a whole I have ever worked with. They had some absolutely superb developers - but sometimes even that isn't enough.
05-09-2008 8:15 AM