Agile v Waterfall - Why are we still fighting about it?

Over the past few years, I've worked with many clients who were implementing Agile like approaches to delivering business initiatives. Unsurprisingly, in a number of cases, this led to disagreements between Agile supporters and those in favour of more "traditional" project management practices.

So why are we still fighting about it? Is it because Agile is still perceived as a new/unproven way of doing things? (Actually Agile  has been in use for over 20 years!)
Is it because "traditional" project managers feel threatened by Agile? 
In reality, both approaches (and everything in between) have their place - it's all a question of choosing which one is right for the circumstances. 
Agile is a proven way of doing things ...sometimes
 When you look at the statistics, Agile stacks up pretty well in terms of delivery success compared with more traditional approaches. In this data from the Standish Group, we can see that Agile significantly outperforms Waterfall on the software development projects in the sample data.
Size really does matter...
However, based on the same dataset, we can also see that whilst Agile outperforms Waterfall on large projects, the actual success rate for large projects is pretty dismal!
And the Agile advantage seems to disappear when we look at smaller projects. So what's going on here?
Predictability is everything
Large projects fail because it's difficult to see far enough into the future to predict all the variables - changes to business strategy, changes to the external market, changes to technology and everything in between. But we've known this for years - certainly pre-Agile.

Many years ago, I would routinely work on projects that had a 5 year delivery schedule, and these projects would be based on a business case that tried to forecast benefits way out on the horizon. And of course, the companies I worked for back then knew that a 5 year knew business case wasn't viable, so we started delivering programmes in tranches, each with a business case limited to a 12 month payback period. So breaking big projects down into smaller ones to ensure they stay aligned with the business direction isn't that radical - and smaller projects have greater success rates whichever approach you use - right?
Horses for courses
The real challenge, I think, is deciding which delivery approach is best for the circumstances. Whilst many organisations position themselves as Agile forward, this does not mean that all their projects can, or should be delivered using Agile. In fact, for certain type of projects, Agile is a bigger risk than a traditional approach. 
So how do you choose to use Agile or more traditional methods? How can you help your organisation or client make the right decision to guarantee delivery? 
  • If your organisation is very hierarchical and regulated, its going to be harder to be “Agile” than it is in a relatively small start up
  • If you are running a large regulatory programme, there maybe less opportunity to build iteratively based on mandatory compliance deadlines
  • If you are bringing new products to market in a highly competitive environment, then Agile is likely your best approach
Here are some questions you can ask to determine what's best in a given situation.
Culture
The culture of the organisation plays a big part in deciding which approach is best. Organisations which have a collaborative and experimental culture have a far better chance of succeeding with Agile approaches. At the same time, organisations with a strict decision hierarchy will struggle to adapt to the Agile model of self managing teams and rapid decision making. 
Clarity
The next decision making factor is clarity of requirements. If detailed requirements and desired outcomes can be clearly defined, then an iterative approach may not be necessary taking the other factors into account. Ditto estimating the effort to deliver the project - if this can be done with sufficient certainty, then a waterfall approach may be best.
Customer
Access to the customer is a critical requirement for Agile like approaches, as they too are involved in product iterations. Consider factors like customer availability (input to design, decision making), the stability of the customer environment (you will need to deliver at a faster pace if the client environment changes frequently) and whether the customer has a fixed deadline or not.  
Capability
Our final factor is choosing the right delivery approach is capability. Think about the success rates of previous projects - did Agile work? Was Waterfall a success? Does the organisation have the right skills and knowledge for the delivery approach being considered? If an organisation is not particularly skilled with Agile, maybe a smaller, lower risk project is a better place to start than a strategically critical initiative. And, particularly with Agile, we need to consider whether the rest of the organisation is set up to support Agile approaches, for example changes to the way projects are funded, and the availability of key people to take on Product Owner roles. 
Created with