Now that the Agile movement is making it into the mainstream, and gaining popularity, it is facing a backlash. Aside from the mainstream pro-Agile trend we hear so much about, I see three reactive trends: rejection by non-practitioners, a mandated reversal in corporate methodology focus, and an adaptive reaction that builds on good software development ideas and practices.
A Source of Division
Sadly, the Agile movement is creating a lot of division in the software development world. A stark division is drawn between Agile practitioners, and those who aren’t “Agile”, therefore to the Agilists, they are “waterfall”, and by default, inferior. This attitude ranges from the relatively harmless, yet tasteless “Waterfall 2006” conference parody set up by well-known Agilists, to companies being torn apart from within as factions rage against each other, grappling for methodology dominance.
I get emails from people all over the world sharing with me the trouble they are now facing. Instead of collaborative, enabling Agilists trying to help a team do better, they are faced with zealous naysayers who try to force process change. One correspondent mentioned that they were fearful of the Agile process evangelists in the company, and other staff were avoiding them. They were tired of being berated for not knowing the Agile lingo, and being belittled as “waterfall”, and “backward” when they would ask innocent questions. The Agile zealots in their company had taken to yelling at the other team members if they did something that was “un-Agile”. They expressed that the team was reasonably cohesive and productive in the past, but now that a couple of team members had bought into the Agile movement, they were disruptive. Obviously, this is behavior that is completely against the values of any Agile methodology I’ve come across. Sadly, the divisive, belittling behavior of some Agile zealots is painting the entire community that way. Some managers are dealing with this situation by removing the people that are disruptive, and fearfully avoiding the Agile processes as a result.
Martin Fowler has aptly described this kind of behavior as “Agile Imposition“. For the reverse of what I described above, read Fowler’s post where he describes “Agile” being imposed on a team by management whether they want it or not.
Another divisive issue is economically-based. It’s important to realize that there are people and companies that are now making a lot of money from Agile processes. Some have told me that they are in danger of losing a customer, because the customer is suddenly concerned that they aren’t “Agile”. The customer is happy with the product or service, but a potential competitor has come along, is using “Agile” as a differentiator, and is trying to use it as a wedge to try to steal away a customer. Some companies react by branding themselves as “Agile”, without altering much of what they are doing. A little lingo adjustment, and some marketing makes short work of the differentiator their potential competitor is trying to use. Unfortunately, this waters down what Agility is supposed to be about. In some communities, you would be hard-pressed to find a company that deals in software development who doesn’t call themselves “Agile”, when few of them are actually using a methodology like XP, Scrum, Crystal, Evo, Feature Driven Development, etc. As people who call themselves “Agile” create messes and some project disasters become known through the local community, the shine certainly wears off.
Other companies react in a more healthy manner. What is this “Agile” fuss? Is it something we need to look at? One person told me that after reading my recent writing that displays some skepticism of Agile methodologies, they realized why they do what they do. They do it because it works. They don’t see a need to change, but are finding the challenge by a potential competitor as a way to make their client relationship stronger. They are looking at their processes, explaining them and making them defensible. Focusing on “what works” is at the heart of many of the Agile methods anyway. This is a healthy way to deal with division: look at what works for your company, and make it defensible. Also, look at areas that can be improved, and look at the Agile movement as only one potential source for innovative ideas.
Methodology Reversals
A troubling trend I have witnessed is a mandated return to old ways of doing things in the face of Agile failures. The story goes something like this: Company A is having process and quality problems. With the support of executives, they adopt an Agile methodology, and implement it with great care. Agile coaches are used, experienced Agilists train the team, and they work to improve their process adoption. In short order, the results are phenomenal. Quality improves, team cohesion improves, and the executives see a happy, healthy, productive team that is helping them meet their objectives. Over time though, productivity drops. Quality once again begins to suffer. The team turns to coaches, training, and work harder to implement the process to the letter. The results aren’t there anymore though. They forget to adapt and improve to changing conditions, and they just stick to the process, and try to follow the formula that worked in the past. As they get more desperate, they make wild claims promising that “Agile” will help them unseat a powerful competitor who is “waterfall”, or will guarantee bug free software, improve all employee morale problems, etc. The wild claims are not met, and executives get nervous. “What else are these Agile evangelists wrong about?”
At least with the old way of doing things, they could exert more control. After all, they had a big paper trail, and isn’t that what the big successful companies do? “We need to stage delivery of each phase of software development and get buy-in and sign-off like the old days. This “Agile” thing was a nice experiment, but it isn’t working. Time to call the big process consultants, and let’s get everyone into their own offices, hire a requirements expert, and go back to what we are used to.” This is not necessarily a healthy way of dealing with the issue. Why not build on what worked, and adapt it? Sadly, this is an Agile implementation failing by people who knew what they were doing. Moving back to the way things were failing before will not necessarily help.
It shouldn’t surprise us that processes stop working over time – stores are full of books to deal with boredom over basic needs such as diet and intimacy. “Spice up your diet!” deals with the problem of yesterday’s favorite dinner becoming today’s tasteless same-old, same-old. If we tire and find repetition difficult to sustain with basic needs, why do we expect our software processes to be necessarily good for all time? This failure to adapt, and blind devotion to a formula causes “Agile” to get a bad name.
Company B on the other hand hears a lot of Agile hype. Every other company in town says they are Agile, and we think we are Agile too. We don’t have much documentation, and we do a release twice a year. It doesn’t get more Agile than that! Someone with some sense attends a user group meeting or a conference, buys a book or two, and starts educating the company on what “Agile” really is. Management isn’t completely comfortable with “Agile”, so they only allow a partial implementation of a methodology. “We’re doing what works for us.” is the common phrase. In some cases this is true, especially as a company iterates towards an Agile implementation, or improves after adoption. In many cases that can be translated into: “We can’t completely adopt an Agile methodology, so we’re going to do what we can, and call it Agile anyway.” As soon as Company B has problems, “Agile” is blamed, and management reverts to the official status quo. “We tried Agile, and it didn’t work.” Again, even though they didn’t implement it properly, “Agile” gets a bad name.
In some extreme cases, association with a particularly corrosive project failure can make it difficult for people to find work in their local community. If project X was a big enough failure to reach the local rumour mill, and project X was also a flagship “Agile” project, some Agile practitioners may find they have to drop the “Agile” label in their community, or they find they are equated with the charlatans.
Innovations
I’ve noticed some small trends that build on Agile methods and are moving forward. One is that most of the early adopters of Agile methods that I know are quietly expressing skepticism over the effectiveness of the processes and are looking for “Agile 2.0”, or the next phase of the Agile movement. Others have left the Agile movement and have moved on. At any rate, these are skilled thinkers who see that Agile processes do not address all the needs of a company. In fact, they see that in some cases Agile methods are inappropriate and ineffective. Others found that they worked for a while, but they needed to drop a methodology and adapt something new to be able to sustain the productivity and quality they required of themselves.
Many have found the Agile methods are too narrow and rigid for them, and don’t fit the bill for the situations they face. Some teams have dealt with this by mastering a process, finding where it didn’t work, and improving on it. Sadly, they face criticism from Agilists who may feel their source of revenue or process religion threatened. These innovators who constantly build on what they know and improve are the people that give me hope, and this trend is something I have labelled “post-agilism“. (Jason Gorman (and probably others) was also thinking about this. Check out his: “Post-Agilism: Beyond the Shock of the New“.)
Another trend I’ve seen is a correlation between companies with stellar process implementations who were financially unstable, and companies with disastrous development shops who had huge sales and were seen as successes in the business community. There are exceptions of course, but this is easily misinterpreted by those who equate a correlation with cause. Gary Pollice has an excellent article: “Does Process Matter?” where he explains how there are many more dimensions to a successful company than their software development methodology. Effective sales, marketing, timing in the market, and having something people want to buy in the first place have much greater bearing on the financial success of a company than the software development methodology that is being used.
The damage to the Agile reputation can occur with thinking like this: “Company X claims to be Agile. Our sales for a month are higher than theirs are for a year, so why should we adopt something that might hurt that? This Agile thing sounds like something to stay away from.” And with that, the ideas are dismissed. The thinking innovators see it differently.
The thinkers realize that the marketing hype of the Agile methods does not address a great deal of what makes a company successful. In fact, they realize that no software development process will make a company financially successful on its own. So they begin to look towards the big picture, and strive to align themselves with the rest of the company. The collaboration skills they have honed while executing some Agile practices can come into play as they move beyond the short-sighted picture of software development processes.
Note: Some people didn’t need the Agile movement to learn these skills. The Agile movement did not invent, and does not have sole ownership of lightweight, iterative processes, frequent communication and strong customer involvement in software projects.
Another troubling trend is the admission that some of the claims of success by Agilists are suspect. Ron Jeffries graciously posted: Misstating the Evidence for Agile and Iterative Development: an accusation from Isaac Gouy as one example. If we subtract the Agile marketing engine hype, discount the books, papers, talks, workshops and conferences that are fueling financial gain for those associated with the Agile movement, what is left? Is all of this Agile hype just that? Hype? Have we been duped by slick-talking process gurus who are giggling maniacally as they drive their sports cars to their banks to cash in their “Agile”-related spoils? This is certainly an impression that is growing with some who have tried Agile processes and found that they just did not work in their unique contexts.
What about the glowing success stories? How do we account for the legions of Agile fans, some of whom display a religious cult-like devotion to Agile methods? Some even appear to be intelligent people. Were they swayed by a leader who with a shakti-pat, or healing, blessing or word of knowledge, convinced the hardened skeptic? I don’t subscribe to conspiracy theories, and I believe that in certain contexts, Agile processes have enabled teams to do extraordinary things. However, measuring the effectiveness of software processes is extremely difficult. Only changing a process, and measuring the result does not take many factors into account, particularly when dealing with a group of humans. In fact, as phenomena like the Hawthorne Effect show us, the measurement itself may impact the changes we see. Rick Brenner calls these Observer Effects.
We may wrongly attribute the success of a software development process when something else may have a stronger impact. One team aspect I didn’t pay enough attention to, and possibly misattributed early Agile successes I observed, is skill. I’ve realized that any skilled team can make a process work for them and do good work, and any team lacking in skill is going to fail no matter what process they are using. Not exactly rocket science, but it was an important lesson for me. There are many other factors that can come into play.
Is misattribution of the reasons behind project success a problem? It can be, particularly when other teams implement the same kind of process, and find they aren’t much further ahead than they were before, over time. The cold reality of their situation, when contrasted with the raucous Agile hype, has caused many to justifiably feel misled. This kind of experience strongly fuels a backlash mentality. The popular (and long) post by Steve Yegge has elements of this, along with some Google hype: Good Agile, Bad Agile. Here is his follow up: Egomania Itself.
Moving Forward
The Agile backlash is here. Some of it is justified, and some of it is not. My intention in this post is to encourage people who find themselves in these different situations I’ve described above.
Are you an Agilist, and you find the particular processes your team subscribes to are working? Are you afraid of returning to the way things were before? Acknowledge that there is a backlash, and for whatever reason, you and your team might be faced with it. Build on what you have learned, and don’t be afraid to adapt and change to keep your productivity and quality at the levels you have achieved. Work to improve what you are doing, and don’t be afraid to discard practices that aren’t working anymore.
Are you a skeptic? Have you found a way to do things that works for your team? Good for you. Don’t give in to the divisive behavior of those who want to capitalize on Agile marketing. You are not alone — there are others who are in the same boat as you. Stick with what works, and don’t be swayed by the hype. Make your position defensible, and have a look at what some of the Agilists are talking about, so you are aware of the issues. Are you a skeptic from within, someone who finds themselves disillusioned with the Agile hype and zealotry? Have you tried Agile methods and find yourself in a similar position that you were in with other methods? This shouldn’t be surprising. Change your course to improve what you are doing. The more difficult the situation you find yourself in should let you know how drastic your changes should be.
Are you a decision maker who is looking for a formula to make more money, or to save more money and be more effective? If you are, I have a bridge to sell you in Saskatchewan. There is no magic formula. There is no cookie cutter approach. The Agile methodologies will not solve all development process ills, and they certainly won’t guarantee success in business ventures. In some cases, they can cause more problems than they address. As Fred Brooks told us twenty years ago, and Gary Pollice recently echoed, there is no silver bullet.
Agile methods are like anything else in life. Applying them involves trade-offs, and unintended consequences. A backlash is certainly one of them. Handling any trade-off well requires solid thought and good problem-solving. Let’s move forward together, and create the new future of software development, standing on the shoulders of the giants who have come before us.