Agile is supposed to be yesterday’s news. Advanced companies? They have jumped to DevOps. Except, a lot of application teams not only have not adopted agile techniques, they do not even see how they could.
This may be surprising until you take a closer look at what these teams need to do: there is very little overlap between their objectives and what agile methodologies deliver.
Agile methodologies, especially Scrum, which many IT managers think is synonymous with agile, are best at what IT does least: develop new applications.
With the exception of the company’s web site and mobile apps, one of IT’s core principles is “buy when you can, build when you have to.” IT licenses something like 90% of all new functionality in the form of commercial off-the-shelf software (COTS) and software as a service (SaaS), leaving 10% for software developed in-house.
Scrum, Kanban, Lean Software Development and most other agile methodologies are designed for the 10%, not the 90%. On top of which, in a typical shop, about 70% of developer hours goes to maintaining and enhancing applications already in production, leaving 30% for implementing new ones.
Do the maths: Agile is mostly useful for 10% of the 30% — in other words, it handles a whopping 3% of what IT’s application teams are called on to do.
Your mileage may vary. Punch in your own numbers. The result will still be small, but it does not have to be.
Maintenance and enhancements the agile way
Taking the easy one first: the maintenance and enhancement requests that constitute so much of what IT does on a day-to-day basis.
You do not even have to squint at the maintenance and enhancement queue to see how much it looks like an agile backlog. Add a product owner, start phrasing requests as user stories (we will get to this in a minute). Apply Kanban: the product owner sets the priorities, and whenever a developer finishes a user story, he or she will pick the next one off the backlog to work on. It is simple, imposes very little overhead, and gets the job done.
Agile COTS
Applying agile methodologies to COTS implementations is not quite as simple as applying them to maintenance and enhancements. But then, you wouldn’t expect it to be: implementing big systems isn’t as simple as handling maintenance and enhancements, period.
But you can apply agile techniques to COTS, too. You just have to recognise two crucial points.
Firstly, implementing COTS, whether on-premises or through software as a service (SaaS), is not just like developing software from scratch, with maybe a tweak or two needed. It is different in profound ways that make Scrum and Kanban severely unsuitable for the job.
The second is something we have already covered, only more so: there are more agile methodologies than Scrum and Kanban.
App dev vs COTS: It starts with the data
When IT teams develop software, they need to understand what “the business” needs the software to do. Once they’ve reached this understanding they don’t start by developing software that does it. They start by designing data structures. The application comes later.
Except, that is, when they do not have to design data structures because an existing database already handles what the new application needs.
Buy-when-you-can/build-when-you-have-to is profoundly different.
When IT implements multiple COTS/SaaS packages, existing databases make the job harder, not easier. Each package comes with its own database, and when the software vendors design their databases they do not take the ones you already have into account because they can’t. They also don’t take the ones other vendors have already designed into account because why would they?
So when IT implements a COTS package it has to track down all the existing databases that store the same data the new application manages too — not to take advantage of it, but to figure out how to keep the overlapping data synchronised.
When it comes to managing data, internal development and package implementations are completely different.
Describing what the software is supposed to do
With the data in hand, application teams have to describe what the business wants the software to do. Most agile variants prescribe the use of “user stories” for this purpose — statements that take the form, “As a <type of user> I want <a goal> so that <good reason>.” For example, “As a telemarketer, I want to store information about customer households so I only call people who might possibly want to buy the next product I’m selling.”
Okay, the example is obviously fiction, but do not worry about that. Worry about this: if you are implementing a CRM system you will waste everyone’s time and annoy the living daylights out of business managers if you insist on this sort of thing. If you are implementing a CRM package that does not store information about all forms of “customer” you are not implementing a CRM package.
When you implement COTS or SaaS software, the usual argument is whether to implement it “plain vanilla” or to tailor it to support the company’s business processes and practices. These are supremely silly alternatives to argue about.
COTS software comes with overt or implied versions of business processes that in many cases are superior to the ones currently used in the business areas that will use it. In many other cases it is not.
Those who argue for customisation (because IT has to make its “internal customers” happy) are lobbying for spending a lot of money implementing a shiny new piece of software so you can go about your business exactly as you are used to.
Big cost, no business improvement, by design. What?
So plain vanilla it is, because there is no chance at all a business process is different, not because it happens to be, but because it has to be.
Sometimes, actually by definition, a company is the best in the business at something or other. By forcing plain vanilla, and IT is insisting on business deterioration as a strategy. If IT and the business it supports are going to have an argument, plain vanilla vs. customisation is a bad argument to have. You can skip it entirely.
Two-stage COTS
When a business (not IT) implements COTS-enabled intentional business change there are two stages to the effort: making the software competent, and optimising the business.
You can handle the first stage by using a little-known agile variant called the Conference Room Pilot (CRP). How it works:
First, IT loads master data — customer data, product data, and so on — into the new package’s database. Then the project manager reserves a large conference room loaded with lots of white boards, large monitors, and other cool stuff. Next comes a large stack of the business transactions the new package will have to handle in order to be competent at the business.
These business transactions might be the result of a carefully crafted test plan. Or, they might just be whatever was processed last month or the month before. Either approach works fine.
Then, you bring in several business users and some application gurus, lock the door, and occasionally throw in a few pizzas and assorted beverages.
What happens behind the locked door? A business user picks up a transaction, tries to make it work with the software as it is, and explains to the gurus, “I can’t process this transaction because …” The developer fixes the “because” and it’s on to the next reason. After a while the becauses become harder to find. As they do, Stage 1 fades out and Stage 2 fades in.
The transition is marked by a change in the conversation. Where in Stage 1 business users explained why they couldn’t process a transaction, the Stage 2 conversations sound like, “I could process this transaction more easily if the software would…”
Voila! The conversations are not about making the software competent anymore. They’re about making processes more effective – about business optimisation.
Oh, and by the way, your developers are not passive order-takers in this process. At any time they are encouraged to say something like, “Making the software do this would be messy and expensive. How about making it do this other thing instead?”
All this without a single argument about whether plain vanilla or chocolate sprinkles is the right way to go. Arguments over sausage and pepperoni or Canadian bacon? Those are healthy discussions.
Darkest secret
Agile has a secret darker than it being useless for most of what IT does: Agile development is all about product delivery. Whether it is Scrum, Kanban, Lean, or what-have you, the project is finished when IT delivers a software product.
But, whether you are using Scrum to develop software from scratch or a package using CRP, product delivery is not what anyone needs. The point had better be intentional business change or there’s no point. The application is a lever the business can use to achieve the desired change.
CRP’s whole second stage is about making the business run differently and better. Intentional business change is baked into it.
Using agile techniques to develop an application, on the other hand, is rife with risk that business change will be Someone Else’s Problem, because everything about Scrum, Kanban, and the rest are carefully crafted to make product delivery the goal. If that were not the case we would not have product owners, and user stories would not be about what the software is supposed to do.
What is needed are techniques that start with designs for doing things better, turn those designs into user stories, and adding tasks to the backlog that are all about changing how business gets done.
Or, even better than this, techniques for iteratively and incrementally improving how business areas go about their work, accomplishing those iterative increments through application changes and enhancements tied to the ideas for business improvement.
In conclusion
Okay, as scandals go, Agile’s darkest secrets are pretty tame. Nonetheless, ignoring them can lead to serious wastes of effort, pointless arguments, and project successes that look a lot like failures once someone asks whether the company is better off than it used to be.
IDG News Service
Ltd
