Fork me on GitHub
Risk First Logo

Home of Risk-First Software Development



Start Here


Home
Contributing
Quick Summary
A Simple Scenario
The Risk Landscape

Discuss


Please star this project in GitHub to be invited to join the Risk First Organisation.

Blog


Consuming Hacker News 14 May 2019
Twitter Cards 22 March 2019
Next - Scrum? 7 March 2019
Available On Amazon 25 February 2019
New Proof 10 February 2019

Publications



Click Here For Details


One Size Fits No-One

Why are Software Methodologies all different?

Previously, we made the case that any action you take on a software project is to do with managing risk, and the last section, A Conversation was an example of this happening.

Therefore, it stands to reason that software methodologies are all about handling risk too. Since they are prescribing a particular day-to-day process, or set of actions to take, they are also prescribing a particular approach to managing the risks on software projects.

Methodologies Surface Hidden Risks…

Back in the Development Process section we introduced a toy software methodology that a development team might follow when building software. It included steps like analysis, coding and testing. We looked at how the purpose of each of these actions was to manage risk in the software delivery process. For example, it doesn’t matter if a developer doesn’t know that he’s going to break “Feature Y”, because the Integration Testing part of the methodology will expose this hidden risk in the testing stage, rather than in let it surface in production (where it becomes more expensive).

… But Replace Judgement

But, following a methodology means that you are trusting something other than your own judgement to make decisions on what actions to take: perhaps the methodology recommends some activity which wastes time, money or introduces some new risk?

Following a software methodology is therefore an act of trust:

In this section, we’re going to have a brief look at some different software methodologies, and try to explain why they are different. Let’s start with Waterfall.

Waterfall

“The waterfall development model originated in the manufacturing and construction industries; where the highly structured physical environments meant that design changes became prohibitively expensive much sooner in the development process. When first adopted for software development, there were no recognized alternatives for knowledge-based creative work.” - Waterfall Model, Wikipedia

Waterfall Actions

Waterfall is a family of methodologies advocating a linear, stepwise approach to the processes involved in delivering a software system. The basic idea behind Waterfall-style methodologies is that the software process is broken into distinct stages, as shown in the diagram above. These usually include:

Because Waterfall methodologies are borrowed from the construction industry, they manage the risks that you would care about in a construction project, specifically, minimising the risk of rework, and the risk of costs spiralling during the physical phase of the project. For example, pouring concrete is significantly easier than digging it out again after it sets.

Waterfall, Specifications and Requirements Capture

Construction projects are often done by tender which means that the supplier will bid for the job of completing the project, and deliver it to a fixed price. This is a risk-management strategy for the client: they are transferring the risk of construction difficulties to the supplier, and avoiding the Agency Risk that the supplier will “pad” the project and take longer to implement it than necessary, charging them more in the process. In order for this to work, both sides need to have a fairly close understanding of what will be delivered, and this is why a specification is created.

The Wrong Risks?

In construction this makes a lot of sense. But software projects are not the same as building projects. There are two key criticisms of the Waterfall approach when applied to software:

“1. Clients may not know exactly what their requirements are before they see working software and so change their requirements, leading to redesign, redevelopment, and re-testing, and increased costs.”

“2. Designers may not be aware of future difficulties when designing a new software product or feature. “ - Waterfall Model, Wikipedia

So, the same actions Waterfall prescribes to mitigate rework and cost-overruns in the building industry do not address (and perhaps exacerbate) the two issues raised above when applied to software.

Waterfall, Applied to a Software Project

As you can see in the diagram above, some of the risks on the left are the same as the ones on the right: the actions taken to manage them made no difference (or made things worse). The inability to manage these risks led to the identification of a “Software Crisis”, in the 1970’s:

“Software crisis is a term used in the early days of computing science for the difficulty of writing useful and efficient computer programs in the required time… The software crisis was due to the rapid increases in computer power and the complexity of the problems that could not be tackled. “ - Software Crisis, Wikipedia

Agile

The Software Crisis showed that, a lot of the time, up-front requirements-capture, specification and fixed-price bids did little to manage cost and schedule risks on software projects. So it’s not surprising that by the 1990’s, various different groups of software engineers were advocating “Agile” techniques which did away with those actions.

Risks, and the practices that manage them in Extreme Programming

In Extreme Programming Explained, Kent Beck breaks down his methodology, ‘Extreme Programming’, listing the risks he wants to address and the actions with which he proposes to address them. The diagram above summarises the main risks and actions he talks about. These are different risks to those addressed by Waterfall, so unsurprisingly, they lead to different actions.

Different Methodologies For Different Risks

Here are some high-level differences we see in some other popular methodologies:

While this is a limited set of examples, you should be able to observe that the actions promoted by a methodology are contingent on the risks it considers important.

Effectiveness

“All methodologies are based on fear. You try to set up habits to prevent your fears from becoming reality.” - Extreme Programming Explained, Kent Beck

The promise of any methodology is that it will help you manage certain Hidden Risks. But this comes at the expense of the effort you put into the practices of the methodology.

A methodology offers us a route through the Risk Landscape, based on the risks that the designers of the methodology care about. When we use the methodology, it means that we are baking into our behaviour actions to avoid those risks.

Methodological Failure

When we take action according to a methodology, we expect the Payoff, and if this doesn’t materialise, then we feel the methodology is failing us. It could just be that it is inappropriate to the type of project we are running. Our Risk Landscape may not be the one the designers of the methodology envisaged. For example:

Choosing A Methodology

Inappropriate Methodologies create their own risks

There is value in adopting a methodology as a complete collection of processes: choosing a methodology (or any process) reduces the amount of thinking individuals have to do, and it becomes the process that is responsible for failure, not the individual (as shown in the above diagram).

It’s nice to lay the blame somewhere else. But, if we genuinely care about our projects, then it’s critical that we match the choice of methodology to the risk profile of the project. We need to understand exactly what risks our methodology will help us with, which it won’t, where it is appropriate, and where it isn’t.

“Given any rule, however ‘fundamental’ or ‘necessary’ for science, there are always circumstances when it is advisable not only to ignore the rule, but to adopt its opposite.” - Paul Feyerabend

An off-the-shelf methodology is unlikely to fit the risks of any project exactly. Sometimes, we need to break down methodologies into their component practices, and apply just the practices we need. This requires a much more fine-grained understanding of how the individual practices work, and what they bring.

Methodologies, Actions, Risks, Goals

As the above diagram shows, different methodologies advocate different practices, and different practices manage different risks. If we want to understand methodologies, or choose practices from one, we really need to understand the types of risks we face on software projects. This is where we go next in Part 2.

Found this interesting? Please add your star on GitHub to be invited to join the Risk-First GitHub group.