2. How to Win

If Development is A Game, How Do I Win?

Fun presentation for a conference about betting and odds in development

Ok, so the next stop is the title of the talk: If Dev is a game, how do I win?

We’re going to build on the idea of work being risk management, by looking at Risk Management in games.

Let’s start with gambling games, with the caveat that I am neither a poker player or professional gambler.

A naive strategy for betting on, say, a horse-race is to try and pick the winner.

The problem with this strategy is that the most likely horse to win is the one with the shortest odds. That means, if you win, you won’t win much money.

Let me give you an example.

So, let’s say there’s a horse race with these three famous fictional horses. Actually Horse C is technically a pony, which probably explains her lower chance of winning.

So, let’s say Horse A has a 50% chance of winning. You might have odds of 3:2, which means if you put two pounds on the bet, you’ll get three back.

Clearly, if you bet on this horse over and over, you’d eventually lose your money.

In fact, that’s true of any of these horses: the odds always pay back worse than the probabilities.

If these probabilities are correct, you’ll always end up losing.

So picking the horse you like best is not going to work. In order to win money horse racing you have to have “an edge”.

So let’s talk about Internal Models. This is a really helpful concept in Risk-First: the idea is, the Internal Model is what you know about the world, or, what you think you know.

An Internal Model could be the information you have in your head. Or, it could be data stored on computer.

There’s no guarantee that it’s correct.

Now, why is this an important concept? How many times have you been in a meeting and had an argument about either what the team should be doing, or the approach a team should take to do something?

Personally, quite a lot.

And usually, the reason for the argument is not that you have a different idea of the goal of the team - that’s usually quite clear.

It’s also not usually that some person is trying to subvert the team and cause it to fail.

And, most often, I am arguing with people who are just as passionate about me regarding the success of a project, so it’s not that I’m trying to persuade lazy people to do work.

Often, the people involved in the argument also have access to pretty much the same facts about the situation.

So, the reason we’re arguing is that our models of the situation are telling us to do different things.

So, Chess is a great example of the power of Internal Models.

If I was to play chess against a grand master, we both have perfect information about what’s going on in the game - it’s a perfect information game - the board contains everything we need to know, and there is nothing random that affects it.

However, the grand master’s Internal Model is much better developed than mine - they’ll see the risks to all their pieces much more clearly than I will.

They’ll understand all the risks of the moves they make much better than me.

So, chess grandmasters spend their lives improving their internal models of chess by studying other games and other players throughout history.

This is Donald Rumsfeld’s famous quote about known unknowns:

"As we know, there are known knowns. There are things we know we know. We also know there are known unknowns. That is to say, we know there are some things we do not know. But there are also unknown unknowns — the ones we don't know we don't know.”

He’s directly talking here about Internal Models. But he was the US Secretary of Defence at the time of 9/11 and the war in Afganistan.

So he’s really talking about the Internal Model of the entire US military operation.

So, I’ve tried to break that down on this slide. Here are the three categories of things in an Internal Model that Rumsfeld suggested, plus a fourth one: things you know, but didn’t know you knew them, which I guess has to exist.

Which I guess might include I don’t know, the plots of movies you forgot you saw. Institutionally, this could happen a lot, with a company having records of things that all the staff have long since forgotten about.

Getting back to the horse racing example, we’re dealing there with known unknowns: the box on the top right - we know that one of the horses will win, just not which one.

And, in the bottom right - the unknown unknowns. In the chess game, these are the ones that are going to get me - the moves I don’t see coming.

And these are the things that wreck our ability to estimate tasks in software development. How many times have you said, “Oh, this will be done next Tuesday,” but then been blind-sided by some completely unpredictable quirk or bug or facet of the problem that just throws the whole thing out?

We’re going to look more closely at what to do about Unknown Unknowns in a bit.

The key to being a successful gambler is not to try and pick the winner, necessarily, but to play the odds. You need to have a better Internal Model of what is going to happen than whoever you’re making the bet with.

So, that is, making your model of Known Unknowns better than the person you’re playing against.

This guy, Joseph Jagger, made a ton of money in the 1880's out of Roulette because he had a better Internal Model. He sent people to casinos to record the results of each spin of the wheel. And he found that some wheels were biased and so certain numbers came up more often than others.

All he had to do was bet on those numbers to win.

For the casinos, the idea that some wheels were biased was an “Unknown Unknown”: they had no conception of it. But Jagger turned it into a “Known Unknown” in his model, and won.

And all he had to do to get this "improved" Internal Model, was to go out and experiment in the real world, and record some observations.

You might be thinking - that’s all very well, but it doesn’t really apply to software development, does it?

Well, I’m going to try to persuade you that it does.

In order to “Win” at software development, you have to consider not just the game that you’re playing, but the meta-game.

For example, the “game” might be the current task. Maybe the task is “add a login box” to the website.

A really naive understanding of this game might be: I’m just going to knock this out as quickly as possible.

In Agile circles, there’s an expression “YAGNI” - You Aren’t Gonna Need It. This captures the idea that you shouldn’t over-engineer the solution. So, this developer has taken that to heart and done the simplest thing that could possibly work.

So, at one level- this wins the game. It fulfils the definition of what was wanted. But it loses the “meta-game”. The game-of-games.

So, the product owner might look at this and say, “actually, that’s not what I wanted. Can we have something that doesn’t look completely terrible”

So, second time around, the developer goes away for a whole week and comes back with something like this.

It’s pulling in stylesheets, graphics, an animated background. It looks amazing. It fulfils the brief, but it also fails at the meta-game level.

So, what’s wrong with both these solutions? Put simply, they are not managing risk well.

The product owner is trying to manage risk on the product, and he or she has an Internal Model of what those risks look like.

The very simple login is going to leave users feeling that the product was built by a five-year-old. This isn’t what you want to convey.

On the other hand the super-awesome login took ages to build, and brings in new stylesheets and images and all sorts. This is adding more complexity than necessary.

So, the game is to implement the requirement. But the meta-game is to implement the requirements whilst managing the risk to the overall product. And this is something that everyone needs to consider.

Now, an interesting thing does happen once you build either of these login pages. And that is, your internal model changes.

Some things that were “unknown unknowns” are now “known unknowns”.

Will users be able to reset their password?

Will they be able to sign up?

How will we store their credentials securely?

Do we have to worry about being hacked and losing people’s logins and email addresses?

Do we have to worry about data protection regulations?

There’s a whole workflow around logins that you need to consider, of emails and resetting passwords and so on.

Now, I’m sure a lot of you are thinking - this is obvious stuff! Wouldn’t they think about all that beforehand? And, I deliberately picked an example which most people would have familiarity of.

But so many times in software we have problems where, when you first look at it, it’s an easy problem, but the more understanding you get, the more intricacies and details there are to consider.

So, at this point, you’re probably thinking, “Couldn’t they use a third-party security library”, or “Shouldn’t they use a social login, like OAuth2”.

And, these might be great solutions - they’ll probably take a lot less time to build, but they come with their own risks attached.

With either of those, you end up with a dependency on some third party software. Dependencies are also risks: you don’t know whether that software will be supported forever, or have a security breach, or stop working, or they change the terms of use, charging you more money than you want to pay. I mean, Oracle are kind of famous for that, right?

Using a social login has it’s own risks too. You might be excluding people if they don’t have a Facebook account, or whatever provider you choose.

By the way, these are all risks that are broken down and described in detail on the Risk-First website.

Ok, so obviously, we know that different solutions have different trade-offs. In a way, whenever we take an action to try and do something on a project, we are making a bet.

Just like in the horse racing, or the roulette example, what we want are bets where payoff is worth what you stake.

So, you might say: a bet on OAuth2 adds the software dependency risk, but at least it doesn’t tie up one developer for a month, pushing up my schedule risk. Things could go south, but this might be the safest bet out of the ones available.

So, this is what I mean about games and meta-games. At the bottom are the games: short term goals that you think you should achieve to “win” in quotes.

However, the person running the project should be concerned about the meta-game. That is, the game-of-games, where you are trying to balance all of the risks in play and end up with a successful project, low complexity, a happy team, happy customers.

To do this, you take shrewd, calculated bets.

One of the ways I talk about this on Risk-First is that these bets are like moves on a “Risk-Landscape”. You can’t escape risk, but you can move your project to a position on the landscape where the risks are more in your favour.

So this diagram above tries to give you a pictorial representation of the risk landscape. In golf, you have to start with your ball at the tee, on the left side, and get to the hole on the right side. The winner is the person to do this in the least number of shots, or strokes.

Every time you hit the ball, it ends up in a new position and you find yourself with a new set of hazards, or risks to think about.

Every time you hit the ball, you’re taking a bet that you’re going to end up somewhere better. And, because of all these hazards, and the fact that you’re not perfect at hitting the ball, you have to manage the risks of where you’re going to end up.

Obviously, this is an analogy, but I think it helps to think about choices we make in software projects as being like the choice of shots we could take in golf.

So, let’s look at some things we do on projects, and see how they are like bets.

We’ve all done refactoring before: the idea is that we spot some duplication of functionality in our code, or a better way to separate the concerns and we implement it to make our lives easier in the future.

So, what’s at stake with the refactoring? Well, the main thing is time. Usually what I find is that I have some idea for a refactoring, and start implementing it, and it touches large tracts of the codebase, and I have to fix all that up, and then I have to fix a load of unit tests too.

In return, the payoff is that I reduce the Complexity of the overall project. In some complex projects, keeping complexity under control is a really big deal, and it saves time in the number of bugs and the amount of testing and so on.

So, it might be a good bet. I might win, if the refactoring achieves its goals, and doesn’t take too long.

On the other hand, I might lose if it takes ages, or halfway through I realise some edge-case is going to ruin the whole thing.

So, it’s a bet - based on my limited Internal Model.

Adding a new feature to a piece of software is also a bet - again, it’s a step into the unknown.

So, let’s look at the Stake:

Adding a new feature is going to take time - that’s schedule risk. It’s going to make your product more complex, that’s Complexity Risk. And, it might make your product harder to understand and use - that’s Conceptual Integrity Risk.

On the other hand, if the features of your product don’t fit the requirements of the client then they may go off and start using a competitor’s product. So, that’s hopefully this feature fit risk we’re reducing on the other side.

Will it work? We don’t know! it’s a bet based on your imperfect information!

So, something that I think people kind of understand intrinsically, or maybe subconsciously, is that an estimate is a bet.

This is especially true if you are on a fixed price contract. A fixed-price contract is actually like a spread-bet, where if you are doing the work and you come in quickly, cheaply early, you make lots of money, whereas if you come in late, you lose.

So, the payoff, for doing Fixed Price Contract is that your business is going to make money - it’s funding risks will be reduced.

But, for the client, the great thing about a Fixed Price Contract is that they know how much the project will cost. So, you’ve taken on their funding risk. And of course, you’ve taken on their Feature Risk - you have to provide the Features of the software that this client needs.

So in order to make accurate estimates, and win these bets, it’s all about having a good internal model of the development process, and how long things take.

There are lots of reasons why, in software, Estimating is really, really hard. I’m not going to dwell any more on this today, except to say that I’ve spent quite a lot of time on Risk-First writing articles about the difficulties with estimating, explaining why it is so hard.

So on the Risk-First website I have a short section covering a few more examples of things like this, which you can look at.

Personally, I feel it is quite liberating to talk about pieces of work not as “issues” or “tasks” but as either “bets” or “experiments”.

The reason this is liberating is because it means psychologically you’re not making the delivery of a feature or other piece of work part of your own “identity”. And, it shouldn’t be: we are all operating in a world of unknown unknowns, with imperfect Internal Models.

So, hopefully we’ve covered a couple of things at this point.

First, I’ve talked about how all work is actually risk management.

Then, in this section, I’ve talked about how to do good work, i.e. to “Win” at software development. And, it basically comes down to not having the best algorithm or the best web page, but to doing the right pieces of work to manage down all the various risks across the project to levels you can cope with.

In order to do that, we need to take bets on the work we do, and in order to win those bets, we need to have a good internal model of the risks that we face.

But actually, all of that is really what we do in our everyday lives. Well, ideally anyway.

If you know you are at risk from heart disease, you might choose to do lots of cardio-vascular exercise, or choose the right diet.

If you have a genetic risk to breast cancer or prostate cancer, you should choose to get tested for those things.

Essentially, you are managing risks in your life.

And, I think we can agree that generally, when we are healthy, we’re able to do better work and look after our families better and so on.

So, doing all this Risk Management stuff I’ve been talking about isn’t crazy complicated - it’s a lot like living a healthy life. And vice-versa. If you want to live a healthy life, you need to develop a good internal model of the risks that face you, and do work to keep them under control.

And, we use terms like “this project is in poor health” to cover things like - oh, there are morale problems here, or, it’s drowning in it’s own complexity, or, the stakeholders aren’t engaging or deadlines are getting missed.

Made with Keynote Extractor.