This is a team exercise to address the problem of two smart people who can’t agree.
When I lead teams in person I do this on a whiteboard. To start I write the following sentence on the board:
In order to be high quality, software must be __________.
And I ask the team to brainstorm as many ideas as possible for filling in the blank. I write down all the ideas in a long list. Here are some common responses:
- Easy to use
- Working
- Fast
- No Bugs
- Can make updates
- 100% test coverage
- Simple
- Low code complexity
Usually the list is long and contains significant overlap so typically I group these into larger chunks. While I’m grouping them, I also chose the name for the chunks so that I can introduce the four terms that I use.
“Code coverage” or anything test related I consider under Testable. Testable describes how easy the system is to write automated tests for, which often correlates with metrics like test coverage.
“Can make updates” is what I call Changeable. It may also include things like “Simple” and refers to how the design impacts our ability to make changes to the system.
“Low complexity” and sometimes “Simple” I group together as facets of Understandable. I’m making modifications to a system and I want to be able to understand how it works.
“Easy to use”, “Working”, “Fast”, and “No Bugs” are all aspects of what I consider Correctness. Does it do what it’s supposed to? Does it only do what it’s supposed to? Is it fast enough? Can people actually use it? Only the end user can validate Correctness.
I’ve now culled the list down to these four:
- Testable
- Changeable
- Understandable
- Correct
Next, I ask the team to rank these four in order of their priority. I remind them that if everything is a priority then nothing is a priority. Being ranked lower doesn’t mean we disregard something, it only means that we never sacrifice a higher priority in pursuit of a lower priority.
To find the highest priority I write the following sentence on the board:
Above all else, quality software must be ______________.
I write a “1” next to that item in the list. To finish ranking the remaining items I write this sentence on the board:
In order for software to be [n-1] it would help if it were also ___[n]____.
If “Testable” is #1, I might prompt with “In order for software to be testable it would help if it were also _________.” Then “In order for software to be [Priority #2] it would help if it were also [Priority #3]”
This is where the debate/argument/disagreements should start to surface. That’s what we want so don’t rush through this—discussion takes as long as you need. Until everyone on the team can get on the same page on these foundational ideas, there is no point in trying to get any other work done. Invest the time now to make sure that everyone is heard and that the team comes away with real agreement. We’re not just building consensus, we’re building a shared understanding of how we build software.
When we have that I can trust that your pair can go off and work on their own and what they will produce will be aligned with what my pair would have done. I’ve seen teams who are so well aligned many team members don’t feel the need to participate in every discussion because they trust their team members have it under control.
My Rankings
Once the team has reached a consensus on their ranking of these priorities, I will spend some time explaining how I rank these four and why. If my rankings match their rankings I’m just re-articulating the rational so they have a solid mental framework. If my rankings are different I’m going to reopen the discussion and hopefully persuade the team towards what I consider the most effective prioritization.
Above all else, quality software must be Correct.
This usually isn’t that controversial. I don’t care if you have this amazing perfectly refactored 100% unit test coverage system—if it doesn’t do what it’s supposed to do, it’s useless. On the other hand, there is plenty of software out there with no one left who knows how to change it without breaking it, but as long as it works, we’ll take it.
The user’s not looking at our [source code] and they don’t actually care very much about how good a time we had when we were writing it. What they care about is what the program does.
Rich Hickey, Simple Made Easy [00:20:00]
He showed this long Java method and he asked the audience, who here thinks this is good code? It looked terrible. It was just lots of ifs and lots of indention, it was terrible. And no one thought it was good code.
Then he said, what if I told you that this horrible looking code was created, it was committed once, it’s been in production for five years, and it works perfectly—it’s never been changed. Now is it good code?
Chad Fowler, Impermanence: The Ironic Key To Systems That Survive
In order for software to be Correct, it would help if it were also Changeable.
Correctness can only be measured at a single point in time. Even if you have a system that has zero defects and you never change it, the world moves on. Laws and regulations change. Business rules and process change. In order to maintain correctness through time we have no choice but to make changes to the system.
Last week we had this hellish outage. The system that was the hardest to change was my Haskell service and the reason is that it’s the best code we have. It’s perfect. It performs better than anything else. It has never had to be changed because if you write Haskell and it compiles, it works. That’s just how it works. I’m only partially joking there, Haskell is an amazing thing.
It’s funny because we were so proud of this thing, but the fact that we had never changed it even though we didn’t need to change it, made it a liability. And it bit us and I spent all day, last Saturday, in my hotel room in Berlin, fixing a problem that was caused by the fact that this Haskell code had never had to be changed.
So even when your code is great, perfect, things change around it—that’s the scary thing. So you really do have to keep changing things.
Chad Fowler, Impermanence: The Ironic Key To Systems That Survive
In order for software to be Changeable it would help if it were also Understandable.
Some old-school TDD/XP guys believe testing can substitute for understanding so they will rank Testable at #3 and Understandable at #4. I disagree.
Testing can never prove that the system is correct, only that it is incorrect. Taleb would say this conflates Absence of Evidence with Evidence of Absence. The absence of any failing tests says nothing about the correctness of our software. How many software teams out there have dashboards/radiators showing that all their tests are green and yet their backlog is full of defects?
…there are inherent limits to what can be achieved by testing, and because informal reasoning (by virtue of being an inherent part of the development process) is always used. The other justification is that improvements in informal reasoning will lead to less errors being created whilst all that improvements in testing can do is to lead to more errors being detected.
Out of the Tar Pit, Ben Moseley & Peter Marks
Testing is useful if you want to get the bugs out, but if you want to avoid putting the bugs in, you have to be able to understand your software.
How can we possibly make things that are reliable that we don’t understand? It’s very, very difficult… For the things we want to understand, and make sure are correct, we’re going to be limited. And our understanding is very limited…
So how do we change our software? Apparently, I heard in a talk today, that Agile and eXtreme Programming have shown that Refactoring and Tests allow us to make change with zero impact. I never knew that. I still do not know that because that’s not actually a knowable thing. That’s fooey. If you’re going to change software you’re going to need to analyze what it does and make decisions about what it ought to do. At least you’re going to have to go and say what is the impact of this potential change… if you can’t reason about your program you can’t make these decisions.
Rich Hickey, Simple Made Easy [00:12:34]
And I really do want to ask the question, agilists, is having a test suite and refactoring tools going to make changing the knitted castle faster than changing the lego castle? No way!
Rich Hickey, Simple Made Easy [00:22:10]
The only way to make changes without breaking anything is to understand and reason about the effects of those changes.
It is possible to write software that is to easy to change that someone who has never seen it before can read it in 15 minutes and understand it well enough they can modify it without breaking anything. It’s not hard to do if you know how to design simple systems. Most people have never even seen a simple system so they have no idea how powerful it can be.
In order to make software Understandable it would help if it were also Testable.
I got this idea from J.B. Rainsberger. The purpose of writing tests is not to check whether the system is working. The purpose of writing tests is to validate whether your design is good. Because if the design is good then the tests will be easy to write. If the tests are not easy to write it means the design is not good.
Understandable systems are simple systems. It’s difficult to design simple systems without immediate feedback on how simple your design is. Fortunately for us, it turns out that everything that makes a design simple also makes it easy to write automated tests for. And everything that makes a design complex and difficult to understand also makes it difficult and tedious to test. Writing tests gives us that instant feedback on how simple our design is.
Once you learn how to build systems this way you start to appreciate that tests are useful for the feedback they give you, but they’re not essential. I would rather make changes to a simple system I can understand even if it has has zero tests, than make changes to spaghetti with hundreds of long brittle integration tests.
It’s not difficult to write well designed software without writing any tests once you know how to design simple software. Still, even Rich Hickey writes tons of unit tests for his code, and so do I. Without the tests it’s just too easy to get sloppy. You want the immediate feedback.
And so, my list of Priorities in Software Development, in order:
- Correct
- Changeable
- Understandable
- Testable