Introduction to the Theory of Constraints

Introduction

I’m obviously a big fan of the Agile methods and Lean/Kanban, as evidenced by how the teams on which I work, consult, or participate use aspects of each for their chosen processes. One of the underlying philosophies of both Agile and Lean/Kanban is the Theory of Constraints (ToC). The Theory of Constraints was developed by Eliyahu Goldratt in the early 1980s to describe how workflows can be understood and improved by focusing on the bottlenecks that constrain the flow of value through the system being measured.

Goldratt explained his reasoning in three novels, The Goal, Critical Chain, and It’s Not Luck. In each of these, different aspects of the ToC are described through mildly entertaining characters and story lines. The first book, The Goal, describes the ToC itself and the Process of Ongoing Improvements (POOGI). Critical Chain talks about Critical Chain Management, a style of thinking that takes the concept of a critical path for a project and expands it to support multiple projects. And finally, It’s Not Luck introduces Goldratt’s Thinking Processes, which are orderly patterns of thinking that help you understand the current situation, define the intended future state, and chart a path from to that future. All great reads, highly recommended.

The Theory of Constraints

His most important point, however, is the ToC and advice about how to apply it. In a nutshell, the ToC boils down to understanding how value flows through your production system, learning where the bottlenecks are, and iterating over improving the efficiency and effectiveness of processing at the bottlenecks. He gives the outline of a process to follow in this improvement, called the 5 Focusing Steps:

  1. Identify the constraint in your system,
  2. Exploit the constraint by making sure that this particular part of your workflow is busy and productive all the time,
  3. Subordinate everything to the constraint by reducing production eveywhere else in your system to avoid having work pile up, waiting to be processed,
  4. Elevate the constraint by improving its ability to process the value by adding capacity, changing the nature of the work, hiring, and so on,
  5. Return to the first step to find the new constraint in the system.

This process never stops, as there is always exactly one bottleneck in a system, and you can always discuss ways to eliminate it.

In The Goal, Goldratt writes a pretty approachable example of the Theory of Constraints in practice when he describes a Boy Scout march the protagonist, Alex, leads with his son. His job is to take the whole troop on a trip through the woods to a camping site before nightfall. The boys start marching through the woods, and for various reasons they aren’t making the kind of progress that he hoped for. One boy in particular, Herbie, is moving a lot more slowly than the rest, and he is both slowing down everyone behind him and allowing everyone in front of him to spread out so far that they can’t keep track of each other.

Through some careful thinking, Alex understands that Herbie is the limiting factor, or constraint, in how fast the team can march. So he exploits how fast Herbie can march by putting him at the front of the line of boys, so that he can always keep moving, and and subordinates the progress of every other boy by telling each to keep a 10 foot margin behind the boy in front. Now everyone is marching at the same rate, and the length of the column is fixed for the most part. Since the speed is now constrained by how fast Herbie can march, he elevates that speed by offloading the junk in Herbie’s pack to the other boys, which lets Herbie and the whole troop march faster. They arrive at their destination, all are happy, and Alex learns a valuable lesson he can take back to work the next day.

It’s a better story in the book, but you get the idea. The ToC is actually a pretty simple theory, but it has amazing applicability to so many situations. Read the books and judge for yourself.

Why Should You Care?

The whole reason why I’m writing about this is that I believe the Theory of Constraints can be used to explain and justify the processes and practices that are part of the individual Agile methods. In future blog posts, I want to take a typical team and talk about how you can optimize the progress made by that team through applying the ToC to the constraints of that team.

I plan on writing posts about the importance of good and important stories, the criticality of acceptance tests to avoiding unnecessary work, and then discuss the Extreme Programming engineering practices of Pair Programming, Refactoring, Test Driven Development, and Continuous Integration/Deployment.

Thanks for reading this far, and I’m looking forward to writing the rest of this series over time.

bab

Additional Reading

A Life Changing Event

After almost 8 years at Asynchrony, I resigned my position as VP of Engineering on Thursday to take a new position working with some of my old friends. The times at Asynchrony were some of the best times in my career, and I feel privileged to have worked with so many bright, driven, passionate people. The reason for this move has nothing to do with Asynchrony – I consider them to be one of the good guys in the industry. They get Agile there, they get Lean/Kanban, and they get delivering quality software. I will miss them, I will  miss the challenges, and I will miss calling myself an Asynchronite.

But, old chapters close in our lives, and new chapters open, and this is one of those times.

I have accepted a position with Tier3, working alongside my old Patterns and Practices friends, Jim Newkirk, Scott Densmore, Tim Shakarian, Brad Wilson, Ade Miller (I think there are one or two more that I’m leaving out, but I only heard the whole list once. Apologies if I left someone off), as well as other teammates I’m very eager to meet. In my new role, I will be Product Owner, community leader, and one of the public faces of Tier3’s Platform as a Service family of offerings. And I even get to write some code!

I have a ton to learn, about the space, the product, the team and people, and that’s probably just touching the surface. I’ve been consulting, mentoring, and training for half of my career, and I’m excited about moving back into the product world. I get to work with world class engineers and an exciting company to create, to quote Ron Jeffries’ words from oh, so long ago, Something Magnificent.

new Chapter().Next().Open();

— bab

“You get what you measure” versus “What you measure, you can manage” – The Agile Metrics Dichotomy

(I just reinstalled Windows Live Writer and reconnected it to my blog, and it pulled down some drafts I wrote years ago. So, I have no idea when I actually wrote this, what the context was that was in my head, but it seems like a decent article. So I’m posting it :))

I’ve long been a fan of the “You get what you measure” school of metrics doubters, especially in agile projects. Its really easy to decide to measure something, have the people being measured act in a totally predictable and rational way, and screw up a team beyond belief. For example, if I measure how productive individual programmers are, then its to the advantage of individuals to focus on their own work and spend less (or no!) time helping others. Completely kills teamwork 🙁

On the other hand, in our environment, where we are responsible for delivering projects to clients on time, on budget, etc, we have a responsibility and an obligation to know what is happening in each of our project teams. Management needs to know if we’re going to make our deadlines, customers need to know what is coming, project leaders need to be able to manage and set expectations, we need to satisfy contractual obligations, etc.

The challenge is to come up with a set of metrics and an unobtrusive way to gather data that allows metrics to be reported to interested stakeholders, and yet does not disrupt the team.

This is my challenge. And what follows are the basic metrics I’ve been using as a way of meeting that challenge. These metrics are balanced between measuring things that meaningfully describe what’s happening inside teams, and creating the right environment and behaviors that will help the team reach its goals rather than incent poor behavior.

How Fast Are We Going?

The gold standard of progress on an agile team is its velocity. This can be measured in story points per week for teams following a more “traditional” agile process like Scrum or XP, or it be measured in cycle time for those teams that have evolved to a Lean or Kanban approach. The team I’m coaching now is somewhere in the middle, closer to Scrumban.

So our standard metric is points per week. We measure weekly, and our measurements are reasonably accurate, varying between 25 and 30 points a week. We have the occasional week when we’re significantly below this, and some weeks where we way overachieve. Our historical velocity chart looks like this:

At first, our velocity varied a lot, mostly because we were still learning the codebase (this project has a large legacy codebase that we are adding functionality to). As of late, though, we’ve become a lot more consistent in what we finish each week. Not perfect, but a lot better.

But What About Quality?

Measuring velocity is not enough, as teams can find ways to go faster when pressed to do so. This is a great example of measuring something that is going to create the wrong behavior. If a team is rated on how fast they go, they are going to go as fast as they need to. They’ll either cut corners and quality will slip, or they’ll inflate the estimates to make the numbers start to look better without actually accomplishing anything more.

What I’ve found is that the best way to avoid incenting the team to cut corners is to keep an eye on quality, too. So, in addition to tracking  velocity, I also track defect counts. These charts look like this:

 

This shows our weekly quality over time since the beginning of our project. We started off pretty high, put in some good effort to get our quality back under control, and have done a decent job of keeping it under control most of the time.

Looking at the Bigger Picture

What I do is to use these two metrics together to help me understand what’s happening on the team and to ask questions about what may be causing ebbs and flows in velocity or quality. We did a good job of keeping the defect count pretty low during February and March, but the count crept back up, pretty quickly, to 20 or so active defects by the beginning of April. It was towards the end of this relatively quiet period that the time began to feel pressure to deliver more functionality more quickly. They did what they thought was best and focused more on getting stories “finished” without spending quite a much time on making sure everything worked. They worked to what was being measured.

Up steps Project Manager Man (that’s me, for those of you following along at home), and I was able to point out what was happening with quality and, with their help, relate that back to the feeling of pressure they were feeling. Together, we came up with the idea that velocity and quality have to improve together or teams create false progress. I helped the team manage this mini-issue on their own by pointing it out to them through metrics.

The Final Velocity/Quality Metric

As a way to keep an eye on the general trend of where the work is going on the team, I also track time spent working on Acceptance Tests versus time spent fixing defects (defects include stories that are kicked back from QA or the Product Owners prior to acceptance) versus time spent developing. My feeling is that if we can maximize development effort and minimize defect remediation time, then the team is operating as effectively as it can, given its circumstances.

The Most Important Point About Metrics

Metrics should never be used to judge a team or used to form conclusions about what is happening. They should be treated as a signpost along the way, an indicator of things that are not easily observed in real time but can be measured to point out trends. They should always serve as the starting point for a conversation, the beginnings of understanding an issue. If a manager or executive begins to use metrics as a way to judge a team, the metrics will be gamed and played from that point on. They will become useless, because they will be a reflection of what reality the manager wants to see, rather than what is truly happening.

In each and every one of these cases, I have used the metrics to confirm what I thought I was seeing, as the starting point of a conversation with the team about what they saw happening, and a way to measure the impact of issues on the team. After speaking with the team, the metrics are also useful to judge whether the changes chosen by the team to improve are effective.

Metrics are no substitute for being on the ground with your team, in their team room, pairing with devs and testers, working with product owners, feeling the vibe in the room. Remember, its people over process, right? 🙂

— bab

Looks like I’m speaking at SQE East in Boston, November 12th, 2013!

My good friend Mitch Lacey had to take a step back from a speaking engagement, and he asked me to cover for him to give a half day tutorial on Getting Started With Agile: An Experiential Workshop at the Better Software Conference in November. I’m very excited about doing this talk, both because of the opportunity to speak at this venue and to this audience, and because this kind of talk is so much fun to give.

This session is described as experiential because this talk about Agile and Scrum is planned and execute using Agile and Scrum, With the audience as the stakeholders and me as their “team”, we go through an initial story-gathering process, writing user stories for questions, estimate the questions, put them into a backlog, track metrics, replan and refine, and so on, all while answering the questions posed by the audience.

Doing this talk requires a bit of knowledge and a LOT of courage on the speaker’s part, as the questions can cover literally anything involved with the subject. I’m very glad Mitch has the confidence in me to be able to stand in for him, as his are some very big shoes to fill. Mitch, I’ll do my best not to let you down 🙂

— bab

New blog home

Hey, sorry for the inconvenience. We lost our hosting at our previous site (Thanks, Peter, for doing it for over 5 years!), so I’m reloading all content here, setting up a nice theme, etc. It might take a while, but it’ll eventually get done.

Thanks for your patience…

— bab