Zero Defects™ – Part 2

As I suspected, my notion of Zero Defects was mildly controversial. I’ll try to clear up some of the questions in this post. But, rather than coming up with a cohesive argument, which I tried to make the first time, I’ll respond to some of the responses I received on the first post. (NOTE: Comments did not come with the posts when I manually imported them to Posterous.)

  • Defects vs. stories is just a semantic layer; they both boil down to work that needs to be done.

Semantics are important. Yes, they’re both just work items. But, a “defect” carries with it a negative connotation, whereas a “user story” does not.

I believe strongly that using the words “defect” or “bug” sets the inappropriate expectation with business partners that all defects/bugs can and will be removed from the software prior to promotion to production. And, I’ve seen first hand how using the term “defect” led to teams killing themselves trying to fix “bugs” that were never called out in test cases by the end of a Sprint. This is a recipe for failure and burnout.

I much prefer to set the expectation that anything that wasn’t explicitly called out by a user story and/or test case in the current Sprint is new work that needs to be prioritized with all other work. The best way to do this is to label that work the same way all other new work is labeled – as a new user story.

  • Defects tells me the QA's are doing their job.

To me, the statement above is akin to saying “Lines of Code tell me my programmers are doing their job.” Defect counts are meaningless. Point me at any modern website or program, I can probably identify 100 defects in an hour. Most of them will be garbage. But, if I file a defect on them, someone (or more likely, some team) will have to triage those things, beat down the stupid ones and prioritize the rest – probably outside their normal work item prioritization process.

  • My experience is our business partners are not be concerned with defects, unless they are not caught and are promoted into production.

My experience differs. I’ve worked with multiple PO’s, both at my current employer and elsewhere, who tracked defect counts and berated the team over them. In fact, my experience is that this is more common than not.

  • The only business partner of ours looking at Rally is our PO.

Yeah, that seems to be common here. But, there’s nothing preventing you from calling a meeting at the end of each Sprint where you demo the new functionality to a broader community of stakeholders. In fact, Scrum recommends this practice. It’s called a Sprint Review Meeting.

  • The requirements in agile are much less defined than in waterfall.

I disagree 100%. There should be zero theoretical difference in the quality of requirements between agile and waterfall. Agile just prefers to discover and document requirements iteratively, one at a time; whereas waterfall tries to define all requirements up front. In practice, this means that agile methods discover and document requirements closer to the time of implementation. They also allow for new requirements to be discovered by looking at how the software actually works. Waterfall doesn’t provide for that iterative feedback, meaning that the requirements get stale.

  • Defects provide us with a good method to track these issues to resolution. If the issue is new scope... then an user story is a much better alternative.

The problem I see in tracking defects separate from user stories is prioritization. How does the developer know what to work on next? Sure, Rally (and other tools) allow you to see all work in one bucket. And, that helps. But, in my experience, teams end up with silly rules like “defects found during the current sprint will be resolved in the current sprint.” That seems like an innocent enough rule, but what happens when you find more bugs than you can fix?

  • My interpretation of reading Alan's blog was that those items that are actual defects (do not meet defined requirements)... should be a new user story.

Ah, therein lies the misperception. In my original post, I meant to state that every requirement MUST be expressed as a test cases on a user story, and that all test cases MUST be passing before I show the work to my business partner(s). Therefore, by definition, the code has met all of the “defined requirements” and can only be deficient if there remain undefined requirements, which should be defined as new test cases on a new user story.

  • Bottom line, i do not think defects are bad...

I disagree. I’ve experienced enough problems from the semantic difference between defect and user story, that I no longer want to use the term defect ever again. (It’s not the IT folks that generally misinterpret the word “defect.”)

But, that's just my opinion. What's yours?

Zero Defects™

Last week, in a conversation with a coworker, he stated that defects do not make developers look bad. Basically, he asserted the following:

  1. Defects are not used to judge developer competence; and,
  2. Defects are simply artifacts for tracking work that needs to be done.

I agree with these points. However, experience teaches me that defects can and do make teams look bad. I’ve seen defects become a point of contention between a development team and their business partner(s). And, once a rift forms between the two, it is very difficult to heal.

So, how can we simultaneously encourage defect creation (to track work that needs to be done) without creating a rift between the development team and their business partners? My preferred approach is to create Zero Defects. Here’s how:

  1. Manage your work with a backlog of user stories.
  2. Create test cases for each user story before you begin development. Review the test cases with your business partner as well as QA to make sure you didn’t miss any. (This can happen iteratively. There’s no need to create test cases for stories you’re not going to work on right away.)
  3. As you develop a story, automate the test cases to prove that the software does what the tests say it should. When all the tests pass, the code is complete. (If possible, the developer who writes the code should not be the developer/tester who automates the tests. And, it’s very helpful to automate the tests before the code exists.)
  4. When you finish a story (and the associated tests), review both the software and the automated tests with your business partner to make sure everything is as they wanted. If your business partner wants to make changes at that point – either new requests, or things we might typically call defects – write them up as new user stories and prioritize them in the backlog.

That’s all there is to it. You, the developer/tester, get to take credit in the current Sprint/iteration for the work you completed. And, your business partner gets to manage a single backlog with all the outstanding work in it. Plus, no one has to fight over what’s a defect and what’s a user story. It’s a win/win+!

Now, I realize that this might be considered controversial. So, I’ll explain my thought process in a future post. Feel free to tell me how this’ll never work in the comments!


As much as I love Rally for managing work, I think I may have found something better. Votebox is where Dropbox users can request features and vote for those they’d most like to see. What I like about it is how simple it is:


Here you see the three most popular feature requests on their site. Each feature request contains a title, a short description, a category (not shown) and some metadata (in red). Users may vote for and comment on feature requests (in blue). And, Dropbox can update the status of a specific request (in green).

This is the simplest, most elegant site I’ve ever seen for managing a backlog of work. It simultaneously a) houses the backlog, b) tracks feedback, c) gauges interest in competing priorities, d) communicates progress, and e) manages expectations. It’s like a suggestion box that worked out really hard, but never took any steroids. It has all the muscle it needs, without all the bulging veins and other side effects (er, features) of larger sites like Rally.

The only thing I’d be hesitant to do with a tool like this is turn over product decisions to the crowd. What makes Votebox work for Dropbox is that Dropbox has stayed true to their original product vision – a simple, 100% reliable way to backup files to the Internet and sync them across multiple computers. Feature requests outside that vision may be popular, but they would dilute the brand, causing more harm than good to the product.

Rather, I see Votebox as a tool to help talented Product Owners with strong visions for their products interact with their audiences.