My approach to agile testing...

I’ve talked about agile testing before, here, here and here. But, a recent thread on the Alt.Net Seattle Google Group got me thinking about it again. Here’s the response I sent to the thread:

Testing is a huge domain. If you’re familiar with Marick’s testing quadrant, you know that there are four basic areas that testing covers:

  • Business Facing tests in Support of Programming (Business Requirements testing – Does the code do what it should?)
  • Business Facing tests to Critique the Product (Business Defect testing – Does the code do something it shouldn’t? Are there missing requirements?)
  • Technology Facing tests in Support of Programming (Technical Requirement testing – Does this method do what the developer intended?)
  • Technology Facing tests to Critique the Product (Technical defect testing – Are there leaks? Can it handle a load? Is it fast enough?)

Typically, testers focus on the business facing tests. And, people with specialized technical skills focus on the technology facing tests. (Developers on the support programming side; Performance testers on the critique product side.)

None of these tests can be run before the software is written. But, the tests in support of technology can be written before the code. And, metrics for perf/load/stress can be defined before the code is written. I recommend doing all of that (unless perf/load/stress isn’t important to you). Obviously, exploratory testing is something that has to wait for the code to be written.

If I were designing an agile team from scratch, I would propose the following approach:

During planning:

  • Track requirements as user stories.
  • Document acceptance criteria with each story, including perf/load/stress criteria (on the back of the 3x5 card, in Rally or TFS, etc.)

During an iteration:

  • One pair works on one story at a time.
  • Acceptance tests are automated first, based on acceptance criteria.
  • Code is written using TDD
  • Story is not functionally complete until all acceptance tests are passing (for the right reasons – no hard coded answers left)

After story is functionally complete:

  • Original pair leverages existing acceptance tests in perf/load/stress tests to determine if those criteria are met.
  • Tweak code as necessary to meet perf/load/stress acceptance criteria.
  • Story is not perf/load/stress complete until all perf/load/stress acceptance tests are passing

Exploratory testing should happen outside the constraints of a single story:

  • Limiting it to a single story would put blinders on that could negatively impact the effort. But, it is important that it happen.
  • Perhaps the team sets aside time during the day or iteration for banging on the software.

Once all acceptance tests are passing:

  • Ship it!


  1. Have the entire team bang out the acceptance tests at the beginning of the iteration.  I’ve seen this done. It works. But, quite often, tests get written for stories that end up getting cut from the iteration due to time constraints. That is excess inventory sitting on the production floor until those stories make it into another iteration. In other words, doing this encourages the accumulation of waste.
  2. If you’re concerned about a single pair working a story from beginning to end, mix it up. Give pairs one day to work on something, or 4 hours, or two, whatever works for you. Then switch things up – preferably by keeping one person on the story and bringing in a new pair. Then, the next time you switch, bring the older pair leaves.
  3. Even though exploratory testing should not be constrained by a single story, it really is important to do it before shipping the software. Microsoft calls this a bug bash. They give away prizes for the most bugs, and the hardest to find bugs. But, they don’t do it until very late in their process. It would be most agile to do it continuously.

How do you do agile testing?

HTML5 and CSS3 in Visual Studio 2010

As a web developer looking to stay current, I’ve been trying to do all of my personal projects in HTML5 and CSS3 for some time now. In fact, browser support is generally outstanding. Unfortunately, the lack of tooling support has made it a bit more difficult. But, now that’s all changed.

Assuming you’re running Visual Studio 2010 SP1, you can download and install the Web Standards Update for Microsoft Visual Studio 2010 SP1, which will add IntelliSense support for the latest updates to HTML, CSS, and JavaScript.

In case you’re looking for more information: here’s a link to the announcement, and a link to Scott Hanselman’s demo of the release.

Deadlines cause poor decisions to be made slowly

In an article published in The Washington Post on July 22, Daniel Carpenter discussed why our government always seems to be working down to the wire. He blames deadlines (both real and conjured), stating that the "eleventh-hour effect" tends to delay decision making until the last possible moment:

When deadlines are imposed, decisions and bargains that could happen more quickly — because of momentum or normal work flow — often end up getting put off until the last minute. Social scientists have referred to this as the "eleventh-hour effect," and we see it both in experiments and in real life.

Furthermore, as a Harvard researcher studying the effect of deadlines on the FDA drug approval process, Mr. Carpenter and his colleagues discovered that drugs approved close to the Congressionally imposed deadline tend to be recalled often than others:

A few years ago I joined some Harvard Medical School colleagues in examining the deadlines that, since 1992, Congress has placed upon the Food and Drug Administration's drug reviews. Our research found that medications approved right before these deadlines were considerably more likely to be pulled from the market or have significant warning labels attached later on.

In other words, deadlines cause poor decisions to be made slowly.

I must admit, as a proponent of agile software development practices such as continuous, iterative planning, I find more than a grain of truth in this statement — and not just within the bounds of our government. In fact, it has been my experience that deadlines rarely encourage people to complete their work early, as the task at hand tends to fill the time available. And, besides, experience has also taught me that deadlines often seem arbitrary, feel capricious, and demotivate teams. (Details in footnote.)

Granted, businesses need to be able to plan around software development projects. There are end-users to train, products to market, and systems to integrate. And, no matter how arbitrary it may seem to software developers, a deadline does provide a business with a stake in the ground around which to plan these other activities. In this sense, deadlines perform a valuable service to the business. However, due to the "eleventh-hour effect" and the rush to finish work on time, deadlines can also do quite a bit of harm to an organization's systems.

Therefore, my preference is to manage projects using a stack-ranked, estimated backlog of user-stories and team velocity. The backlog shows me what's complete, how much work remains, and in what order it will be completed. The team velocity tells me when I can expect a specific story to be completed.

So, if I have a specific business need to see story X completed by date Y, then I can make the appropriate adjustments to the stack-ranking of the backlog without putting undue pressure on the development team. Furthermore, because user-stories are discreet, end-to-end slices of functionality, I can deploy them as soon as there’s sufficient business value to offset the costs of deployment.

The only question I have is could this management strategy be effective in Congress?


  1. Early in my career, I worked for a man who threatened to fire his entire development team if we did not ship all the projects we were working on by the end of the month. This is what I mean by an arbitrary, capricious and demotivating deadline. Oh, we managed to meet the deadline. It took some serious corner cutting. But, we shipped the software. In the end, though, it didn’t matter. We all left the company within three months, anyway.
  2. Hat tip to Ezra Klein, also of The Washington Post, for [the reference to the Carpenter article](

The Transformation Priority Premise

I just read the most brilliant new idea that I’ve seen since I learned TDD and Refactoring ten years ago. Naturally, it comes from the brain of Uncle Bob Martin. It’s called “The Transformation Priority Premise.” I’ll describe it briefly here. But, if you want to really understand it, you should go read Uncle Bob’s blog yourself:

Ok… So, what is the Transformation Priority Premise?

If you are familiar with Test Driven Development, you know the phrase “red, green, refactor.” That phrase encapsulates the process of writing tests before you write the code:

  • Red = Write a failing test and watch it fail.
  • Green = Write just enough code to make the test pass.
  • Refactor = Remove any duplication in the code without changing it’s behavior (all tests should still pass).

And, if you’re familiar with Martin Fowler’s book “Refactoring,” you know that a refactoring is a simple change to the structure of the code that is guaranteed not to change the behavior of the code. These are things like:

  • Introduce Variable – Replaces a constant with a variable.
  • Extract Method – Extracts one of more lines of code from one method into another (new) method.
  • Rename – Renames a variable, method or class.

These changes are so easy to implement that we can actually build tools that do them for us. Visual Studio comes with a handful of built-in refactorings. ReSharper adds significantly to that list.

What Uncle Bob is proposing is that there are similarly simple “Transformations” that change the behavior of a code base. Furthermore, he states that there is an order or priority in which we should apply these transformations – from simple to complex. He proposes the following (admittedly incomplete and probably mis-ordered) list:

  • [] –> nil
  • nil –> constant
  • constant –> constant+
  • constant –> scalar
  • statement –> statements
  • unconditional –> if
  • scalar –> array
  • array –> container
  • statements –> recursion
  • if –> while
  • expression –> function
  • variable –> assignment

Don’t worry if that doesn’t make any sense. I really just blogged it so I’ll have a quick reference. But, what’s cool about them is that they can actually help us pick which tests we should write in what order. For example, if we decide to write a test that requires us to add an if statement to our code, that test requires the “unconditional –> if” transformation. Knowing that, we can try to think of other tests that rely solely on transformations higher up the list.

The ultimate reward is that by writing our tests using transformation priority order, we should not ever get to a point where one test causes us to rewrite an entire method. Furthermore, because these transformations are all relatively simple in and of themselves, smart tools vendors may be able to build support for them directly into tools like ReSharper.

I’ve probably just confused the heck out of you with all this. Suffice to say that Uncle Bob’s posts are excellent. They include source code that he uses to show you how not using the priority order causes pain, and how using the priority order resolves that pain.

What are you waiting for? Go read those posts!

(Oh, and for those of you unfamiliar with Uncle Bob’s work, he references something called a Kata several times. The term comes from martial arts. It refers to a movement that is practiced over and over and over until it becomes second nature. Think “wax on, wax off”. Uncle Bob is a big fan of solving the same problems over and over and over to gain new insights into the potential solutions. And, he’s published several Katas ranging from calculating a bowling score to identifying prime factors of an integer. They’re worth your time, too.)

iPad @ Home

The week after Thanksgiving, my wife called me at work from our treadmill at home:

“We need an iPad for the treadmill,” she puffed.

“Um, well… Okay!” I replied, trying not to sound too eager. “But, only since you’ve identified a use case.”

“Funny,” she huffed over the sound of her eyes rolling.

“Um… Okay. You’re serious?” I say, skeptically.

“Yes, as long as I can watch Netflix on it.”

“Sure. You can watch Netflix on it.”



“You pick it out. We’ll make that our big Christmas present to each other.”

“Really!?! You sure you want me choosing?”

“Yup. Just don’t, you know, go crazy.”

“Oh, well, of course not.”

Long story short, my household is now the proud owner of a shiny new iPad 16GB 3G. And, surprise! We love it. Jill’s been doing Netflix. I’ve been reading the news and surfing. The kids really like the Disney Read-Along™ story book versions of Toy Story and The Princess and the Frog. And, really, who wouldn’t love Angry Birds and Fruit Ninja? There’s just one problem…

We only have the one.

But, there’s no sense buying another one right now, since the iPad 2 should be here this spring.

Apple’s iPad finds a place in the enterprise

This article in MacWorld, today, reminded me of something I learned last week at a Microsoft technology briefing. During a discussion of Microsoft’s strategy related to the iPad, Microsoft’s representative asked how we’re using iPads today. I already knew we were developing sales tools and looking to find ways to make our applications accessible from the device. But, I didn’t know that our board is using iPads extensively. In fact, all of the documents for our last board meeting were distributed and reviewed on iPads. Everyone in the meeting had an iPad and could follow along in the documents as the presenter talked. And, apparently, even our septuagenarian board members loved it!

How else might we use iPads (or other tablet computers)? Clearly, those of us who write software every day won’t be using them for that. But, what about email and note taking in meetings? I’ll bet most managers in the company would gladly give up their laptop for an iPad, as long as it was able to open documents on SharePoint, crunch a few numbers and provide access to their email, and do it all securely.

What are your thoughts? Is there a job function around here that could benefit from a smaller, more portable computer?

Securing WCF Services: Preventing Unauthorized Access

Suppose you are writing a web service to perform a sensitive calculation. Your web service should only be accessible by authorized users and/or applications. Here’s my advice for configuring your service:

Use .NET to Your Advantage

.NET has built in features for ensuring that the code calling yours is authorized to do so. For example, take a look at this snippet of code:

[PrincipalPermission(SecurityAction.Demand, Role = "MyLocalSecurityGroup"]
public SearchResults Find(string contractNumber)

Notice the [PrinciplePermission] attribute. This attribute tells .NET that only principals who are members of the "MyLocalSecurityGroup" role are allowed to run this method (where a principal is a user/service account, and a role is a locally defined security group on the server). In other words, in order to run this method, the caller must be running under an account that is a member of the local security group specified.

Local Security Groups

By telling your code to check a security group on the local server, you get around the problem of having to care about the different test regions in your code. The local group on your DEV server will contain only those accounts which are authorized to access the service in DEV. Likewise, the TST and PRD local security groups will only contain accounts authorized to access TST and PRD respectively.

Domain Security Groups (NT Groups)

Of course, you don’t want to manage access at the local server level. Rather, you want to let User Account Services (UAS) manage access centrally. The way to do this is to create domain security groups for each test level. For the middleware, I created these groups.

  • MyDomainSecurityGroup.DEV
  • MyDomainSecurityGroup.TST
  • MyDomainSecurityGroup.PRD

Then update the local security groups on your servers to ONLY contain one of these domain security groups. If the server is a DEV server, put the DEV make the DEV domain group the only member of the local security group. All other user/service accounts should be removed from the local security group and added to the appropriate domain security group.


If you follow this pattern to secure your web services, you’ll be able to prevent unauthorized access to your service by adding a single line of code to every method that needs to check authorization. Furthermore, in a more complex scenario – say one where users may have read, write or admin access to your service – you can create as many local and domain security groups as you need to ensure that users and applications have the lowest level of access required to do their work.

One final thought: If you web service is hosted in across a pool of servers, remember to create the local security groups on all of the servers. You could probably do this with your installer. Or, you could make it a manual step in your implementation plan that only gets executed once.

Friday funnies

Ok. This thing completely nailed me – right down to the “unavoidable stubble-beard!” How about you? My only quibble is that the infographic designer saw $85,430 as a full cup. Maybe there’s a bigger cup somewhere – call that the venté employer.

ReSharper Tip #123: Creating new classes and moving them to their own file

As I was writing all the source code for my last post, I relied heavily on ReSharper: first to generate classes for me, then to move them to a new file named the same thing as the class. This is a particularly handy technique if you practice Test Driven Development. Here’s how:

When I write a test, I write the whole thing, even though some of the classes may not exist, like this:


By placing my cursor on the class name (the red text) and pressing ALT+Enter, I receive the following ReSharper menu:


This generates the class in the same file as the test fixture. This is convenient at first. But, it’ll be impossible to find later, so I use ReSharper to move the class to a new file, again by placing the cursor on the class name and pressing ALT+Enter:


Which results in this:


Me like!