On Clarity and Abstraction in Functional Tests

Consider the following tests:

[Test]
public void LoginFailsForUnknownUser1()
{
    string username = "unknown";
    string password = "password";

    bool loginSucceeded = User.Login(username, password);

    Assert.That(loginSucceeded == false);
}
[Test]
public void LoginFailsWithUnknownUser2()
{
    using (var browser = new IE(url))
    {
        browser.TextField(Find.ById(new Regex("UserName"))).Value = "unknown";
        browser.TextField(Find.ById(new Regex("Password"))).Value = "password";

        browser.Button(Find.ById(new Regex("LoginButton"))).Click();
        bool loginSucceeded = browser.Url.Split('?')[0].EndsWith("index.aspx");

        Assert.That(loginSucceeded == false);
    }
}

Note the similarities:

  • Both methods test the same underlying functional code; and,
  • Both tests are written in NUnit.
  • Both tests use the Arrange / Act / Assert structure.

Note the differences:

  • The first is a unit test for a method on a class.
  • The second is a functional test that tests an interaction with a web page.
  • The first is clear. The second is, um, not.

Abstracting away the browser interaction

So, what’s the problem? Aren’t all browser tests going to have to use code to automate the browser?

Well, yes. But, why must that code be so in our face? How might we express the true intention of the test without clouding it in all the arcane incantations required to automate the browser?

WatiN Page Classes

The folks behind WatiN answered that question with something called a Page class. Basically, you hide all the browser.TextField(…) goo inside a class that represents a single page on the web site. Rewriting the second test using the Page class concept results in this code:

[Test]
public void LoginFailsWithUnknownUser3()
{
    using (var browser = new IE(url))
    {
        browser.Page<LoginPage>().UserName.Value = "unknown";
        browser.Page<LoginPage>().Password.Value = "password";

        browser.Page<LoginPage>().LoginButton.Click();
        bool loginSucceeded = browser.Page<IndexPage>().IsCurrentPage;

        Assert.That(loginSucceeded == false);
    }
}
public class LoginPage : Page
{
    public TextField UserName
    {
        get { return Document.TextField(Find.ById(new Regex("UserName"))); }
    }
    public TextField Password
    {
        get { return Document.TextField(Find.ById(new Regex("Password"))); }
    }
    public Button LoginButton
    {
        get { return Document.Button(Find.ById(new Regex("LoginButton"))); }
    }
}

Better? Yes. Now, most of the WatiN magic is tucked away in the LoginPage class. And, you can begin to make out the intention of the test. It’s there at the right hand side of the statements.

But, to me, the Page Class approach falls short. This test still reads more like its primary goal is to automate the browser, not to automate the underlying system. Plus, the reader of this test needs to understand generics in order to fully grasp what the test is doing.

Static Page Classes

An alternative approach I’ve used in the past is to create my own static classes to represent the pages in my web site. It looks like this:

[Test]
public void LoginFailsWithUnknownUser4()
{
    using (var browser = new IE(url))
    {
        LoginPage.UserName(browser).Value = "unknown";
        LoginPage.Password(browser).Value = "password";

        LoginPage.LoginButton(browser).Click();
        bool loginSucceeded = IndexPage.IsCurrentPage(browser);

        Assert.That(loginSucceeded == false);
    }
}
public static class LoginPage
{
    public static TextField UserName(Browser browser)
    {
        return browser.TextField(Find.ById(new Regex("UserName")));
    }
    public static TextField Password(Browser browser)
    {
        return browser.TextField(Find.ById(new Regex("Password")));
    }
    public static Button LoginButton(Browser browser)
    {
        return browser.Button(Find.ById(new Regex("LoginButton")));
    }
}

This is the closest I have come to revealing the intention behind the functional test without clouding it in all the arcane incantations necessary to animate a web browser. Yes, there are still references to the browser. But, at least now the intention behind the test can be inferred by reading each line from left to right. Furthermore, most of the references to the browser are now parenthetical, which our eyes are accustomed to skipping.

What do you think?

I’d like to know what you think. Are your functional tests as clear as they could be? If so, how’d you do it? If not, do you think this approach might be helpful? Drop me a line!

Using NUnit to run 32-bit tests on a 64-bit platform

Esoteric? Yes. But, these are the problems I run into…

I recently installed the 64-bit version of Windows Vista in order to gain access to all 4GB of memory on my computer. (32-bit versions of Windows can only access about 3.5GB.) But, when I tried to run some NUnit tests, I started running into problems.

First, my tests use the Microsoft Jet driver to access a Microsoft Excel spreadsheet. Unfortunately, there is no 64-bit version of the Jet driver. So, when I tried to execute my tests, they failed to load the driver.

64-bit Vista does come with a 32-bit version of the Jet driver. In order to access the 32-bit driver, your code must be 32-bit. So, I told Visual Studio to use the x86 configuration rather than the “Any CPU” configuration.

But, once I recompiled, NUnit refused to load my 32-bit DLL. It claimed that it could not load the DLL or one of its references:

Turns out that NUnit was also compiled with the “Any CPU” option. So, when it launches on my 64-bit computer, it runs as a 64-bit app. This means that it cannot load 32-bit DLLs. Thus, the error.

The solution was to use a utility called CORFLAGS to add a 32-bit flag to NUnit.exe. Here’s the command I used:

corflags nunit.exe /32BIT+

Here’s a link to the article I found explaining this: http://geekswithblogs.net/Lance/archive/2006/12/28/102191.aspx.