Authoring stories with NBehave 0.3

As Joe already mentioned, Behave# has merged with NBehave.  The merged NBehave will still be hosted on CodePlex, and the old project site will redirect to the new CodePlex NBehave site.  With the announcement of the merge comes the first release of our merging efforts (0.3), which you can find here.

The new release added quite a few features over the previous release, including:

• Pending scenarios
• Console runner
  • Decorate your tests with "Theme" and "Story" attributes
  • Scenario result totals
  • Dry run output

The major addition is the console runner feature.  One problem we always ran into was that as developers, we wanted to get pass/fail totals based on scenarios (not tests) so that we could have more meaningful totals that matched our original stories.  A single story could have several scenarios, but would only report as one "Pass" or "Fail" to NUnit.  Additionally, this is the first release that compares fairly evenly with the first release of rbehave.

So how can we use NBehave to accomplish this?  As always, we'll start with the story.

The Story

I've received a few comments for a different example than the "Account" example.  I'll just pull an example from Jimmy Nilsson's excellent Applying Domain-Driven Design and Patterns book (from page 118):

List customers by applying a flexible and complex filter.

Not story-friendly, but a description below lets me create a meaningful story:

As a customer support staff
I want to search for customers in a very flexible manner
So that I can find a customer record and provide them meaningful support.

I'm playing both developer and business owner, but normally these stories would be written by the customer.  Otherwise, so far so good.  What about a scenario for this story?  Looking further into the book, I found a suitable scenario on page 122.  Reworded in the BDD syntax, I arrive at:

Scenario: Find by name

Given a set of valid customers
When I ask for an existing name
Then the correct customer is found and returned.

Right now, I only care about finding by name.  It could be argued that the original story is too broad, but it will suffice for this example.  I'm confident those conversations will take place during iteration planning meetings in any case.

Now that we have a story and a scenario, I can author the story in NBehave.  First, I'll need to set up my environment to use NBehave.

Setting up the environment

I use a fairly common source tree for most projects:

All dependencies (i.e. assemblies in the References of my project) go into the "lib" folder.  All tools, like NAnt, NUnit, etc. that are used as part of the build go into the "tools" folder.  For NBehave, I've copied the "NBehave.Framework.dll" into the "lib" folder, and the entire NBehave release goes into its own folder in the "tools" folder.  For more information about this setup, check out Tree Surgeon.

Now that I have NBehave in my project, I'm ready to write some NBehave stories and scenarios.

The initial scenario

Before I get started authoring the story and scenario, I need to create a project for these scenarios.  If I have a project named MyProject, its scenarios will be in a MyProject.Scenarios project.  Likewise, its specifications will be in a MyProject.Specifications project.  You can combine the stories and specifications into one project if you like.  Finally, I create a class that will contain all of the stories in my "customer search" theme.

I don't name the class after the class it might be testing, instead I name it after the theme.  The reason is that the implementation of the stories and scenarios can (and will) change independently of the story and scenario definition.  Stories and scenarios shouldn't be tied to implementation details.

After adding a reference to NBehave and NUnit from the "lib" folder, Here's what my solution tree looks like at this point:

Note that I named my file after the theme, not the class I'm likely to test (CustomerRepository).  Here's my entire story file:

using NBehave.Framework;

namespace NBehaveExample.Core.Specifications
{
    [Theme("Customer search")]
    public class CustomerSearchSpecs
    {
        [Story]
        public void Should_find_customers_by_name_when_name_matches()
        {
            Story story = new Story("List customers by name");

            story.AsA("customer support staff")
                .IWant("to search for customers in a very flexible manner")
                .SoThat("I can find a customer record and provide meaningful support");

            story.WithScenario("Find by name")
                .Pending("Search implementation")

                .Given("a set of valid customers")
                .When("I ask for an existing name")
                .Then("the correct customer is found and returned");
        }
    }
}

A few things to note here:

• Theme classes are decorated with the Theme attribute
  • Themes have a mandatory title
• Story methods are decorated with the Story attribute
• The initial story is marked Pending, with an included reason

The attributes are identical in function to the "TestFixture" and "Test" attributes of NUnit, where they inform NBehave that this class is a Theme class and it contains Story methods.  NBehave finds classes marked with the Theme attribute, and executes methods marked with the Story attribute.

Now that we have a skeleton story definition in place, I can run the stories as part of my build

Using the console runner

New in NBehave 0.3 is the console runner, which runs the Themes and Stories and collects metrics from those runs.  To run the above stories, I use the following command:

NBehave-Console.exe NBehaveExample.Core.Scenarios.dll

From the console runner, I get the following output:

NBehave version 0.3.0.0
Copyright (C) 2007 Jimmy Bogard.
All Rights Reserved.

Runtime Environment -
   OS Version: Microsoft Windows NT 5.2.3790 Service Pack 1
  CLR Version: 2.0.50727.1378

P
Scenarios run: 1, Failures: 0, Pending: 1

Pending:
1) List customers by name (Find by name): Search implementation

I only have one scenario thus far, but NBehave tells me several things so far:

• Dot results
  • A series of one character results shows me I have one scenario pending (similar to the dots NUnit outputs)
• Result totals
  • Includes total scenarios run, number of failures, and number of pending scenarios
• Individual summary result
  • List of failing and pending scenarios
  • Name of story, scenario, and pending/failing reason

Let's say I want a dry-run of the scenario output for documentation purposes:

NBehave-Console.exe NBehaveExample.Core.Scenarios.dll /dryRun /storyOutput:stories.txt

I've set two switches for the console runner, one to do a dry run and one to have a file where the stories will be output.  Story output can be turned on regardless if I'm doing a dry run or not.  Here's the contents of "stories.txt" after I run the statement above:

Theme: Customer search

	Story: List customers by name
	
	Narrative:
		As a customer support staff
		I want to search for customers in a very flexible manner
		So that I can find a customer record and provide meaningful support
	
		Scenario 1: Find by name
			Pending: Search implementation
			Given a set of valid customers
			When I ask for an existing name
			Then the correct customer is found and returned

This output provides a nice, human-readable format describing the stories that make up my system.

Now that I have a story, let's make the story pass, using TDD with Red-Green-Refactor.

Make it fail

First, I'll add just enough to my story implementation to make it compile:

[Story]
public void Should_find_customers_by_name_when_name_matches()
{
    Story story = new Story("List customers by name");

    story.AsA("customer support staff")
        .IWant("to search for customers in a very flexible manner")
        .SoThat("I can find a customer record and provide meaningful support");

    CustomerRepository repo = null;
    Customer customer = null;

    story.WithScenario("Find by name")
        .Given("a set of valid customers",
            delegate { repo = CreateDummyRepo(); })
        .When("I ask for an existing name", "Joe Schmoe",
            delegate(string name) { customer = repo.FindByName(name); })
        .Then("the correct customer is found and returned",
            delegate { Assert.That(customer.Name, Is.EqualTo("Joe Schmoe")); });
}

All I've done is removed the Pending call on the scenario and added the correct actions for the Given, When, and Then fragments.  The "CreateDummyRepo" method is just a helper method to set up a CustomerRepository:

private CustomerRepository CreateDummyRepo()
{
    Customer joe = new Customer();
    joe.CustomerNumber = 1;
    joe.Name = "Joe Schmoe";

    Customer bob = new Customer();
    bob.CustomerNumber = 1;
    bob.Name = "Bob Schmoe";

    CustomerRepository repo = new CustomerRepository(new Customer[] { joe, bob });

    return repo;
}

I compile successfully and run NBehave, and get a failure as expected:

F
Scenarios run: 1, Failures: 1, Pending: 0

Failures:
1) List customers by name (Find by name) FAILED
  System.NullReferenceException : Object reference not set to an instance of an object.
   at NBehaveExample.Core.Specifications.CustomerSearchSpecs.<>c__DisplayClass3.b__2() in C:\dev\NBehaveExample\src\NBehaveExample.Core.Specifications\CustomerSearchSpecs.cs:line 28
   at NBehave.Framework.Story.<>c__DisplayClass1.b__0()
   at NBehave.Framework.Story.InvokeActionBase(String type, String message, Object originalAction, Action actionCallback, String outputMessage, Object[] messageParameters)

Now that I've made it fail, let's calibrate the test and put only enough code in the FindByName method to make the test pass.

Make it pass

To make the test pass, I'll just return a hard-coded Customer object:

public Customer FindByName(string name)
{
    Customer customer = new Customer();
    customer.Name = "Joe Schmoe";
    return customer;
}

NBehave now tells me that I have 1 scenario run with 0 failures:

.
Scenarios run: 1, Failures: 0, Pending: 0

The dot signifies a passing scenario.  Now I can make the code correct and put in some implementation.

Make it right

Since CustomerRepository is just sample code for now, it only uses a List<Customer> for its backing store.  Searching isn't that difficult as I'm not involving the database at this time:

public Customer FindByName(string name)
{
    return _customers.Find(delegate(Customer customer) { return customer.Name == name; });
}

With this implementation in place, NBehave tells me I'm still green:

.
Scenarios run: 1, Failures: 0, Pending: 0

I can now move on to the next scenario.  If I had additional specifications for CustomerRepository not captured in the story, I can go to my Specifications project to detail them there.

Where we're going

With NBehave's console runner, I can now easily include NBehave as part of my build.  I'm not piggy-backing NUnit for executing and reporting tests, as I'm writing Stories and Scenarios, not Tests.  This option is still available to me and I can create stories inside of tests, so we're not forcing anyone to use the Theme and Story attributes if they don't want to.

It's a good start, but there are a few things still lacking:

• Integration with testing/specification frameworks
  • The story for authoring the "Then" part of the scenario still isn't that great
• Features targeted for automated builds/CI
  • XML output
  • A nice XSLT formatter for XML output
  • HTML output of stories in addition to raw text
  • Integration with CC.NET
• Setup/Teardown for stories/themes, with appropriate BDD names
  • Not sure, since having everything encapsulated in the story can direct my API better

Happy coding!