Thursday, January 10, 2008

Converting tests to specs is a bad idea

When I first started experimenting with BDD, all the talk about the shift in language led me to believe that to "do BDD" all I needed to do was to change my "Asserts" to some "Shoulds".  At the root, it looked like all I was really doing was changing the order of my "expected" and "actual".

In my admittedly short experiences so far, I've found that BDD is much more than naming conventions, language, and some macros.  For me, unit testing was about verifying implementations while specifications were about specifying behavior.  Although I'm not supposed to test implementations with unit testing, conventions led me down this path.  It's easy to fall into the trap of testing implementations, given constraints we put on ourselves when writing unit tests.

Starting with tests

Typically, my unit tests looked something like this:

[TestFixture]
public class AccountServiceTests
{
    [Test]
    public void Transfer_WithValidAccounts_TransfersMoneyBetweenAccounts()
    {
        Account source = new Account();
        source.Balance = 100;

        Account destination = new Account();
        destination.Balance = 200;

        AccountService service = new AccountService();

        service.Transfer(source, destination, 50);
        
        Assert.AreEqual(50, source.Balance);
        Assert.AreEqual(250, destination.Balance);
    }
}

When I wrote the implementation test-first, I first got a requirement or specification from the business.  It sounded something like "We want to be able to transfer money between two accounts".

Before I started writing my tests, I had to figure a few things out.  Our naming conventions forced us into a path that made us choose where the behavior was supposed to reside, as we named our test fixtures "<ClassUnderTest>Tests".  In the example above, we're testing the "AccountService" class.  Additionally, our individual tests were named "<MethodName>_<StateUnderTest>_<ExpectedBehavior>".  We took these naming conventions because it organized the class behavior quite nicely according to members.

Converting to BDD syntax

I wanted to try BDD, so the quickest way I saw to do it was to change the names of our tests and switch around our assertions:

[TestFixture]
public class AccountServiceTests
{
    [Test]
    public void Transfer_WithValidAccounts_ShouldTransfersMoneyBetweenAccounts()
    {
        Account source = new Account();
        source.Balance = 100;

        Account destination = new Account();
        destination.Balance = 200;

        AccountService service = new AccountService();

        service.Transfer(source, destination, 50);

        source.Balance.ShouldEqual(50);
        destination.Balance.ShouldEqual(250);
    }
}

Now that I see the word "Should" everywhere, that means I'm doing BDD, right?

Just leave it alone

BDD is much more than naming conventions and the word "should", it's more about starting with a context, then defining behavior outside of any hint of an implementation.  When creating my Transfer test initially, before I could start writing ANY code, because of our naming conventions I had to decide two things:

  • What class does the behavior belong
  • What method name should be assigned to the behavior

But when writing true BDD-style specs, I don't care about the underlying class or method names.  All I care about is the context and specifications, and that's it!  If I was writing the Transfer behavior BDD-first, I might end up with this:

[TestFixture]
public class When_transfering_money_between_two_accounts_with_appropriate_funds
{
    [Test]
    public void Should_reflect_balances_appropriately()
    {
        Account source = new Account();
        source.Balance = 100;

        Account destination = new Account();
        destination.Balance = 200;

        source.TransferTo(destination, 50);

        source.Balance.ShouldEqual(50);
        destination.Balance.ShouldEqual(250);
    }
}

The key difference here is nowhere in the fixture nor the test method name will you find any mention of types, member names, or anything that hints at an implementation.  I'm driven purely by behavior, which led me to a completely different design than my test-first design.  In the future, if I decide to change the underlying implementation, I don't need to do anything with my BDD specs.  When I've decoupled my implementation of behavior completely from the specification of behavior, I can make much more dramatic design changes, as I won't be bound by my tests.

I've also found I don't modify specifications nearly as much as I used to modify tests.  If behavior is changed, I delete the original spec and add a new one.

So don't trick yourself into thinking that you need to modify your tests to become BDD-like.  Just leave those tests alone, they're doing exactly what they're designed to do.  A single context is likely split across many, many test fixtures, so it's just not an exercise worth undertaking.  Start your BDD specs fresh, unencumbered from existing test code, and the transition will be much, much easier.

7 comments:

Ray Houston said...

Jimmy, this has been a really helpful post for me towards understanding BDD. Thanks! Here is an inspired post.

chad said...

Jimmy, the example you posted looks more like an acceptance test than a unit test. Is this because a.) it was just a contrived example and I'm reading too much into it or b.) BDD is not concerned with unit tests (they are a separate thing), or c.) BDD suggests we don't do unit tests at all.

When I say "unit tests", I mean things like the example you showed, plus things like: Invalid accounts, null accounts, other edge cases.

Do we just not test every case because it's a "last mile" (100% problem) with diminishing returns, or what do you recommend for some of these more mundane, nuts-and-bolts testing scenarios?

Jimmy Bogard said...

@chad

Good question, to be followed by another post :)

Dave Laribee said...

I have to take issue with this post.

Sometimes class behaviors do specify implementation. For example, your spec there is really describing a composite behavior and should be broken down into two specifications (reduce the balance of the source, increase the balance of the target).

Not only that, but outside of model examples, what then? Could we end up with specifications on a class that is fairly technical where the implementation leaks into the language of the spec? Yes. I will try and post an example here as I realize I'm being nebulous.

Jimmy Bogard said...

@Dave

I'm really talking about the practice I've seen of converting existing tests, switching the language around, and assume it's a spec. I don't see any value messing with existing tests. They're there, leave them alone.

But my understanding of technical or implementations specs is still lacking, so please enlighten me!

Dave Laribee said...

Not sure I have enlightenment to offer; we're really coming at this as a community and it's exciting to see posts like this. I do not speak from a position of authority as my BDD career only goes about 800 specs deep at this point.

And I don't totally disagree with you. It's really not a matter of Jimmy vs. Dave and who's right, it's more about producing a dialog from which a deeper understanding emerges, so I hope you'll take my comment in that spirit!

That said, I did port tests to specs. I see the value there. I will promise to get a post that references yours and maybe supplies a subtle counter argument before week's end.

Colin Jack said...

Really good post, I'd definitely be interested in hearing more on the topic and how to use BDD at lower levels because thats one area that confuses me.