Improving Tests One Name at a Time

Let’s take a look at a simple object and a typical test. The code examples here are PHP, but this advice applies to any language or testing framework.

<?php
class Truty
{
    public function isTruthy($value)
    {
        return filter_var($value, FILTER_VALIDATE_BOOLEAN);
    }
}

class TruthyTest extends \PHPUnit_Framework_TestCase
{
    public function testIsTruthy()
    {
        $truthy = new Truthy();
        $this->assertTrue($truthy->isTruthy("yes"));
        $this->assertFalse($truthy->isTruthy("no"));
    }
}

Looks great right? I would argue that the test, while valid, is not as useful as it could be because the name of the test is terrible.

Test Names and Behaviors

Test method names should do more than tell a reader (or the test framework) what method is under test. A test method name should describe what’s going on in the test itself: it should describe the expected behavior of the object.

Think of a description in prose of what the object should do, then use that to improve the tests:

Truthy::isTruthy should return true for the following truthy values: “true”, “yes”, “on”, “1”, 1, and `true`. It should return false for anything else.

That’s a bit wordy for a test method name, but we can translate into something that’s useful and makes sense.

If we analyze the behavior under test, its easy to see two situations: a whitelist of truthy values should return `true` from `Truthy::isTruthy`. Everything else should return false. Rather than testing both of those behaviors in a single test we should split it into two descriptive tests. Additionally, the list of truthy values implies we should test both scenarios with multiple
values.

<?php
class TruthyTest extends \PHPUnit_Framework_TestCase
{
    private $truthy;

    public function truthyValueProvider()
    {
        return array(
            array("yes"),
            array("1"),
            array("true"),
            array(1),
            array(true),
        );
    }

    /**
     * @dataProvider truthyValueProvider
     */
    public function testIsTruthyReturnsTrueForTrutyValues($value)
    {
        $this->assertTrue($this->truthy->isTruthy($value));
    }

    public function falsyValueProvider()
    {
        return array(
            array("no"),
            array("false"),
            array("0"),
            array(0),
            array(false),
            array(null),
            array("not yes")
        );
    }

    /**
     * @dataProvider falsyValueProvider
     */
    public function testIsTruthyReturnsFalseForNonTruthyValues($value)
    {
        $this->assertFalse($this->truthy->isTruthy($value));
    }

    protected function setUp()
    {
        $this->truthy = new Truthy();
    }
}

This is pretty close to SpecBDD style testing:

describe("Truthy#isTruthy", function () {
    it("should return true for truthy values", function () {
        // ...
    });

    it("should return false for non truthy values", function () {
        // ...
    });
});

The benefit here is that test failure messages are descriptive. Just by looking at the test method name a developer would know what featured failed. The addition of data providers gives improves the test as well: if one of those fails we’ll get a nice descriptive message about the specific value that failed along with the feature.

It’s a Test’s Job to Fail

Tests make sure our code works, but that’s not really a test’s job. A test’s job is to fail when something is broken. With that in mind, we should strive to make sure those failures are as useful as possible.