Or: Safety First
A reader of these posts wrote to me recently to point out a problem. The assertions I used, she wrote, do not follow the Guidewire Cloud Standards for unit testing. She’s right. They don’t. And they’re less readable and more costly to maintain because of it. In this post, I’ll unpack the subtle effects that assertions have on readability and maintainability. Then I’ll rewrite my tests so that they comply with Guidewire Cloud Standards.
In my previous blog post, I argued that good unit tests provide fast feedback on code changes. Every change we make risks breaking something, and sometimes things break in unexpected ways. So we need a safety net to catch us when we slip up. Our safety net is our suite of unit tests. But a test must do more than simply fail when a change breaks something: it must tell us why it failed so we know how to repair the damage.
The Wrong Assertions Can Create Confusion and Distrust
Let’s look again at a GUnit test from my post on stubs:
function testThatMissingAddressLine1InRequestThrowsIllegalArgumentException()
{
// GIVEN
var stubRequest = new AddressLookupRequest() {
override property get AddressLine1() : String {
return ""
}
override property get PostalCode() : String {
return "02840"
}
}
// WHEN
var t : Throwable
try {
new AddressLookup().doLookup(stubRequest)
} catch (e : Exception) {
t = e
}
// THEN
assertTrue("Exception thrown is not an IllegalArgumentException",
t typeis IllegalArgumentException)
}
This test has a single assertion in the // THEN section. Looking at the second argument of the assertion, we see that it evaluates to true when the address lookup throws an IllegalArgumentException. When this happens, execution continues, the test completes, and the test framework reports the test as passed.
If, on the other hand, the address lookup doesn’t throw an exception, or if it throws an exception that is not of type IllegalArgumentException, the assertion evaluates to false and throws an AssertionFailedError. When this happens, test execution stops and the test framework reports the test as failed. The test framework also displays the explanatory message we pass as the first argument to the assertion: “Exception thrown is not an IllegalArgumentException.”
This assertion is fine as far as it goes, but it presents two problems. The first is that the assertion is not at the right level of abstraction. It’s an example of what Lasse Koskela (in his book Effective Unit Testing: A guide for Java developers) calls a primitive assertion. “A primitive assertion,” writes Koskela, “is one that uses more primitive elements than the behavior it’s checking.” The primitive elements here are the language operator typeis and the concepts of true and false. These are problematic because they obscure the intent of the test and create cognitive work that readers of our tests shouldn’t need to do.
To lighten the cognitive load, we add an explanatory message as above. This message also helps debug a failure, because without it the test framework would display no information to help developers understand the failure. Developers would have to go to the test and read the assertion to understand why the test failed. Again, this is extra work that they shouldn’t need to do.
But while an explanatory message helps developers, it incurs a maintenance cost. And this is the second problem with the assertion. If the assertion is refactored, we must remember to refactor the message as well. This is often overlooked. And when it is, we have an assertion with a condition and a message that say different things — perhaps even contradictory things. When readers of the tests uncover this sort of ambiguity, they lose faith in our tests.
Fluent Assertions Create Trust and Safety
We can address these problems and safeguard trust in our tests by using fluent assertions. Fluent assertions are an example of a fluent interface, a design practice that has become popular in the last two decades. A fluent interface uses method names to create a domain-specific language (DSL) and chains method calls to make code read more like natural language.
Here’s my GUnit test rewritten to use fluent assertions:
uses gw.testharness.v3.PLAssertions#assertThat(Throwable)
…
function testThatMissingAddressLine1InRequestThrowsIllegalArgumentException()
{
// GIVEN
var stubRequest = new AddressLookupRequest() {
override property get AddressLine1() : String {
return ""
}
override property get PostalCode() : String {
return "02840"
}
}
// WHEN
var t : Throwable
try {
new AddressLookup().doLookup(stubRequest)
} catch (e : Exception) {
t = e
}
// THEN
assertThat(t).isInstanceOf(IllegalArgumentException)
}
The assertion in this test is expressed in terms of the DSL for assertions — specifically “assert that” and “is instance of” — which brings it to the right level of abstraction and makes its intent clear. And method chaining makes it easier to read. This means we no longer need the message in our code.
Moreover, we no longer need the message to help developers debug a failure. When this assertion fails, the test framework displays a useful message by default. If the test fails because the exception thrown is a NullPointerException, for example, the test framework displays the following message:
java.lang.AssertionError: expected instance of:<java.lang.IllegalArgumentException> but was instance of:<java.lang.NullPointerException>
This tells developers exactly what caused the failure: we were expecting one thing but got another. We can still add a custom message to our fluent assertion if we wish, but (as explained above) this incurs a maintenance cost, and the default message is usually enough to tell developers what they need to know. If developers need more context, they can use the names of the test class and test function, which are in the stack trace for the assertion error, which is also displayed by the test framework:
at org.fest.assertions.Fail.failure(Fail.java:228)
at org.fest.assertions.Assert.failure(Assert.java:149)
at org.fest.assertions.ObjectAssert.isInstanceOf(ObjectAssert.java:56)
at org.fest.assertions.ThrowableAssert.isInstanceOf(ThrowableAssert.java:75)
at acme.cc.integration.addresslookup.WhenLookingUpAddressTest.testThatMissingAddressLine1InRequestThrowsIllegalArgumentException(WhenLookingUpAddressTest.gs:64)
…
The combination of default failure message and expressive names for test functions and test classes clearly communicates the reason for the failure in terms of the technical behavior being tested. When we look up an address, we test that missing address line 1 in the request throws an IllegalArgumentException, but we got a NullPointerException instead. The developer knows what went wrong, why it went wrong, and where to go to fix it — all without needing to read the test code.
It’s Your Turn to Put Fluent Assertions to the Test
In my post on writing maintainable unit tests, I drew heavily on concepts from John Ferguson-Smart’s book BDD in Action: Behavior-driven development for the whole software lifecycle. I draw on that essential resource again to emphasize the benefits that fluent assertions bring to our tests (p.297):
Fluent assertion libraries are in no way specific to BDD [behavior-driven development], and they can be used to make any unit tests easier to understand. But their emphasis on readability, expressiveness, and communication makes them well aligned with the BDD philosophy.
My GUnit test rewritten to use fluent assertions is more readable, communicates its intent more clearly, and is more maintainable. In short, it’s a stronger knot in our safety net. More importantly, however, it’s compliant with the Guidewire Cloud Standards for unit testing (login required). But readers shouldn’t take my word for it. They should check out the standard for themselves. And if there’s still a problem, please drop me a line.
About the Author
Rob Kelly
Senior Engineering Manager