Tests Brittleness and Verbosity slows you down

In entry/testing-makes-you-faster-day-one Testing Makes You Faster Day one article I claimed that writing tests makes you faster. However, there are test patterns that most likely will slow you down and you need to be ready to handle them.Writing tests for a simple logic with a simple input is a breeze. Take for example a test for a simple calculator class. Two numeric inputs and we are done: package com.example.portfolio; import org.junit.Assert; import org.junit.Test; public class SimpleCalculatorTest { @Test public void zeroMultiply() { SimpleCalculator simpleCalculator = new SimpleCalculator(); Assert.assertEquals(0, simpleCalculator.multiply(100.0, 0.0), 0.0); Assert.assertEquals(0, simpleCalculator.multiply(-100.0, 0.0), 0.0); } } However real apps require real domain, and real domain is rarely represented by a couple of numbers. Let's take Finance as an example. It is full of types with multiple fields such as Transaction , and it has a lot of business logic.One example of such business logic is ProfitCalculator . Its job is to calculate profit based on the executed Transactions .After consulting with the business people, we are ready to give our implementation a shot. Transaction is a data class with a few fields (number of fields is reduced for simplicity). package com.example.portfolio; import javax.persistence.Entity; import javax.persistence.Id; @Entity public class Transaction { private String id; private double lot; private double price; private String symbol; @Id public String getId() { return id; } public void setId(String id) { this.id = id; } public double getLot() { return lot; } public void setLot(double lot) { this.lot = lot; } public double getPrice() { return price; } public void setPrice(double price) { this.price = price; } public String getSymbol() { return symbol; } public void setSymbol(String symbol) { this.symbol = symbol; } } Here is our first ProfitCalculator implementation, ready to be tested. package com.example.portfolio; import java.util.List; public class ProfitCalculator { public ProfitCalculator() { } public double calculate(List<Transaction> transactions) { return 0; } } And here is the first version of our test. package com.example.portfolio; import org.junit.Test; import java.util.Arrays; import static org.junit.Assert.assertEquals; public class ProfitCalculatorWithoutEncapsulationTest { private ProfitCalculator profitCalculator = new ProfitCalculator(); @Test public void profitShouldBeZeroIfNoLotsSet() { Transaction t1 = new Transaction(); t1.setId("T1"); t1.setSymbol("SYM.B"); t1.setLot(0); t1.setPrice(8); Transaction t2 = new Transaction(); t1.setId("T2"); t1.setSymbol("SYM.C"); t1.setLot(0); t1.setPrice(19); assertEquals(0, profitCalculator.calculate(Arrays.asList(t1, t2)), 0); } } This test is already on the verbose side. As requirements for the ProfitCalculator evolve, the number of setters we need to validate new business logic will grow, as will grow the number of instances that we may need to create.Question: But why verbose test can be bad? A verbose test can be bad, because the boilerplate code used to setup the test makes it harder to read intent behind the test.Additionally it may discourage you to write separate test scenario and instead you may want to clump test scenarios together and re-use the test data setup.Test data re-use can lead to a random test being broken as you setup data for an unrelated scenario. Verbosity is not the only potential problem here. The way Transaction instance is being constructed for testing purposes can lead to a maintenance burden.The First iteration of Transaction class uses setters to set the data. A future iteration may switch to using fluent API instead of setters . transaction("T1") .symbol("SYM.B") .lot(0) .price(0); From the ProfitCalculator 's perspective the way Transaction instances are created is irrelevant as it doesn't affect the calculation logic. But refactoring Transaction will break ProfitCalculatorTest . Transaction is going to be used in other tests. As the number of tests that use Transaction instances increases you will be more and more reluctant to do refactoring. All your tests that create Transaction instances are brittle tests now. They are brittle because they won't survive Transaction refactoring.If you ever want to refactor Transaction class you will have two choices:Refactor and waste time fixing tests.Convince yourself that Transaction is good as it is.Neither is a good choice.I want you to have a way to protect your future self by encapsulating test input preparation

Test Input Initial Encapsulation

Let's extract Transaction creation code into TestTransactions package com.example.portfolio; import org.junit.Test; import java.util.Arrays; import static com.example.portfolio.TestTransactions.createTransaction; import static org.junit.Assert.assertEquals; public class ProfitCalculatorWithBasicEncapsulationTest { private ProfitCalculator profitCalculator = new ProfitCalculator(); @Test public void profitShouldBeZeroIfNoLotsSet() { Transaction t1 = createTransaction("t1", "SYM.B", 0, 8); Transaction t2 = createTransaction("t2", "SYM.C", 0, 19); assertEquals(0, profitCalculator.calculate(Arrays.asList(t1, t2)), 0); } } public static Transaction createTransaction(String id, String symbol, double lot, double price) { Transaction transaction = new Transaction(); transaction.setId(id); transaction.setSymbol(symbol); transaction.setLot(lot); transaction.setPrice(price); return transaction; } Less verbosity. A good start.Question: Can you still spot the problem? As we add more properties to Transaction class, some of the tests may need to set extra transaction properties.Additionally some of the properties will be completely irrelevant to the business logic under a test, yet you will be forced to set them anyway.

Test Input Encapsulation With Webtau Table Data

I was dealing with test problems like above for years and eventually came up with a solution that fits my needs. I hope it will fit your needs as well. The solution is to use flexible data structure like https://twosigma.github.io/webtau/guide/reference/table-data Webtau TableData package com.example.portfolio; import com.twosigma.webtau.data.table.TableData; import org.junit.Test; import static com.example.portfolio.TestTransactions.createTransactions; import static com.twosigma.webtau.WebTauCore.*; // table and underscores are defined in WebTauCore public class ProfitCalculatorWithTableDataTest { private ProfitCalculator profitCalculator = new ProfitCalculator(); @Test public void profitShouldBeZeroIfNoLotsSet() { TableData transactionsData = table( "id", "symbol", "lot", "price", // webtau TableData definition ________________________________, // header and values separator "t1", "SYM.B" , 0.0 , 8.0, "t2", "SYM.C" , 0.0 , 19.0); double margin = profitCalculator.calculate(createTransactions(transactionsData)); actual(margin).should(equal(0)); } } Note: https://github.com/twosigma/webtau Webtau open source project started as my answer to common testing problems I encountered. While I am not working at https://www.twosigma.com/ Two Sigma anymore I am still contributing and using it on a regular basis. public static List<Transaction> createTransactions(TableData tableData) { return tableData.rowsStream() .map(row -> { Transaction transaction = new Transaction(); transaction.setId(row.get("id", genTransactionId())); // default value is auto generated transaction.setSymbol(row.get("symbol")); transaction.setLot(row.get("lot", 0.0)); // default value is hardcoded transaction.setPrice(row.get("price", 0.0)); return transaction; }) .collect(toList()); } private static String genTransactionId() { return "t-id-" + idCount.incrementAndGet(); } Notice how createTransactions defaults values when they are not present? As a result of this, tests that don't need, say, id or lot are free to ignore them. @Test public void profitShouldBeZeroIfNoLotsSet() { TableData transactionsData = table("symbol", "lot", ________________, "SYM.B" , 0.0, "SYM.C" , 0.0); double margin = profitCalculator.calculate(createTransactions(transactionsData)); actual(margin).should(equal(0)); } If later a new required property will be added to Transaction , you won't have to change all your existing tests. Instead you will update createTransaction with a new default value. The only time you will have to change the tests to use the new property is if the new property is affecting the logic under test.Essentially your test will use as little data to prove logic works as possible.Question: Why it is important to minimize the data in your test? Test is crucial to understanding complex business logicThe more data the harder it to comprehend the logicWe want business people to look at our data sets (stay tuned to see a good way to expose test data to business)

Try it out

<dependency> <groupId>com.twosigma.webtau</groupId> <artifactId>webtau-core</artifactId> <version>1.18</version> <scope>test</scope> </dependency> https://twosigma.github.io/webtau/guide/ Webtau user guide https://github.com/twosigma/webtau Webtau github

Summary

Tests input preparation can be verbose and brittle.Identify core domain objects and provide a convenient way to create them.Use as little data as necessary to prove logic correctness. Default the rest.