Code Coverage VS Data Coverage

Code Coverage helps to reveal what paths of business logic you forgot to exercise. But it doesn't help as much with HTTP API testing: a single call to an API may exercise the entire code path.One of the main artifacts of HTTP API testing is a response. Does response have correct data? Are there data paths that we never asserted on?WebTau looks across all the executed tests and called HTTP API methods to track what calls have response fields that were never observed. At the end report provides Data Coverage you can act on:Remove fields from the response Add an extra assertion

Detect Fields That Were Skipped

Let's consider an end-point that returns a weather temperature in Fahrenheit and Celsius. And Later we decided to add time field to the response.Our tests were asserting on both temperature fields, but we have no assertions on time field. Groovy package scenarios.rest.coverage import static org.testingisdocumenting.webtau.WebTauGroovyDsl.* scenario("check fahrenheit temperature") { http.get("/city/NewYork") { weather.temperatureF.shouldBe > 80 } } scenario("check celsius temperature") { http.get("/city/NewYork") { weather.temperatureC.shouldBe > 26 } } Java package com.example.tests.junit5; import org.junit.jupiter.api.Test; import org.testingisdocumenting.webtau.junit5.WebTau; import static org.testingisdocumenting.webtau.WebTauDsl.*; @WebTau public class NewYorkWeatherJavaTest { @Test public void checkFahrenheitTemperature() { http.get("/city/NewYork", (header, body) -> { body.get("weather.temperatureF").shouldBe(greaterThan(80)); }); } @Test public void checkCelsiusTemperature() { http.get("/city/NewYork", (header, body) -> { body.get("weather.temperatureC").shouldBe(greaterThan(26)); }); } } When we run our test suite, we will see in the output that both temperatures were asserted ( ~~ in the response). But since we didn't touch time field, at the end of test run, WebTau will print a warning about missing validation scenario checkFahrenheitTemperature (NewYorkWeatherJavaTest) > executing HTTP GET http://localhost:42621/city/NewYork > reading HTTP routes definition from class path resource data/http-routes.txt . read HTTP routes definition from class path resource data/http-routes.txt (1ms) > mapping operation id . mapped operation id as "GET /city/:cityId" (1ms) . body.weather.temperatureF greater than 80 (0ms) . header.statusCode equals 200 (1ms) response (application/json): { "time": "2018-11-27 13:05:00", "weather": {"temperatureF": ~~88~~, "temperatureC": 31} } . executed HTTP GET http://localhost:42621/city/NewYork (9ms) [.] checkFahrenheitTemperature (NewYorkWeatherJavaTest) scenario checkCelsiusTemperature (NewYorkWeatherJavaTest) > executing HTTP GET http://localhost:42621/city/NewYork > mapping operation id . mapped operation id as "GET /city/:cityId" (0ms) . body.weather.temperatureC greater than 26 (0ms) . header.statusCode equals 200 (0ms) response (application/json): { "time": "2018-11-27 13:05:00", "weather": {"temperatureF": 88, "temperatureC": ~~31~~} } . executed HTTP GET http://localhost:42621/city/NewYork (4ms) [.] checkCelsiusTemperature (NewYorkWeatherJavaTest) scenario after all tests (Teardown) [.] after all tests (Teardown) Data Coverage HTTP routes that have non validated response fields GET /city/:cityId root.time WebTau prints only the first three routes with the skipped fields, and only first three fields. More details is available in the produced HTML report.HTTP Data Coverage leads to more details on Routes that have fields that were skipped Expanding a Route will list all the ignored fields

Routing

WebTau needs to know how to group URLs you call. In the example above /city/NewYork and /city/London belongs to the same group. And if you validate Celsius field when calling NewYork and Fahrenheit when calling London, fields will be considered as covered.This is because WebTau identifies both URLs as /city/:id scenario checkFahrenheitTemperature (NewYorkWeatherJavaTest) > executing HTTP GET http://localhost:42621/city/NewYork > reading HTTP routes definition from class path resource data/http-routes.txt . read HTTP routes definition from class path resource data/http-routes.txt (1ms) > mapping operation id . mapped operation id as "GET /city/:cityId" (1ms) . body.weather.temperatureF greater than 80 (0ms) There are two ways to provide routing information:Provide HTTP/openAPI-spec Open API spec Provide http routes text files

Text Routes

If you don't have HTTP/openAPI-spec Open API spec, you can define a plain text files that enumerates your API routes like this GET /city/:cityId Note: variant with curly brackets for parameters also works /city/{id} Specify file path (or resource for java) in your config file to enable it Groovy httpRoutesPath = "data/http-routes.txt" Java httpRoutesPath = data/http-routes.txt

JSON Output

Use config option to output Data Coverage in a separate JSON file for further processing. Groovy httpDataCoverageOutput = "generated/data-coverage.json" Java httpDataCoverageOutput = generated/data-coverage.json Data Coverage HTTP routes that have non validated response fields GET /city/:cityId root.time . generated HTTP Data Coverage: /home/runner/work/webtau/webtau/webtau-junit5-examples/generated/data-coverage.json (0ms) [ { "touchedPathsCount" : 2, "untouchedPathsCount" : 1, "untouchedPaths" : [ "root.time" ], "id" : "GET /city/:cityId", "untouchedPercent" : 33 } ]