
////
This guide is maintained in the main Quarkus repository
and pull requests should be submitted there:
https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc
////
= Measuring the coverage of your tests

include::./attributes.adoc[]

:toc: macro
:toclevels: 4
:doctype: book
:icons: font
:docinfo1:

:numbered:
:sectnums:
:sectnumlevels: 4


Learn how to measure the test coverage of your application. This guide covers:

* Measuring the coverage of your Unit Tests
* Measuring the coverage of your Integration Tests
* Separating the execution of your Unit Tests and Integration Tests
* Consolidating the coverage for all your tests

Please note that code coverage is not supported in native mode.

== Prerequisites

include::includes/devtools/prerequisites.adoc[]
* Having completed the xref:getting-started-testing.adoc[Testing your application guide]

== Architecture

The application built in this guide is just a JAX-RS endpoint (hello world) that relies on dependency injection to use a service.
The service will be tested with JUnit 5 and the endpoint will be annotated via a `@QuarkusTest` annotation.

== Solution

We recommend that you follow the instructions in the next sections and create the application step by step. However, you can go right to the completed example.
Clone the Git repository: `git clone {quickstarts-clone-url}`, or download an {quickstarts-archive-url}[archive].

The solution is located in the `tests-with-coverage-quickstart` {quickstarts-tree-url}/tests-with-coverage-quickstart[directory].

== Starting from a simple project and two tests

Let's start from an empty application created with the Quarkus Maven plugin:

:create-app-artifact-id: tests-with-coverage-quickstart
:create-app-extensions: resteasy
include::includes/devtools/create-app.adoc[]

Now we'll be adding all the elements necessary to have an application that is properly covered with tests.

First, a JAX-RS resource serving a hello endpoint:

[source,java]
----
package org.acme.testcoverage;

import javax.inject.Inject;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

@Path("/hello")
public class GreetingResource {

    private final GreetingService service;

    @Inject
    public GreetingResource(GreetingService service) {
        this.service = service;
    }

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    @Path("/greeting/{name}")
    public String greeting(@PathParam("name") String name) {
        return service.greeting(name);
    }

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() {
        return "hello";
    }
}
----

This endpoint uses a greeting service:

[source,java]
----
package org.acme.testcoverage;

import javax.enterprise.context.ApplicationScoped;

@ApplicationScoped
public class GreetingService {

    public String greeting(String name) {
        return "hello " + name;
    }

}
----

The project will also need a test:

[source,java]
----
package org.acme.testcoverage;

import io.quarkus.test.junit.QuarkusTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.Tag;

import java.util.UUID;

import static io.restassured.RestAssured.given;
import static org.hamcrest.CoreMatchers.is;

@QuarkusTest
public class GreetingResourceTest {

    @Test
    public void testHelloEndpoint() {
        given()
          .when().get("/hello")
          .then()
             .statusCode(200)
             .body(is("hello"));
    }

    @Test
    public void testGreetingEndpoint() {
        String uuid = UUID.randomUUID().toString();
        given()
          .pathParam("name", uuid)
          .when().get("/hello/greeting/{name}")
          .then()
            .statusCode(200)
            .body(is("hello " + uuid));
    }
}
----

== Setting up Jacoco

Now we need to add Jacoco to our project. To do this we need to add the following to the build file:

[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
.pom.xml
----
<dependency>
  <groupId>io.quarkus</groupId>
  <artifactId>quarkus-jacoco</artifactId>
  <scope>test</scope>
</dependency>
----

[source,gradle,role="secondary asciidoc-tabs-target-sync-gradle"]
.build.gradle
----
testImplementation("io.quarkus:quarkus-jacoco")
----

This Quarkus extension takes care of everything that would usually be done via the Jacoco Maven plugin, so no additional
config is required.

WARNING: Using both the extension and the plugin requires special configuration, if you add both you will get lots of errors about classes
already being instrumented. The configuration needed is detailed below.

== Running the tests with coverage

Run `mvn verify`, the tests will be run and the results will end up in `target/jacoco-reports`. This is all that is needed,
the `quarkus-jacoco` extension allows Jacoco to just work out of the box.

There are some config options that affect this:

include::{generated-dir}/config/quarkus-jacoco-jacoco-config.adoc[opts=optional, leveloffset=+1]

== Coverage for tests not using @QuarkusTest

The Quarkus automatic Jacoco config will only work for tests that are annotated with `@QuarkusTest`. If you want to check
the coverage of other tests as well then you will need to fall back to the Jacoco maven plugin.

In addition to including the `quarkus-jacoco` extension in your pom you will need the following config:

[role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
.pom.xml
****
[source,xml]
----
<project>
    <build>
        <plugins>
            ...
          <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <executions>
              <execution>
                <id>default-prepare-agent</id>
                <goals>
                  <goal>prepare-agent</goal>
                </goals>
                <configuration>
                  <exclClassLoaders>*QuarkusClassLoader</exclClassLoaders>  <1>
                  <destFile>${project.build.directory}/jacoco-quarkus.exec</destFile>
                  <append>true</append>
                </configuration>
              </execution>
              <execution>
                <id>default-prepare-agent-integration</id> <2>
                <goals>
                  <goal>prepare-agent-integration</goal>
                </goals>
                <configuration>
                  <exclClassLoaders>*QuarkusClassLoader</exclClassLoaders>
                  <destFile>${project.build.directory}/jacoco-quarkus.exec</destFile>
                  <append>true</append>
                </configuration>
              </execution>
            </executions>
          </plugin>
        </plugins>
    </build>
</project>
----
<1> This config tells it to ignore `@QuarkusTest` related classes, as they are loaded by `QuarkusClassLoader`
<2> This is only needed if you are using Failsafe to run integration tests
****

[role="secondary asciidoc-tabs-target-sync-gradle"]
.build.gradle
****
[source,gradle,subs=attributes+]
----
plugins {
    id 'jacoco' <1>
}

test {
    finalizedBy jacocoTestReport
    jacoco {
        excludeClassLoaders = ["*QuarkusClassLoader"] <2>
        destinationFile = layout.buildDirectory.file("jacoco-quarkus.exec").get().asFile <2>
    }
    jacocoTestReport.enabled = false <3>
}
----
<1> Add the `jacoco` gradle plugin
<2> This config tells it to ignore `@QuarkusTest` related classes, as they are loaded by `QuarkusClassLoader`
<3> Set this config to `false` if you are also using the `quarkus-jacoco` extension and have at least one `@QuarkusTest`.  The default `jacocoTestReport` task can be skipped since `quarkus-jacoco` will generate the combined report of regular unit tests and `@QuarkusTest` classes since the execution data is recorded in the same file.
****

WARNING: This config will only work if at least one `@QuarkusTest` is being run. If you are not using `@QuarkusTest` then
you can simply use the Jacoco plugin in the standard manner with no additional config.

=== Coverage for Integration Tests

To get code coverage data from integration tests, the following need to be requirements need to be met:

* The built artifact is a jar (and not a container or native binary).
* Jacoco needs to be configured in your build tool.
* The application must have been built with `quarkus.package.write-transformed-bytecode-to-build-output` set to `true`

WARNING: Setting `quarkus.package.write-transformed-bytecode-to-build-output=true` should be done with a caution and only if subsequent builds are done in a clean environment - i.e. the build tool's output directory has been completely cleaned.

In the `pom.xml`, you can add the following plugin configuration for Jacoco. This will append integration test data into the same destination file as unit tests,
re-build the Jacoco report after the integration tests are complete, and thus produce a comprehensive code-coverage report.

[source, xml]
----
<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <executions>
                <execution>
                    <id>default-prepare-agent-integration</id>
                    <goals>
                        <goal>prepare-agent-integration</goal>
                    </goals>
                    <configuration>
                        <destFile>${project.build.directory}/jacoco-quarkus.exec</destFile>
                        <append>true</append>
                    </configuration>
                </execution>
                <execution>
                    <id>report-it</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>report</goal>
                    </goals>
                    <configuration>
                        <dataFile>${project.build.directory}/jacoco-quarkus.exec</dataFile>
                        <outputDirectory>${project.build.directory}/jacoco-report</outputDirectory>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>
----

In order to run the integration tests as a jar with the Jacoco agent, add the following to your `pom.xml`.
[source, xml]
----
<build>
    ...
    <plugins>
        ...
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>${surefire-plugin.version}</version>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                    <configuration>
                        <systemPropertyVariables>
                            <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
                            <maven.home>${maven.home}</maven.home>
                            <quarkus.test.arg-line>${argLine}</quarkus.test.arg-line>
                        </systemPropertyVariables>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>

----

WARNING: Sharing the same value for `quarkus.test.arg-line` might break integration test runs that test different types of Quarkus artifacts. In such cases, the use of maven profiles is advised.

== Setting coverage thresholds

You can set thresholds for code coverage using the Jacoco Maven plugin. Note the element `<dataFile>${project.build.directory}/jacoco-quarkus.exec</dataFile>`.
You must set it matching your choice for `quarkus.jacoco.data-file`.

[role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
.pom.xml
****
[source,xml]
----
<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <version>${jacoco.version}</version>
            <executions>
                <execution>
                    <id>jacoco-check</id>
                    <goals>
                        <goal>check</goal>
                    </goals>
                    <phase>test</phase>
                    <configuration>
                        <dataFile>${project.build.directory}/jacoco-quarkus.exec</dataFile>
                        <rules>
                            <rule>
                                <element>BUNDLE</element>
                                <limits>
                                    <limit>
                                        <counter>LINE</counter>
                                        <value>COVEREDRATIO</value>
                                        <minimum>0.8</minimum>
                                    </limit>
                                    <limit>
                                        <counter>BRANCH</counter>
                                        <value>COVEREDRATIO</value>
                                        <minimum>0.72</minimum>
                                    </limit>
                                </limits>
                            </rule>
                        </rules>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>
----
****

[role="secondary asciidoc-tabs-target-sync-gradle"]
.build.gradle
****
[source, gradle]
----
jacocoTestCoverageVerification {
    executionData.setFrom("$project.buildDir/jacoco-quarkus.exec")
    violationRules {
        rule {
            limit {
                counter = 'INSTRUCTION'
                value = 'COVEREDRATIO'
                minimum = 0.80
            }
            limit {
                counter = 'BRANCH'
                value = 'COVEREDRATIO'
                minimum = 0.72
            }
        }
    }
}
check.dependsOn jacocoTestCoverageVerification
----

Excluding classes from the verification task can be configured as following: 

[source,gradle]
----
jacocoTestCoverageVerification {
    afterEvaluate { <1>
        classDirectories.setFrom(files(classDirectories.files.collect { <2>
            fileTree(dir: it, exclude: [
                    "org/example/package/**/*" <3>
            ])
        }))
    }
}
----
<1> `classDirectories` needs to be read after evaluation phase in Gradle
<2> Currently, there is a bug in Gradle JaCoCo which requires the `excludes` to be specified in this manner - https://github.com/gradle/gradle/issues/14760.  Once this issue is fixed, excludes
<3> Exclude all classes in `org/example/package` package
****

== Conclusion

You now have all the information you need to study the coverage of your tests!
But remember, some code that is not covered is certainly not well tested. But some code that is covered is not necessarily *well* tested. Make sure to write good tests!
