Code Coverage - JaCoCo with Maven multi-module project

In this post, we will see how to configure the JaCoCo for Maven multi-module projects.
Previously, JaCoCo did not support the multi modules maven project, but with the latest version, it adds in the supports. JaCoCo has provided its sample at Git under its maven plugin test at it-report-aggregate. Let's use this sample project to see how it works.

First download the code by using command

svn checkout https://github.com/jacoco/jacoco/trunk/jacoco-maven-plugin.test/it/it-report-aggregate

This project is a parent-child project. It contains four modules, the child1, child1-test, child2 and report. The report project is used to create a centralized code coverage report. 

This project is a child project in Jacoco project by default, it has configured to have "setup-parent" as its parent project in the pom file. To make this project works as an independent project, we need to do a little modifications.

First, remove the parent configuration at it-report-aggregate/pom.xml and add in the groupId and version as below (remove the parent and add in the lines flagged as blue)

 <parent>

    <groupId>jacoco</groupId>

    <artifactId>setup-parent</artifactId>

    <version>1.0-SNAPSHOT</version>

  </parent>

  <groupId>jacoco</groupId>

  <artifactId>it-report-aggregate</artifactId>

  <version>1.0-SNAPSHOT</version>

  <packaging>pom</packaging>

Second, copy in all the dependences and plugin configurations from Jacobo/jacoco-maven-plugin.test/it/setup-parent/pom.xml to this pom.xml.

Third, change the "@project.groupId@" to "org.jacoco" for plugin configuration in this pom.xml and also the one under folder it-report-aggregate and it-report-aggregate/report.

Run the command "mvn clean verify", the report will be generated under report/target/site/jacoco-aggregate. However, it only includes the report for child1 and child1-test. Check at the file report/pom.xml. you will see that the module child1 and child2 are configured at different scope as dependency.

  <dependencies>

    <dependency>

      <groupId>jacoco</groupId>

      <artifactId>child1</artifactId>

      <version>${project.version}</version>

      <scope>compile</scope>

    </dependency>

    <dependency>

      <groupId>jacoco</groupId>

      <artifactId>child1-test</artifactId>

      <version>${project.version}</version>

      <scope>test</scope>

    </dependency>

    <dependency>

      <groupId>jacoco</groupId>

      <artifactId>child2</artifactId>

      <version>${project.version}</version>

      <scope>runtime</scope>

    </dependency>

  </dependencies>

Change the scope of child2 to compile, and run command "mvn clean verify" again, you will see the report generated with all three modules then.It automatically generates the report for all the modules.

Let's come back to see what configurations it has done. 

Check on all the pom.xml in the projects, we will find only the pom files under it-report-aggregate and it-report-aggregate/report have been added in the configurations for JaCoCo. 

• it-report-aggregate/pom.xml:  It defined the JaCoCo plugin at "prepare-agent". All the child modules will inherit this configurations, so that's why you will see the Jacobo.exec generated under all child module target folder.

  <build>

    <plugins>

      <plugin>

        <groupId>@project.groupId@</groupId>

        <artifactId>jacoco-maven-plugin</artifactId>

        <executions>

          <execution>

            <id>prepare-agent</id>

            <goals>

              <goal>prepare-agent</goal>

            </goals>

          </execution>

        </executions>

      </plugin>

    </plugins>

  </build>

• it-report-aggregation/report/pom.xml: it defined all other three modules as dependencies and also add in the JaCoCo plugin at "report-aggregate". This is the new feature that JaCoCo just added in for supporting this multi module case at version 0.7.7.

  <dependencies>

    <dependency>

      <groupId>jacoco</groupId>

      <artifactId>child1</artifactId>

      <version>${project.version}</version>

      <scope>compile</scope>

    </dependency>

    <dependency>

      <groupId>jacoco</groupId>

      <artifactId>child1-test</artifactId>

      <version>${project.version}</version>

      <scope>test</scope>

    </dependency>

    <dependency>

      <groupId>jacoco</groupId>

      <artifactId>child2</artifactId>

      <version>${project.version}</version>

      <scope>runtime</scope>

    </dependency>

  </dependencies>

  <build>

    <plugins>

      <plugin>

        <groupId>org.jacoco</groupId>

        <artifactId>jacoco-maven-plugin</artifactId>

        <executions>

          <execution>

            <id>report-aggregate</id>

            <phase>verify</phase>

            <goals>

              <goal>report-aggregate</goal>

            </goals>

          </execution>

        </executions>

      </plugin>

    </plugins>

  </build>

This is very clear on how to configure the JaCoCo, below is the summary steps.

1. add in the JaCoCo plugin for prepare-agent at parent pom

2. create a separate module in the project and announce it as a child from the parent

3. add all modules as dependences into this new module

4. add in the JaCoCo plugin for report-aggregate at the new module's pom

Refer the official Wiki for more information.

Work-Arounds

The life is much better if we could have a backup plan B. The JaCoCo Ant task has provide the alter way to create the centralized report for Maven multi module project. To use the ant tasks in maven plugin, we will need to use the plugin "maven-antrun-plugin".  Let's continue to use the same sample project to see how could we make it.

Create a new module named "report2" to demo this method. In the pom.xml, announce the same parent and dependences as the report module. Then we add in the maven-dependcy-plugin to copy the jacocoant.jar file to local. 

<plugin>

   <groupId>org.apache.maven.plugins</groupId>

   <artifactId>maven-dependency-plugin</artifactId>

   <executions>

<!-- Copy the ant tasks jar. Needed for ts.jacoco.report-ant . -->

<execution>

        <id>jacoco-dependency-ant</id>

        <goals>

<goal>copy</goal>

        </goals>

        <phase>process-test-resources</phase>

        <inherited>false</inherited>

        <configuration>

<artifactItems>

             <artifactItem>

                <groupId>org.jacoco</groupId>

                <artifactId>org.jacoco.ant</artifactId>

                 <version>0.7.6.201602180812</version>

             </artifactItem>

</artifactItems>

<stripVersion>true</stripVersion>

<outputDirectory>${basedir}/target/jacoco-jars</outputDirectory>

        </configuration>

</execution>

    </executions>

</plugin>

After this configure the ant task, the ant task will use the existing jacocoant.jar and computing the coverage. The configuration is very similar with JaCoCo Ant Task.

<plugin>

   <groupId>org.apache.maven.plugins</groupId>

   <artifactId>maven-antrun-plugin</artifactId>

   <version>1.6</version><!--$NO-MVN-MAN-VER$ -->

   <inherited>false</inherited>

   <executions>

      <execution>

<phase>post-integration-test</phase>

<goals>

           <goal>run</goal>

</goals>

<configuration>

           <target>

              <echo message="Generating JaCoCo Reports" />

              <taskdef name="report" classname="org.jacoco.ant.ReportTask">

                <classpath path="../target/jacoco-jars/org.jacoco.ant.jar" />

              </taskdef>

              <report>

                <executiondata>

                   <fileset dir="../child1/target">

                       <include name="jacoco.exec" />

                   </fileset>

                   <fileset dir="../child1-test/target">

                      <include name="jacoco.exec" />

                    </fileset>

                   <fileset dir="../child2/target">

                      <include name="jacoco.exec" />

                   </fileset>

                 </executiondata>

                <structure name="rmdaalser Coverage Project">

                   <group name="rmdaalser">

                      <sourcefiles encoding="UTF-8">

                           <fileset dir="./child1/src/main/java" />

                        <fileset dir="../child2/src/main/java" />

                      </sourcefiles>

                      <classfiles>

                        <fileset dir="../child1/target/classes" />

                        <fileset dir="../child2/target/classes" />

                      </classfiles>

                    </group>

                </structure>

                 <html destdir="${basedir}/target/jacoco-coverage-report/html" />
               </report>

           </target>

</configuration>

      </execution>

   </executions>
   <dependencies>

<dependency>

<groupId>org.jacoco</groupId>

<artifactId>org.jacoco.ant</artifactId>

<version>0.7.6.201602180812</version>

</dependency>

   </dependencies>

</plugin>

Run "mvn clean verify", you will see the coverage report under report2/target/jacoco-coverage-report/html.

Problem: JaCoCo - Skipping JaCoCo execution due to missing execution data file

When configured JaCoCo with maven project, it is very common to meet the problem "Skipping JaCoCo execution due to missing execution data file" when running the test. It is not an error, the build will still succeed, however, the JaCoCo will not generate the report because it could not find the Jacobo.exec file.

There are many reasons for this problem. But most common case is that the JaCoCo agent is failed to configured into the command line. Using the command "mvn -X clean verify" to check on the debug logs. At the "TESTS" step, you will be able to see a line for "Forking command line".

If the JaCoCo agent is configured correctly, you will be able to see similar logs as below.

-------------------------------------------------------

 T E S T S

-------------------------------------------------------

-------------------------------------------------------

 T E S T S

-------------------------------------------------------

[DEBUG] boot classpath:  /Users/jhuang8/.m2/raptor2/org/apache/maven/surefire/surefire-booter/2.17/surefire-booter-2.17.jar  /Users/jhuang8/.m2/raptor2/org/apache/maven/surefire/surefire-api/2.17/surefire-api-2.17.jar  /Users/jhuang8/Documents/workspace/JaCoCoPractice/target/test-classes  /Users/jhuang8/Documents/workspace/JaCoCoPractice/target/classes  /Users/jhuang8/.m2/raptor2/junit/junit/4.8.1/junit-4.8.1.jar  /Users/jhuang8/.m2/raptor2/org/apache/maven/surefire/surefire-junit4/2.17/surefire-junit4-2.17.jar

[DEBUG] boot(compact) classpath:  surefire-booter-2.17.jar  surefire-api-2.17.jar  test-classes  classes  junit-4.8.1.jar  surefire-junit4-2.17.jar

Forking command line: /bin/sh -c cd /Users/jhuang8/Documents/workspace/JaCoCoPractice && /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/bin/java -javaagent:/Users/jhuang8/.m2/raptor2/org/jacoco/org.jacoco.agent/0.7.6.201602180812/org.jacoco.agent-0.7.6.201602180812-runtime.jar=destfile=/Users/jhuang8/Documents/workspace/JaCoCoPractice/target/jacoco.exec -Xmx1024m -XX:MaxPermSize=512m -jar /Users/jhuang8/Documents/workspace/JaCoCoPractice/target/surefire/surefirebooter3359878682012651006.jar /Users/jhuang8/Documents/workspace/JaCoCoPractice/target/surefire/surefire8553706350128891199tmp /Users/jhuang8/Documents/workspace/JaCoCoPractice/target/surefire/surefire_04828513279405888630tmp

The JaCoCo part is flagged out as blue.

And how can we add in these JaCoCo agent info? according to the office site Jacobo:prepare-agent,  you can obviously add in the @{argLine} at your maven-surefire-plugin configurations as

<plugin>

   <artifactId>maven-surefire-plugin</artifactId>

   <configuration>

      <forkMode>once</forkMode>

      <forkCount>1</forkCount>

      <argLine>@{argLine} -Xmx1024m </argLine>

      <reuseForks>false</reuseForks>

   </configuration>

</plugin>

The default name is argLine, but sometime, the JaCoCo configuration will be overwrote by some other argLine setup or plugin in pom. A way to avoid this is to configure a new property name at JaCoCo plugin and put it at the maven-surefire-plugin. Below is the sample configruation.

Set a new propertyName at JaCoCo plugin

<plugin>

   <groupId>org.jacoco</groupId>

   <artifactId>jacoco-maven-plugin</artifactId>

   <version>0.7.6.201602180812</version>

   <executions>

<execution>

<id>prepare-agent</id>

<goals>

         <goal>prepare-agent</goal>

</goals>  

<configuration>

<propertyName>jaCoCoArgLine</propertyName>

      </configuration>

</execution>

<execution>

<id>post-unit-test</id>

<phase>test</phase>

<goals>

<goal>report</goal>

</goals>

</execution>

   </executions>

</plugin>

and put the new propertyName at maven-surefire-plugin like below

<argLine>jaCoCoArgLine -Xmx1024m </argLine>

This time, check at the maven logs, it will shows as jaCoCoArgline under JaCoCo maven plugin for prepare-agent stage. And you will be able to see the Jacobo.exec file generated under that path.

[INFO] --- jacoco-maven-plugin:0.7.6.201602180812:prepare-agent (prepare-agent) @ JaCoCoPractice ---

[INFO] jaCoCoArgLine set to -javaagent:/Users/jhuang8/.m2/raptor2/org/jacoco/org.jacoco.agent/0.7.6.201602180812/org.jacoco.agent-0.7.6.201602180812-runtime.jar=destfile=/Users/jhuang8/Documents/workspace/JaCoCoPractice/target/jacoco.exec

If the problem still shows up, then double check the maven-surefire-plugin, according to official site Maven Plug-in it has the restrictions as below.

When using the maven-surefire-plugin or maven-failsafe-plugin you must not use a forkCount of 0 or set the forkMode to never as this would prevent the execution of the tests with the javaagentset and no coverage would be recorded.

Code Coverage - JaCoCo Tutorial

JaCoCo is a very popular tool for code coverage now. Comparing with Cobertura, it has the benefit that it is totally based on Java Byte code so it can work without source files. Its remote protocol and JMX control supports requesting the execution data dumps from the coverage agent at any point in time, which makes life much easier for analyzing the code coverage for integration test. Most import it is able to compute the code coverage for unit test together with integration test, which gives you a better report on how the code coverage like for the whole project.

As what it says at the official site

JaCoCo should provide the standard technology for code coverage analysis in Java VM based environments. The focus is providing a lightweight, flexible and well documented library for integration with various build and development tools.


Coverage Analysis Mechanism
Let's first see how JaCoCo works. Here is a diagram coming from the JaCoCo Documentation. It gives an overview on the techniques for code coverage and the highlighted ones are the method JaCoco used.





JaCoCo instrument the byte code by hooking a Java Agent to the JVM. It use the ASM for instrumentation.

The Jave Agent is made available in Java 1.5 for monitoring and profiling JVMs. It is able to dynamically modify Java classes as they are being loaded. The JVM starts with the agent, and Whenever the classes is loaded, the JaCoCo will instrument the classes. It collects the data when the class , method or lines are called on fly and generate the report files when the JVM terminates.

JaCoCo with Maven Project
Here I am using a sample project to demonstrate how JaCoCo works with Maven projects.
My Maven project has the class Palindrome.java and PalinedromeTest.java as below

public class Palindrome {

    public boolean isPalindrome(String inputString) {

    if (inputString.length() == 0) {

    return true;

    } else {

    char firstChar = inputString.charAt(0);

    char lastChar = inputString.charAt(inputString.length() - 1);

    String mid = inputString.substring(1, inputString.length() - 1);

    return (firstChar == lastChar) && isPalindrome(mid);

    }

   }

}

public class PalindromeTest extends TestCase {

   

    public PalindromeTest( String testName )

    {

        super( testName );

    }

    public static Test suite()

    {

        return new TestSuite( PalindromeTest.class );

    }

    public void testPalindrome()

    {

    Palindrome palindromeTester = new Palindrome();

    assertTrue(palindromeTester.isPalindrome(""));

    }

}

The file structure of the the project is like this 

Add in the JaCoCO Plugin configuration into pom.xml as below to execute the JaCoCo prepare-agent and generate the report

<build>

  <plugins>

<plugin>

   <artifactId>maven-surefire-plugin</artifactId>

   <configuration>

<forkMode>once</forkMode>

<forkCount>1</forkCount>

<argLine>@{argLine} -Xmx1024m -XX:MaxPermSize=512m</argLine>

<reuseForks>false</reuseForks>

   </configuration>

</plugin>

<plugin>

   <groupId>org.jacoco</groupId>

   <artifactId>jacoco-maven-plugin</artifactId>

   <version>0.7.6.201602180812</version>

   <executions>

<execution>

<id>prepare-agent</id>

<goals>

           <goal>prepare-agent</goal>

</goals>

</execution>

<execution>

<id>post-unit-test</id>

<phase>test</phase>

<goals>

            <goal>report</goal>

</goals>

</execution>

   </executions>

</plugin>

</plugins>

</build>

After saving all the changes, run the maven command "mvn clean verify", The jacoco.exec file will be generated under target folder. And the report is generated under target/site/jacoco.


Open the index.html, the code coverage result shows up


Drill down to see the coverage for specific class. The red line are the ones haven't been covered and the green lines are the ones covered.

JaCoCo With Eclipse Plugin
You can use JaCoCo in Eclipse with zero configuration by using EclEmma Eclipse.
Install the EclEmma Java Code Coverage in Eclipse, you will get an icon as

 at your tool bar.


Click on the button and unit test will run automatically with code coverage reporting under "Coverage".



References:

• http://www.baeldung.com/jacoco
• http://www.eclemma.org/jacoco/trunk/doc/index.html