the correct way to use integration tests in your build process
February 7, 2013

How to Use Java Integration Testing in Your Build Process

Java Testing
Java Application Development

In my previous article on unit testing, I showed you how to mock all the dependencies of a single Java class in order to isolate it from the rest of the system. This way the class under test is in a controlled environment where you can test its own business logic with nothing else in the way. There are times, however, when you should NOT use mocking during testing. For these cases, you need Java integration testing. All the previous articles in the testing series talked about unit tests so far. And yes, there is a difference!

We use Java integration testing when we are actually interested in the interface of this class with its dependencies. You zoom out from a single class and you want to test a component of the system which not only is based on multiple classes but it might also use other systems such as a database or a queue. 

What is a Java Integration Test?

Sometimes there is not a clear distinction on what is an integration test and what is a unit test. My basic rule of thumb is that if

  1. a test uses the database
  2. a test uses the network
  3. a test uses an external system (e.g. a queue or a mail server)
  4. a test reads/writes files or performs other I/O

...then it is an integration test and not a unit test. I have seen several developers who talk about “tests” and either they mean both or just integration tests.

Unit Test vs. Integration Test

Unit testIntegration test
Results depends only on Java codeResults also depends on external systems
Easy to write and verifySetup of integration test might be complicated
A single class/unit is tested in isolationOne or more components are tested
All dependencies are mocked if neededNo mocking is used (or only unrelated components are mocked)
Test verifies only implementation of codeTest verifies implementation of individual components and their interconnection behaviour when they are used together
A unit test uses only JUnit/TestNG and a mocking frameworkAn integration test can use real containers and real DBs as well as special java integration testing frameworks (e.g. Arquillian or DbUnit)
Mostly used by developersIntegration tests are also useful to QA, DevOps, Help Desk
A failed unit test is always a regression (if the business has not changed)A failed integration test can also mean that the code is still correct but the environment has changed
Unit tests in an Enterprise application should last about 5 minutesIntegration tests in an Enterprise application can last for hours


How to Write Integration Tests in Java

Writing an integration test is heavily dependent on your environment. The first thing that you should decide is the scope of your integration test. So if we keep following Matt from the previous article who keeps on working on the gigantic CRM, he could write integration tests for:

  • Verifying correct integration of the CRM with the printer (in a staging environment of course)
  • Verifying correct integration of the CRM with the mail server
  • Verifying correct reading/writing of invoices from/to the DB
  • Verifying the whole data flow of receiving an order, creating an invoice, saving it to the DB and mailing it to the client. This is an End-To-End integration test

Java Integration Testing Best Practices

Since some of the integration tests in the case of Matt, use a staging environment (e.g. the mail server) it is also important to document these dependencies so that his fellow developers know about them. I always hate it when I run the test suite on a new application and half the tests fail because my workstation has no network access to the testing database!

1. Use Detailed Logging

A second point with integration tests that must be accounted is the use of detailed logging. When a unit test fails it is very easy to understand why since the scope is very narrow. When an integration test fails, things are not so simple. Because by definition an integration tests is based on many components and a specific data flow, identifying the failure cause is not always straightforward.

My recommended way to alleviate this problem, is the use of detailed logging statements (that are always needed in an Enterprise application regardless of unit tests). This way, when an integration test fails you can examine the logs and understand if the issue is in the code or in an external resource used by the test.

2. Don't Use Java Integration Testing and Unit Testing Concurrently

Now we reach the most important point regarding unit tests. In a big enterprise application integration and unit tests MUST be handled differently. Here is an all too common scenario that I have personally seen multiple times.

Matt has created a lot of unit and integration tests. All of them are executed by Maven when the test goal is run. However during a server migration some of the integration tests stop working. However everyone on the team is busy and nobody fixes the IPs in the configuration files.

Soon after some integration tests that depend on an external system run really slowly. But nobody has time to investigate the cause. Developers no longer run tests before committing code because the test suite is very slow. More unit tests break as a result, since developers do not maintain them.

New developers come into the team. They start working on the CMS and soon find out that half the test suite is broken. Most of them do not even bother with unit tests anymore.

A valiant developer comes into the team and says that this madness must stop. He spends a day and finds out that the effort required to fix all tests is not realistic for the current timeframe. He also finds out that in several cases the unit tests are broken because of changes in the business requirements. So fixing the tests is not a straightforward process since somebody has to adapt them to new code.

By this point it is clear that tests are not actually used in this project. New developers simply declare that “writing unit tests is a waste of time” and they are right from their point of view, since nobody wants to work with a broken test suit This is a scenario that we need to avoid.

3. Split Unit and Integration Tests

There are many ways to split unit and integration tests. My suggestion is to use the Maven failsafe plugin. Unit tests should follow the naming convention introduced in the first part of this series. Unit test classes are named with “name of class + Test”. Then they are placed in the test directory of the Maven project structure. The unit tests are executed automatically when the test goal is run. Next you should add the failsafe plugin in your pom.xml file.


  …
  
    
      …
      
        org.apache.maven.plugins
        maven-failsafe-plugin
        2.13
        
          
            integration-test
            
              integration-test
            
          
          
            verify
            
              verify
            
          
        
      
    
  


Your integration tests however hava a different naming convention. They are named as “name of class + IT”. IT stands for Integration Test. Now the test goal will ignore them. Instead these tests will be executed by the integration-test goal which is a built-in goal into Maven. Here is a table that summarizes this split.

 Unit testsIntegration Tests
Located inMaven test directoryMaven test directory
Naming conventionname of class + Testname of class + IT
Example class nameBasketWeightTest.javaInvoicingProcessorIT.java
Managed byMaven surefire pluginMaven failsafe plugin
Executed in test goalYesNo
Executed in integration-test goalNoYes

How to Run Integration Tests in Your Build Process

Now that all these changes are done you have great flexibility on how you run unit tests. Most importantly your build server (e.g. Jenkins) should contain a mixture of jobs that deal with both kinds of tests. Here is a overview of suggested jobs.

Job typeScheduleDescriptionTests
Main buildEvery 15 minutes or half hour.Only compiles and runs unit tests. Should finish in 15-20 minutes maxOnly unit tests
Integration buildEvery 24 hours (usually at night)Runs integrations tests. Can run for 2-3 hoursAll tests
QA buildManuallyDeploys to a QA environmentAll tests

The suggested workflow is the following

  1. Developers run the test goal during development
  2. Developers run the test goal before any commit
  3. Developers run the integration-test goal before a major commit with many side effects
  4. Build server compiles code and runs the test goal every 15-30 minutes (main build)
  5. Build server compiles code and runs the integration-test goal every day (integration build)
  6. Build server compiles code and runs the integration-test goal before a release to QA

With this workflow it is clear that developers get fast feedback from the unit tests so anything that breaks can be fixed immediately. The boring process of running integration tests is left to the build server which runs them automatically in a well defined schedule.

Java Integration Testing Frameworks

As we mentioned at the beginning of the article, developers often utilize a Java integration testing framework. A few of the more popular Java integration testing frameworks include Arquillian, DbUnit, and JUnit

  • Arquillian - a Java integration testing framework specific to Java EE
  • DbUnit - as the name implies, DbUnit can run both unit and integration tests
  • JUnit - while most use JUnit for unit testing, many use JUnit for integration testing

Final Thoughts

In this post we have finally tackled Java integration testing. We showed you the differences with unit tests regarding test focus, external systems and running time. We also hope we convinced you that they must be handled differently. Our suggested method for splitting tests is the maven failsafe plugin. Finally we proposed some combinations of build jobs and how they run different types of tests.

Improve Performance During Development

Want to improve performance during development?

XRebel identifies performance issues in your code, even in distributed applications. This means better performing applications, and happier end users.

Try XRebel for Free