TestContainers: Making Java Integration Tests Easy
In this post I want to share a word about an awesome library for integration testing in Java -- TestContainers. We’ll provide a little background on why integration testing is so important for us, our requirements for integration tests and how TestContainers helps us with our own integration testing. You'll also find a fully-functional Java container example with an integrated test for a Java agent.
Integration Testing at ZeroTurnaround
At ZeroTurnaround, our products integrate with a large portion of the Java ecosystem. More specifically, JRebel and XRebel are based on Java agent technology to integrate with Java applications, frameworks, application servers, etc.
Java agents instrument Java code to add the desired functionality. To test how the application behaves after patching, we need to start it with the pre-configured Java agent. Once the application is up and running, we can execute an HTTP request to the application in order to observe the desired behavior.
To run such tests at scale we demand an automated system that can start and stop the environment. This includes an application server and any other external dependencies that the application depends on. It should be possible to run the same setup in a continuous integration environment as well as on the developer’s computer.
Since there are a lot of tests and they aren’t very fast, we want to execute the tests concurrently. This means that the tests should be isolated and as such need to avoid resource conflicts. For instance, if we start many instances of Tomcat on the same host, we want to make sure that there will be no port conflicts.
A nice little Java library that helps us with the integration testing is TestContainers. It is really helpful with the requirements we have and was a major productivity boost since we adopted it for our tests. Here's an example.
TestContainers
The official documentation of TestContainers states the following:
“TestContainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.”
TestContainers provides the API to automate the environment setup. It will spin up the required Docker containers for the duration of the tests and tear down once the test execution has finished. Next, we will take a look at a few concepts based on the official examples found in GitHub repository.
GenericContainer
With TestContainers you will notice the GenericContainer class being used quite frequently:
public class RedisBackedCacheTest {
@Rule
public GenericContainer redis = new GenericContainer("redis:3.0.6")
.withExposedPorts(6379);
The constructor of GenericContainer takes a string as a parameter where we can specify the specific Docker image that we want to use. During start up, TestContainers will download the required Docker image, if it is not yet present on the system.
An important thing to notice is the withExposedPorts(6379) method, which declares that 6379 should be the port that container listens on. Later we can find the mapped port of for that exposed port by calling getMappedPort(6379) on the container instance. Combining it with getContainerIpAddress() we can get the full URL to the service running in the container:
String redisUrl = redis.getContainerIpAddres() + “:” + redis.getMappedPort(6379);
You will also notice that the field in the example above is annotated with the @Rule annotation. JUnit’s @Rule annotation declares that we will get a new instance of GenericContainer for every single test method in this class. We could also use @ClassRule annotation if we wanted to reuse the container instance for the different tests.
Specialized Containers in Java
The specialized containers in TestContainers are created by extending the GenericContainer class. Out of the box, one can use a containerized instance of a MySQL, PostgreSQL or Oracle database to test your data access layer code for complete compatibility.
PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer("postgres:9.6.2")
.withUsername(POSTGRES_USERNAME)
.withPassword(POSTGRES_PASSWORD);
Simply by declaring an instance of the container will get us a PostgreSQL database running for the duration of the test. No need to install the database on the machine where the tests will be running. Huge win if we need to run tests with the different versions of the same database!
Custom Containers in Java
By extending GenericContainer it is possible to create custom container types. This is quite convenient if we need to encapsulate the related services and logic. For instance, we use MockServer to mock the dependencies in a distributed system where apps talk to each other over HTTP:
public class MockServerContainer extends BaseContainer {
MockServerClient client;
public MockServerContainer() {
super("jamesdbloom/mockserver:latest");
withCommand("/opt/mockserver/run_mockserver.sh -logLevel INFO -serverPort 80");
addExposedPorts(80);
}
@Override
protected void containerIsStarted(InspectContainerResponse containerInfo) {
client = new MockServerClient(getContainerIpAddress(), getMappedPort(80));
}
}
In this example, once the container has been initialized we use a callback, containerIsStarted(...), to initialize an instance of MockServerClient. This way, we have declared all the container specific details in the new custom container type. So we can clean up the client code and provide a nicer API for the tests. As we will see further, custom containers will help us to structure our environment for testing Java agents.
Testing a Java Agent With TestContainers
To demonstrate the idea we will use the example kindly provided by Sergei @bsideup Egorov, who is the co-maintainer of TestContainers project.
Test application
Let’s start with a sample application. We need a web application that responds to HTTP GET requests. We don’t need a huge application framework to implement that. So, why not to use SparkJava for this task? To make it more fun, let’s use Groovy! Here is the application that we will use for testing:
//app.groovy
@Grab("com.sparkjava:spark-core:2.1")
import static spark.Spark.*
get("/hello/") { req, res -> "Hello!" }
A simple Groovy script, that uses Grape to download the required SparkJava dependency, and specifies an HTTP GET endpoint that responds with the message, “Hello!”.
Using Java Agent
The agent that we want to test patches the embedded Jetty server by adding an extra header to the response.
public class Agent {
public static void premain(String args, Instrumentation instrumentation) {
instrumentation.addTransformer(
(loader, className, clazz, domain, buffer) -> {
if ("spark/webserver/JettyHandler".equals(className)) {
try {
ClassPool cp = new ClassPool();
cp.appendClassPath(new LoaderClassPath(loader));
CtClass ct = cp.makeClass(new ByteArrayInputStream(buffer));
CtMethod ctMethod = ct.getDeclaredMethod("doHandle");
ctMethod.insertBefore("{ $4.setHeader(\"X-My-Super-Header\", \"42\"); }");
return ct.toBytecode();
} catch (Throwable e) {
e.printStackTrace();
}
}
return buffer;
});
}
}
In the example, Javassist is used to patch the JettyHandler.doHandle method by adding an extra statement that sets the new X-My-Super-Header header. Of course, to become a Java agent, it has to be properly packaged and include the corresponding attributes in the MANIFEST.MF file. The build script handles it for us, see build.gradle in the GitHub repository.
The Java Agent Test
The test itself is quite simple: we make a request to our application and check the response for the specific header that the Java agent is supposed to add. If the header is found and the header value equals to the value that we expect then the test passes.
@Test
public void testIt() throws Exception {
// Using Feign client to execute the request
Response response = app.getClient().getHello();
assertThat(response.headers().get("X-My-Super-Header"))
.isNotNull()
.hasSize(1)
.containsExactly("42");
}
We can run this test from our IDE, or from the build tool, or even in a continuous integration environment. TestContainers will help us to actually start the application with the agent in an isolated environment, the Docker container. To start the application we need a Docker image with Groovy support. Just for our own convenience we have zeroturnaround/groovy Docker image hosted at Docker Hub. Here’s how we can use it by extending the GenericContainer class from TestContainers:
public class GroovyTestApp>
extends GenericContainer {
public GroovyTestApp(String script) {
super("zeroturnaround/groovy:2.4.5");
withClasspathResourceMapping("agent.jar", "/agent.jar", BindMode.READ_ONLY);
withClasspathResourceMapping(script, "/app/app.groovy", BindMode.READ_ONLY);
withEnv("JAVA_OPTS", "-javaagent:/agent.jar");
withCommand("/opt/groovy/bin/groovy /app/app.groovy");
withLogConsumer(new Slf4jLogConsumer(LoggerFactory.getLogger(script)));
}
public String getURL() {
return "http://" + getContainerIpAddress() + ":"
+ getMappedPort(getExposedPorts().get(0));
}
}
Note that the API provides us with the methods to acquire the container IP address along with the mapped port which is actually randomized. Meaning that the port will be different on every test execution and there will be no port conflicts if we run such tests concurrently! Now it is easy to use the GroovyTestApp class to execute Groovy scripts and our test application specifically:
GroovyTestApp app = new GroovyTestApp(“app.groovy”)
.withExposedPorts(4567); //the default port for SparkJava
.setWaitStrategy(new HttpWaitStrategy().forPath("/hello/"));
Running the tests:
$ ./gradlew test 16:42:51.462 [I] d.DockerClientProviderStrategy - Accessing unix domain socket via TCP proxy (/var/run/docker.sock via localhost:50652) … … … 16:43:01.497 [I] app.groovy - STDERR: [Thread-1] INFO spark.webserver.SparkServer - == Spark has ignited ... 16:43:01.498 [I] app.groovy - STDERR: [Thread-1] INFO spark.webserver.SparkServer - >> Listening on 0.0.0.0:4567 16:43:01.511 [I] app.groovy - STDERR: [Thread-1] INFO org.eclipse.jetty.server.Server - jetty-9.0.2.v20130417 16:43:01.825 [I] app.groovy - STDERR: [Thread-1] INFO org.eclipse.jetty.server.ServerConnector - Started [email protected]{HTTP/1.1}{0.0.0.0:4567} 16:43:02.199 [I] ?.4.5] - Container zeroturnaround/groovy:2.4.5 started AgentTest > testIt STANDARD_OUT Got response: HTTP/1.1 200 OK content-length: 6 content-type: text/html; charset=UTF-8 server: Jetty(9.0.2.v20130417) x-my-super-header: 42 Hello! BUILD SUCCESSFUL Total time: 36.014 secs
The test isn’t very fast. Grapes take some time to download, but only once. However, it is a full integration test example that starts up a Docker container, an application with HTTP stack and makes an HTTP call. Besides, it runs in isolation and it is really simple as well. All thanks to TestContainers!
Conclusions
“Works on my machine” shouldn’t be an excuse any longer. With container technology becoming more accessible to developers, coupled with proper automation, we are now able to achieve a very deterministic way to test our applications. TestContainers brings some sanity to integration testing of Java apps. It is very easy to integrate into your existing tests. You will no longer have to manage the external dependencies yourself, which is a huge win, especially in a CI environment.
If you liked what you just learned, we encourage you to take a look at the video recording from our GeekOut Java conference where Richard North, the original author of the project, gives an overview of TestContainers, including the future plans for improving it, or at least glance through his slides for that presentation!
Want more developer insights? Visit our resources page for the latest Java news, webinars, white papers and cheat sheets.
