Using Java Executors for Background Tasks
One of the biggest advantages Java has over other platforms is that it is spectacularly good at utilizing resources for parallel computations. However, making sure those computations are always correct is a job for Java Executors. But which one do you choose?
In this article, we give an overview of available Java executors,and which one to choose based on your needs.
Why Do We Need Java Executors?
On the JVM it’s ridiculously easy to execute a piece of code in the background and consume the result of that computation when it is ready and when we actually need it. At the same time this allows the developer to make better use of all that computational power modern hardware has to offer. Perhaps the most challenging task for the developer is to create programs that are always correct, rather than the familiar “works on my machine” correct.
What Are Java Executors?
In a nutshell, a Java ExecutorService is an interface that aims to decouple the declaration of the task to be done in the background from the actual computation:
public interface Executor { void execute(Runnable command); }
It accepts tasks in the form of a Runnable instance. A thread at some point of time will pick tasks up and execute the Runnable::run method.
However, the real trick is often to figure out which implementation of Executor to use.
Picking a Java Executor Type
There’s a bunch of default choices ready for you in the Executors class. Let’s see what they are and when to pick the different types. In general, when choosing a Java Executor for your background computation needs, you have to consider 3 main questions about the nature of the work it will perform:
- How many threads do you want to run in parallel by default?
- What should the executor do with a submitted task when all available threads are busy working? Hint: usually it is either spawn more threads or to put the task into a queue.
- Do you want to limit the task queue size for the tasks and what should happen if it gets full?
Creating a ThreadPoolExecutor
If you want to explicitly control the answers to these questions, you can use the flexible API provided by the JDK and create your own ThreadPoolExecutor. The constructor for ThreadPoolExecutor explicitly requires you to answer all these questions: ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler).
Here’s a description of what the parameters mean and how they answer the questions above:
- int corePoolSize - the number of threads you want to start the Thread pool off with, initially
- int maximumPoolSize - when the core threads are busy, do you want to grow the pool size up to the maximum?
- long keepAliveTime, TimeUnit unit - do you want to shutdown spare threads if there’s no work for them? How long should the pool wait until then?
- BlockingQueue workQueue - how the tasks that cannot be handled right away should be stored? Do you want to limit the task queue size?
- RejectedExecutionHandler handler - What to do if the task cannot be accepted by the executor? Should we throw an exception? Should the caller perform the task itself?
ExecutorServices Created by Factory Methods in Executors Class
Here's a short summary of how ExecutorServices that are created by the factory methods in the Executors class differ from each other. I hope this will shed some light on how do they resolve the important questions above.
- newFixedThreadPool(int nThreads) - n threads will process tasks at the time, when the pool is saturated, new tasks will get added to a queue without a limit on size. Good for CPU intensive tasks.
- newWorkStealingPool(int parallelism) - will create and shut down threads dynamically to accommodate the required parallelism level. It also tries to reduce the contention on the task queue, so can be really good in heavily loaded environments. Also good when your tasks create more tasks for the executor, like recursive tasks.
- newSingleThreadExecutor() - creates an unconfigurable `newFixedThreadPool(1)`, so you know that only one thread will process everything. Good when you really need predictability and sequential tasks completion.
- newCachedThreadPool() - doesn't put tasks into a queue. Consider this as the same as using a queue with the maximum size of 0. When all current threads are busy, it creates another thread to run the task. Sometimes it can reuse threads. Good for denial of service attacks on your own servers. The problem with a cached thread pool is that it doesn’t know when to stop spawning more and more threads. Imagine the situation where you have computationally intensive tasks that you submit into this executor. The more threads that consuming the CPU, the slower every individual task takes to process. This has a domino effect in that it means more work gets backlogged. As the result you’ll end up with more and more threads spawned making task processing even slower. This negative feedback loop is a hard problem to solve. So for almost all intents and purposes,
Executors::newFixedThreadPool(int nThreads)should be your goto choice for when you need a thread pool. For computationally intensive tasks it will probably give you close to optimal throughput and for IO heavy tasks you won’t be that much worse than anything else. At least until you profile that you got a problem with this kind of executor, you shouldn't mindlessly try to optimize it.
The Default Java ExecutorService: ForkJoinPool
Of course there is the default choice for the Executor on the JVM: the common ForkJoinPool, that is preconfigured by the JVM and is used for parallelizing streams and executing tasks given through the CompletableFuture::supplyAsync and alike. It sounds delicious, right? Your very own preconfigured, always ready and available, state-of-the-art work-stealing thread pool at your fingertips. What else might one wish for?
Well, there’s a caveat. As always, if something is too good to be true, there’s a caveat. The common ForkJoin pool is amazing, except it is common, meaning shared across the whole JVM, used by all and every component that is running in the same JVM process. If you carelessly pollute it with inappropriate tasks, you might stall the performance of the whole JVM process you have. So if you accidentally block a worker thread in the common pool, you’re not doing it right.
ForkJoinPool and ManagedBlockers
The ForkJoinPool was designed with the idea that some tasks might block its worker threads, so it contains the API to accommodate for such blocking cases. Welcome, ManagedBlocker -- the way to signal to the ForkJoinPool that it should extend its parallelism, to compensate for potential blocked worker threads.
Say you want, for example, to query the internet, or one of your web-services through the network. Let’s change the ManagedBlocker example from the documentation to do that. Imagine that we have a Call instance, similar to Retrofit 2 Call, which has all the information about the endpoint to query and how to convert the result back to an object. I’ve recently blogged about getting started with Retrofit 2, and despite that post focusing on Android, the general principles are the same for using Retrofit on the JVM. And it has a really nice API for making HTTP calls, you should check it out!
class WS implements ManagedBlocker {
private final Call call;
volatile E item = null;
public WS(Call call) {
this.call = call;
}
public boolean block() throws InterruptedException {
if (item == null)
item = call.execute().body();
return true;
}
public boolean isReleasable() {
return item != null;
}
public E getItem() { // call after pool.managedBlock completes
return item;
}
}
And now with this code, whenever we want to call Call::execute, we should make sure we call it through the ForkJoinPool::managedBlock method
WS ws = new WS(call);
ForkJoinPool.managedBlock(ws);
ws.getItem(); // obtain the result
Now obviously, this doesn’t make any sense when run outside of the FJP, but on the pool it might be worth the effort. The FJP will accommodate for blocking threads by spawning additional worker threads when needed. Mind you, this is no silver bullet, and most probably it’s also kinda wrong, since ManagedBlocker API was created to accommodate the synchronizers that may block. Here we’re just doing a blocking network call, but it beats the case where we query 4 urls and the computational resources of the FJP are depleted.
Conclusion
In this post we looked at the options that the Executors class offers us, and discussed when it is worth using each Executor strategies. For the CPU intensive tasks, newFixedThreadPool should mostly do the trick, unless you know for sure that another option is best. However, it’s not that easy for the IO bound tasks. One can try to wrap the IO calls into a ManagedBlocker and leverage that the ForkJoinPool is smart enough to enhance its internal parallelism when ManagedBlocker hints are present.
Just to be absolutely clear, this post is not the golden truth about how to parallelize your application tasks, I just wanted to talk about the executors and see where would it lead me. So, stay smart, test your code, understand what’s happening there and where your bottlenecks exist. Then make informed decisions about your threads and pools.
Looking for some additional reading? Our article on Java concurrency is a great place to start.