image-blog-rebel-how-to-improve-microservices-performance-java

March 26, 2020

Common Performance Problems With Microservices

Microservices

Java Application Development

Microservices were supposed to be faster, right? But for many developers working in microservices, the reality is a new layer of complexity on top of already complex applications. For some applications, that means considerable performance problems with microservices.

In this article, we look at eight common performance problems in microservices, and the ideas, techniques, and tools developers can use to resolve them.

Related Reading >> Explore microservices trends and usage in the 2023 Java Developer Productivity Report.

Common Performance Problems With Microservices and How to Fix Them
Additional Resources

Common Performance Problems With Microservices and How to Fix Them

Not all microservices performance issues are created equally. Some, like the N+1 Problem, can be as simple as changing a fetch type. Unfortunately, not all are so easy.

Picking the wrong data store for a service, for example, can mean additional hardware cost, higher risk of timeouts and unavailability, and a bad end-user experience.

Even fixes, as we detail in our section on antipatterns, can create unintended performance consequences for your application.

Dive Into Microservices
Learn more about microservices and Java development through videoes, articles, and more.
EXPLORE RESOURCES

1. Solving N+1 Problems

Object oriented languages like Java often need to work with relational databases. That either means a developer or database administrator needs to write (optimized) SQL requests, or they need to use an intermediary layer, like an ORM framework, that generates compatible requests for that database. While functionally great, ORM frameworks have a reputation for creating unoptimized queries — including N+1 queries.

For an example N+1 problem, our blog on N+1 problems gives a good overview of the process.

2. Using Asynchronous Requests

Determining when to use synchronous vs asynchronous calls has a large impact on application performance. And, depending on the circumstance, calling a service synchronously can cause significant performance bottlenecks for other services and for the combined application.

By using asynchronous requests, a service can make a request to another service and return immediately while that request is fulfilled. That allows for more concurrent work within individual services, and more efficient requests for the combined application.

Keep in mind, developers still need to make sure that the receiving service can fulfill those asynchronous requests within an acceptable time frame, and scale to accommodate request load.

Asynchronous Messaging Technologies

Some of the most popular open source asynchronous messaging systems used in microservices architectures:

Apache Kafka - Apache Kafka is an open source stream processing software platform. It allows developers to publish, process, and store streams of data in distributed and replicable clusters.
RabbitMQ - RabbitMQ is a high scale, high availability open source message broker used for message queuing, routing and more.
ActiveMQ - ActiveMQ is the most popular, multi-platform Java-based messaging server. It’s used for load balancing, availability fail safes, and more.

📕 Related Resource: Need Apache Kafka support? Talk to an expert!

3. Mind Your Antipatterns

Sometimes trying to solve a problem can create a bigger problem. For example, adding timeout and retry functionality to a service sounds like a good idea, but if another service it calls is chronically slow and always triggers the timeout, the retry will put additional stress on an already overloaded service, causing a bigger latency issue than the original fix tried to resolve.

An Asynchronous Antipattern

As we discussed in the last section, asynchronous calls can help to avoid a single slow response slowing down the entire response chain. But developers also need to be careful to avoid antipatterns with these asynchronous calls.

For example, a developer puts a message queue between two services to handle short term call bursts. This helps the service to handle more calls without getting overloaded, but it doesn’t fix the underlying issue — the service is still slow.

In the end, the message queue quickly maxes out, calls start to fail, and the dependent services are more difficult to restart.

To make a bad situation worse, making upgrades to the receiving service is now more difficult because messages in an older format may need to be processed alongside a newer format.

4. Throttling Overactive Services

Is one of your microservices receiving too many requests to handle? Throttling requests or using fixed connection limits on a service by service basis can help your receiving services keep up. Throttling also helps with fairness by preventing a few hyperactive services from starving others.

While throttling does ensure availability of the service for your application, it will make it work slower. But it’s a better alternative than having the application fail altogether.

Technologies for Throttling, Load Balancing, and Scaling

Developers don’t need to reinvent the wheel with every microservice or microservices-based application. Using a service mesh like Istio or Linkerd can help developers to create better performing microservices — without the overhead of in-house solutions for throttling, load balancing, and scaling.

Istio -Istio provides a dedicated layer that facilitates communications between microservices, including security, observability, and traffic management.
Linkerd - Linkerd is a service mesh that provides runtime debugging, observability, security, and traffic management via proxies attached to individual services.

At a logistical level, they can help add network configuration, security, traffic management, and telemetry to your application. At the application level, these services can help to apply resilience patterns like load balancing, retries, failover, and circuit breaker. For deployment, these services can help support canary and blue/green releases for better overall application quality.

5. Managing Third-Party Requests

Even if your microservices are running efficiently with one another, sometimes the limitations of a third-party service or API can cause significant issues for an application.

Using text detection in images? Your requests to the Google API will play a role in your application performance. Authenticating your users with Facebook? If they’re having a slow response, now you are too. Using Amazon Polly for voice recognition? You get the picture.

With the increasing presence of third-party services and APIs within applications, it’s important that developers take proper action to ensure these services and APIs don’t lead to application failure.

Know the Limitations of Third-Party Services

It’s important for developers to understand the limitations of a third-party service before relying on them at scale. Can they keep up with your expected demand while maintaining the performance you require? Is their stated SLA compatible with yours? For example, if you promise 99.99% uptime, but one of your service providers only guarantees 99.9%, your customers will eventually be disappointed and blame you.

Ensure Resiliency to Slow Third-Party Requests

Developers also need to be proactive. Applications must be resilient to slow third-party requests by utilizing best practices like caching, pre-fetching, or using resiliency patterns like the circuit breaker to prevent services from causing cascading failures.

6. Avoiding Application Ceiling

Even properly configured and optimized services can have performance ceilings.

If you’ve already determined that all your requests are necessary and optimized, and you’re still overloading your service, consider load balancing across additional containers to improve scalability.

You might even consider autoscaling to dynamically adjust to incoming request load by adding and removing containers as necessary. If you go this route, be sure to implement a maximum container count and have a plan for defending against DDoS (Distributed Denial of Service) attacks, especially if your application is deployed in a public cloud.

That said, even the best-planned and coded applications and services have hardware ceilings. Applications and services that need to maintain large databases with true ACID transactions, for example, will always need to have enough processing power to fulfill requests to those databases — even if that load is balanced across multiple servers.

Consider clustering technology and potentially moving some of your services to NoSQL solutions that can offer scale higher than an RDBMS. However, be prepared to deal with eventual consistency and compensating operations if you need ACID-like transactions across services.

7. Choosing the Right Data Store

Microservices give the flexibility to use multiple data stores within a single application. But picking the wrong kind of storage can cause significant performance issues.

Imagine the hardware cost for a streaming video service if they used a RDBMS to dynamically update content recommendations for 169 million users based on their individual viewing habits!

It’s important for developers to choose data stores for microservices at a service by service level and to make sure that the selected data store is the best tool for each particular job.

Data Stores for Changing, Unstructured Data

Using a RDBMS for a service that processes a large amount of changing, unstructured data will mean working against the primary benefit of the relational database architecture – consistency. Keeping data consistent across these rapidly changing databases would be unnecessary and expensive.

In these cases, it’s better to use a scalable and schemaless NoSQL data store like MongoDB or Cassandra.

Data Stores for Consistent Data

Traditional relational databases choose to be consistent even if it means becoming unavailable during a hardware or software failure. If your use case allows for occasional downtime and ACID transactions are paramount, or eventual consistency causes more problems than it’s worth, consider a tried and true RDBMS. These workhorse databases don’t grab headlines like they used to, but they’re still as valuable as ever when the use case fits.

8. Caching Database Calls

When a service requests a field of data across multiple databases, each of those databases has the capacity to hold up that request. If that information is frequently accessed by hat service, consider caching that information in an easily accessible place that doesn’t rely on multiple databases.

Making sure your request is targeted at a single, cached database, with rules that add request destinations upon a set request time limit can help make sure your database calls never timeout, but also don’t cause excessive calls.Memcached is used in high performance and distributed systems to store arbitrary data in-memory – allowing for better utilization of available memory. Memcached is used primarily for key-value memory structures.

Redis, like Memcached, is used for high performance in-memory data storage but is more functionally robust. It supports Hash, List, String, Set, and Sorted Set data types, and can also swap cached memory to disk if not used frequently enough to warrant storage in-memory.

Some database systems offer native in-memory caching. Cassandra, for example, can be configured to store data in-memory for Key and Row data types while compacting SSTables by default.

Caching and Projections

In Event-Driven Architectures (EDA), the system of record may be an ordered collection of events that already happened (e.g., “customer created”, “order shipped”, “added $10 to account ABC”).

To determine someone’s current account balance, you might have to look through millions of records to sum all the deposits and withdrawals for a particular account, which is obviously too slow for a waiting user.

In this case, you might create a “Projection” from the main event stream that contains only account balance-related transactions and store it in another data store more suited for quick lookups. At that point, you could specifically cache account balances in an in-memory data store like memcached or Redis for even faster queries if that query becomes your primary bottleneck to performance.

Popular Caching Technologies

Caching can be complicated, but open-source technologies like Memcached and Redis can make caching easier to integrate.

Memcached - Memcached is used in high performance and distributed systems to store arbitrary data inmemory – allowing for better utilization of available memory. Memcached is used primarily for key-value memory structures.
Redis - like Memcached, Redis is used for high performance in-memory data storage but is more functionally robust. It supports Hash, List, String, Set, and Sorted Set data types, and can also swap cached memory to disk if not used frequently enough to warrant storage.

Some database systems offer native in-memory caching. Cassandra, for example, can be configured to store data in-memory for Key and Row data types while compacting SSTables by default.

9. Database Connection Pools

One of the most effective ways to reduce overhead in microservices that access and alter databases (aside from caching) is to pool their connections to that database.

A typical service will establish a connection to a data store, issue some queries, and close the connection very quickly. Connecting and disconnecting adds quite a bit of overhead to a short-lived connection, thus limiting the amount of work that can be done. A connection pool typically establishes a fixed set of connections to a data store when it starts up and lets the calling service re-use an existing connection from the pool instead of opening and closing them with every request. The result is a much more efficient service.

Connection Pooling Frameworks

While in the past, some hardy developers may have needed to create their own connection pooling services, there are now frameworks that make connection pooling in Java easier to implement.

Popular connection pooling frameworks for Java include:

Commons DBCP – The Commons DBCP implements database connection pooling for JDBC.
HikariCP - HikariCP is a high-performance JDBC connection pooling.
C3PO - C3PO is a library that augments JDBC drivers for the enterprise.

Discover how much development time you could save with JRebel during your 14-day free trial.
START TRIAL

Additional Resources

Check out another one of our on-demand microservices webinars or blog posts below.

2024 Java Developer Productivity Report