Java Application Server Rankings Blog
May 21, 2013

The Great Java Application Server Debate

Enterprise Development
Java Application Development

What type of Java application server should you choose for your next project? Well, that kinda depends on what kind of app you’re building, what your needs are, what type of organization you work in, and lots of other factors too. Hence the debate. So perk up your ears, and get ready for a showdown.

What makes an App Server
Tooling support
Server Configuration
Documentation & Community

What is a Java Application Server?

What makes an application server is a contentious question to many, as the definition is unclear. Wikipedia, the unquestionable source of all knowledge, states:

“Java Platform, Enterprise Edition or Java EE (was J2EE) defines the core set of API and features of a Java Application Server.”

Well, sorry Wikipedia, but for the sake of this report, we don’t care about pedantic definitions and full Java EE implementations, we care more about what a developer wants and uses.

Most developers work on web applications and rarely use all of the bells and whistles that come with the EE specification. In fact many of the application servers available today with only the basic functionality are the most used, as our Developer Productivity Report section on application servers recently showed:

java application servers

What we’re looking at in this part of the report are the real developer concerns and metrics including:

  • Ease of download and installation

  • Real performance metrics

  • Tooling support

  • Server Configuration

We’re going to compare and contrast many aspects of these java application servers, scoring each and placing each of them based on our findings. We then address the big questions: Which app server is the best? Doesn’t someone in a big company have different needs than a hobbyist developer geek working at home? How should we weight what’s important to different profiles of people? And so on...

The Application Servers we will be discussing in this report are Tomcat, Jetty, GlassFish, IBM WAS Liberty Profile and JBoss (aka WildFly):

java application servers

At first, we were going to include IBM WebSphere and Oracle WebLogic into this report, following the same pattern as with the other app servers. But it felt unfair, as these servers are really targeted for large-enterprises and apps in production rather than lightweight development; however, we didn’t want to dismiss them completely, so they’ll have their own section at the end :-)

Ranking the Top Java Application Servers

When you’re getting started with a new tool, things like installation, configuration, tooling support and documentation can affect your experience in a major way. In Part I, we look at starting off with each of the app servers.

This section will compare and contrast each of the Application Servers for each of the following areas ranking them with a score out of 5 in addition to our comments. As we at ZeroTurnaround are all about developer productivity, the unit of measure will be the Rebel:

 zeroturnaround-logo-rebel

  • Download and Installation

  • Tooling support

  • Server Configuration

  • Documentation & community

Next, in Part II will look into some of the deeper topics as we go undercover, Ninja style.

Download and Installation

The first contact is all important as it often gives your initial glimpse of how the rest of the experience might turn out!

Jetty

Jetty is just 8MB in size! The process is easy:

  1. Download installation package

  2. Extract the archive

  3. You are ready to go!

Jetty has preserved its simplicity in starting: java -jar start.jar

Score:

 score-5

Reason: Smallest download, maven dependency integration, unzip install and good startup scripts.

Tomcat

Tomcat is identical in experience to Jetty, although it’s another few meg at 12.8.

  1. Download installation package

  2. Extract the archive

  3. You are ready to go!

To start Tomcat, simply run bin/startup.sh - easy

Score:

 score-5

Reason: Small download, maven dependency integration, unzip install and good startup scripts.

JBoss

JBoss AS 7.1.1.Final, which was release on March 9th 2012, but still is the latest community edition version, archive is 127Mb.

  1. Download installation package

  2. Extract the archive

  3. You are ready to go!

To start, RedHat have made it simple as well with the bin/standalone.sh script, unless you want to run a clustered environment. Also there are several default configuration files in “standalone/configuration” directory, which allow to turn on and off default clustering and choose between web or full EE profiles, so one can pick the closest to what is needed and tweak it minimally.

Score:

score-4

Reason: Largest download, unzip install, slower, “standalone” named startup scripts.

Liberty Profile

  1. Accept Licence agreement

  2. Download installation package

  3. Extract the archive

  4. Accept Licence agreement again - seriously?

  5. You are ready to go!

Although you have to accept the license agreement twice, it’s not all that bad as most people just click accept anyway :) Does anyone actually read them? The download is 41.4 MB which isn’t too bad given the extra features it has over say Tomcat or Jetty.

Score:

score-4

Reason: Reasonable size, license have to be accepted twice, startup scripts need docs :(

GlassFish

There are two options available: GlassFish Open Source Edition and Oracle GlassFish Server. We grabbed the GlassFish Server Open Source Edition installation package, which surprisingly, was not so big, only 53 MBs in size! This is the installation package download and comes in a self-extractable archive. Once downloaded you need to run a large .sh “script” and after a number of steps through a graphical wizard, GlassFish was in place. If you don’t like the idea of an installer, you can also choose the unzip install option which comes in a download around 30MB larger. This gives you the three step experience as most of the others:

  1. Download installation package

  2. Extract the archive

  3. You are ready to go!

Score:

 score3-5

Reason: No straightforward way to start the server, multiple install methods including unzip install is good, reasonable download size, but still large.

Download and Installation Scores: Jetty and Tomcat Win

java application servers

Tooling Support

Developers are as loyal with their IDEs as they are their appservers, so it’s important for Application Servers to support the main IDEs so they don’t force their developers into using a platform they don’t want to, or missing out on Tooling support entirely.

For many, good integration with build tools can be as important as good IDE integration, sometimes more important. The two main build tools used which we will be looking at are maven and ant.

Jetty

Jetty has an Eclipse WTP plugin (http://wiki.eclipse.org/Jetty_WTP_Plugin) that enables you to run web apps directly from Eclipse. Also, check out Run Jetty Run, another eclipse plugin.

JetBrains also has a listed plugin for IntelliJ as does NetBeans and can also be leveraged using Jetty’s Maven plugin and NetBeans together.

Jetty’s build tool integration is quite well developed. It’s has integration with Ant, Maven out the box and can also be integrated with whichever tool you like, with minimal effort.

It is possible to configure your development environment very closely to the one that you are running in production. The maven plugin for Jetty allows you to configure almost everything in the pom.xml file.

The thing that we liked with Maven’s Jetty plugin is that Jetty is so tiny. So it won’t take long for Maven to download all the internets.

Score:

 score3-5

Reason: The Eclipse plugin for earlier versions of Jetty is not great, light server management support.

Tomcat

The big three IDEs, Eclipse, IntelliJ IDEA and Netbeans, all have integration support for Tomcat out the box. This means you can import a server into the tooling and deploy projects onto the server. NetBeans offers a distribution which wraps a Tomcat image so that you can deploy as soon as NetBeans installs :o) Eclipse also has the additional download feature which allows you to download and install Tomcat from within your tooling, meaning you don't have to do it via a browser and later import it.

Tomcat seems to be very tightly integrated with Ant, and most documentation includes ant examples rather than Maven, although Maven has good integration as well.

Score:

 score-4-5

Reason: Great IDE support, and download/install through eclipse gives a good experience

JBoss

The umbrella project for multiple Eclipse plugins, jboss-tools, includes among other things JBoss Developer Studio which has a server adaptor for JBoss Servers and allows you to manage jbosses from your IDE. IDEA has a closed source “JBoss Integration” plugin to manage servers, and NetBeans has integration and ability to manage AS 7-th since version 7.2.1.

There’s a maven plugin to manage your jboss instances. Its functionality includes starting and stopping, deploying and redeploying apps, managing resources and executing commands like through a jboss CLI.

Score:

 score-4-5

Reason: Good cross IDE support, able to make light server config changes, good maven support

Liberty Profile

The Liberty Profile tools, named IBM WebSphere Application Server Developer Tools for Eclipse (a product name that almost needs punctuation) is only supported on the Eclipse platform. If you’re an IntelliJ IDEA or NetBeans fan, you’re out of luck. The tools let you download and install the whole server easily, and provides feature rich development for the programming models which the Liberty Profile supports. It also provides rich editor support for server config changes in a similar style to a deployment descriptor editor which you’ve likely seen/used. Overall the tooling is good, on Eclipse.

The Liberty profile also includes a Maven plugin which provides allows you to drive some actions to a server, You can install, start, stop, package and create server using predefined goals. Driving these is fairly easy:

mvn liberty:package-server -DserverHome=/path/to/server_home -DserverName=[server_name] -DpackageFile=/path/to/packaged server file location

Score:

 score-4

Reason: Great eclipse support and rich server config changes. Good Maven support, but lacking in cross IDE support.

GlassFish

GlassFish has plugins available for all major IDEs these days. But the problem with the plugin is that the information is scattered around. The plugin for NetBeans is already bundled with the IDE, as you’d expect given the vendor behind it, but others need to find the correct website to install the plugin. Fortunately, GlassFish Tools are available in the Eclipse Marketplace for Eclipse users - this makes Eclipse user’s lives a lot easier.

IntelliJ IDEA users are lucky too - they have the plugin already bundled with the IDE.

GlassFish has also plugin for Maven and Ant which are quite well developed, enabling functionalities like deploy, undeploy, start and stop the server.

Score:

 score-4-5

Reason: Info about where to find plugins is poor, great IDE support once the plugins were found (particularly with NetBeans and Eclipse).

Server Configuration

Jetty

You can make your own XML based configuration file and pass it along with the startup command:

java -jar start.jar /path/to/your-jetty-conf.xml.

Or those who are impatient, can just start up their instance with JVM arg settings passing in additional config. Also, it is possible to pass multiple configuration files to Jetty like configuration for HTTP and HTTPS, which is very useful for sharing snippets of config around a team.

Jetty’s XML based configuration is reflection-based. This means that all options in XML are actually corresponding to the Java class fields.

The biggest downside with this reflection-based (not so well documented) approach is that you need to understand how Jetty works under the covers and learn some of its internals. But hey - if you know your server well, you’ll never run into anomalies and won’t let the server to go out of control, right?

Score:

 score3

Reason: Able to create Jetty config but need to know Jetty internals, restarts often required, JVM args to override config is a nice feature, config is small.

Tomcat

Configuration in Tomcat is scattered across a number of files in the tomcat/conf directory but mainly resides in the server.xml file. This file can be modular to allow reuse and sharing across a development team. If you wanted to make a quick change, perhaps one that needed to be undone on next restart for a test, the best way is to add a system property when starting the JVM up as system properties override xml configuration.

The default configuration file is quite verbose, but this is actually mostly comment lines with ‘how-to’s. If you look at the active lines of xml, it’s actually a very small file.

Every server.xml file will require you to recycle the server as the configuration files are only checked by the core runtime during server startup.

Score:

 score-4

Reason: Restarts required for config changes, scattered across multiple files, small file, easy to update, nice examples in comments.

JBoss

The domain model is quite understandable and straightforward, so getting the setup of your dreams is easy. Here is how one would enable ssl connections on some arbitrary port. Locate the following part of the standalone/configuration/standalone.xml and add a connector element for “https” (bold line in the following table). 

This will enable connector, but if you also want to change the port where https binds, change “” element to have the desired value. The whole default config is just 15KB (300 lines) of xml, located in a very predictable location. It also contains a couple of larger sample configuration for clustering. Changes are pretty static, configuration files are read at the start and isn’t reread automatically. However, a CLI command: [jboss-as-7.1.3.Final]$ bin/jboss-cli.sh --connect :reload This will do the thing for you without restarting the server.

Score:

 score-4

Reason: Single file, 300 lines of xml :( Easy to config, well structured “subsystems” configuration, CLI reloads rather than restarts (manual process), several examples out-of-the-box.

Liberty Profile

The Liberty Profile has a single config file called server.xml. This file can include other config files if desired, so that the config can be logically separated or shared across a team. The default config file is tiny and the most interesting part is the feature manager:

   jsp-2.2

This allows a user to construct their application server with the features they want to run, so initially, only the jsp-2.2 feature is enabled and running. Oh and by the way, you can change and update the server.xml and see the changes reflected in the running server. Neat!

Score:

score-5

Reason: Top class config model, dynamic updates mean there’s no need to restart, small file, simple config.

GlassFish

Everything that is needed to manage and maintain a GlassFish server is put into the asadmin utility. It is even possible to manage remote servers with it, which is cool!

One thing that asadmin does well at is scripting. Yeah – it is possible to write various asadmin scripts for managing your deployments and servers. These scripts can be executed inside asadmin as if you are running regular shell scripts.

Some configuration changes require that you restart the DAS or GlassFish Server instances for the changes to take effect. Other changes are applied dynamically without requiring that the DAS or instances be restarted.

You can determine whether a domain or instance requires a restart from asadmin

asadmin> list-domains
domain1 running, restart required to apply configuration changes Command list-domains executed successfully.
asadmin> list-instances pmd-i1
pmd-i1 running; requires restart
Command list-instances executed successfully

 

With dynamic configuration, changes take effect while the DAS or instance is running. The following configuration changes of developer interest do not require restart:

  • Adding or deleting add-on components
  • Adding or removing JDBC, JMS, and connector resources and pools (Exception: Some connection pool properties affect applications.)
  • Changing a system property that is not referenced by a JVM option or a port
  • Changing logging levels
  • Enabling and disabling resources and applications
  • Deploying, undeploying, and redeploying applications

Score:

 score3

Reason: Config is hard to get at, hard to read xml file, asadmin is the recommended approach, but it’s hard to get started with, need docs and experience.

Documentation & Community

Jetty

The Jetty team has done a  lot of work recently on improving and structurizing their docs. Currently there is a “Jetty Documentation Hub” available, which is easy to read and navigate. It features docs for different Jetty versions. There are different mailing lists for Jetty developers users available. No official forums that we could find, though.

To get some professional and commercial support you could find a third party company that can provide this. Unfortunately, the Jetty website does not have a list of such companies available.

Score:

 score3

Reason: There is no list of “real experts” or companies, who can provide support, provided.

Tomcat

The documentation for Tomcat is very good (particularly for new users) as is supported by a vast community. This same community is where the Tomcat support comes from, both as descriptive help as well as code changes and bug fixes. Most people will be comfortable with this, but if you need more of a guarantee, there are vendor support contracts available, which very often include Tomcat committers.

Score:

 score-5

Reason: Great, vibrant community, established over years. Responsive, docs are great, site is easy to use, support via the community, 3rd party vendors available for more guaranteed support.

JBoss

The JBoss community is one of the best things one can think of regarding JBoss AS7. They have a myriad of projects under their wing and they work together well. If not, almost everything can be found on forums or discussion groups. If everything else fails you can read the documentation, which is quite good.

Score:

 score-4

Reason: Slower releases, but docs are good, large community.

GlassFish

Crawling through the docs, forums and mailing list gives you a secure feeling. The forum is quite active, there are many mailing lists specializing on different parts and areas of GlassFish and the documentation looks pretty good.

Score:

 score-4

Reason: good docs, awkward to navigate to, some docs are outdated, good sized community, mailing lists, active forums.

Liberty Profile

Documentation can mostly be found on the new community site, WASdev.net. It’s run by the development team so isn’t full of the usual fluff. There is also the traditional info center which provides more formal documentation for the product. You can get support from the WASdev forum which is reasonably active, and again, you’re talking with the development team. This is best can do support, without guarantees. If you choose to pay the big bucks you’ll also get the full IBM support where you can afford to scream at IBM and still get help :o)

Score:

score-4

Reason: Good quality docs & sample code, small community, but growing good response times on forum, often very corporate answers as not much can be said about futures, slow bug fixes.


Now there is more to this report. You've read the part 1 of our findings, where we scored the servers based on their download and installation easy, community, documentation completentess, etc. However, we also did some performance measurements, tried to match the servers and their strength to the roles in a team to see who would prefer what server, and much more!

Want to continue reading about Java application servers? Read our write-up on Jetty by clicking the button below.

Read the Article