Professional Documents
Culture Documents
Executive Summary
In this white paper, we give an overview of the best practices to ensure secure,
performant, and resilient Apache Tomcat deployments, including sections on
clustering, load balancing, performance, security, and more.
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
Contents
Closing Thoughts...................................................15
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
applies to server configuration, is installing nodes of rarely have a pure horizontal or vertical cluster
the cluster in geographically different locations. This configuration. Most systems are hybrids. A hybrid
provides safety against regional power outages and other cluster is a mixture of vertical and horizontal clustering to
locational risks like storms and floods. facilitate a specific need and/or to match the hardware
provided.
FINDING THE BEST CLUSTERING APPROACH FOR
YOUR ARCHITECTURE HOMOGENOUS VS. HETEROGENOUS
CLUSTERING
Everyone wants to build a reliable, stable, and available
application container platform. But, in order to do so you Is your setup going to be for multiple applications, or just
need to determine which clustering configuration fits you a few, or just one? Do you have applications that require
and your business the best. specific hardware? This determines whether or not you
decide to use a heterogeneous or homogeneous setup.
In determining your configuration, you must evaluate
the resources at hand. This section will discuss possible A homogeneous setup is very common. Companies will
options for your resources, without actually taking your often duplicate their Tomcat environment, launching
resources into consideration. The next section will make servers on many devices with a simple copy of the
suggestions as to which configurations your company Tomcat directory. A Tomcat cluster that has the same
may leverage depending on the resources available. web applications deployed on all nodes is considered
homogeneous.
VERTICAL CLUSTERING
Homogeneous setups can be hard to keep truly
A vertical cluster expands vertically. A horizontal cluster
identical. Sometimes, especially after node failure and
expands, you guessed it, horizontally. What does this
replacement, it can be hard to synchronize the Tomcat
mean? A vertically expanding cluster has a limited
instances. The best way to do this is to create an image of
horizontal layout. A horizontal layout would consist of
the Tomcat setup from a node designated as the primary
multiple systems and/or resources.
node. As long as this image stays up to date you can
A vertical cluster is on a single machine. A machine can distribute it over as many Tomcat setups as you prefer.
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
Heterogenous setups, on the other hand, can have There are definitely some security (no TLS/SSL support)
different web applications deployed on different nodes and support (no HTTP 1.2 support) considerations
of a cluster. With a handful of applications, heterogenous that need to be made here, but from a performance
clustering can be fairly simple. When an organization perspective the fact that AJP is a binary protocol means it
has a large number of unique web applications, setting can provide some performance benefits.
up, configuring, and maintaining that cluster can be
In most environments, it is not nearly as important as it
complicated.
used to be. Back when 10/100 ethernet was the norm
LOAD BALANCING and bandwidth came at a premium, using a binary
protocol over a text-based protocol like HTTP(s) was a no
Load balancing happens outside of the Tomcat cluster
brainer.
but is still an important consideration for teams deploying
Tomcat. For the purposes of this document, we are Today, where gigabyte ethernet is the standard for
concerned with Apache Httpd server and the built-in load any network backplane, the binary aspect of AJP has
balancing/gateway features, as this is a free and common become a lot less important. Bandwidth constrained
solution within many enterprise systems. However, the environments still exist, so it is common to see small
open source version of NGINX — a common and free Tomcat environments running on remote IoT installations.
load balancing option — can be implemented in similar In such cases choosing a binary based protocol like AJP in
fashion to Apache HTTPD. spite of its security and support issues might make sense.
mod_proxy/mod_proxy_balancer mod_jk
Required if secure Limited load balancing Support for more Insecure protocol, no
communication over configurations, only basic advanced load support for TLS. AJP is a
HTTPS is needed load balancing available balancing options binary protocol, but it is not
encrypted, even when the
Can still be uses with the Does not support Better node failure shared secret is configured
AJP protocol domain- based clustering detections
(mod_proxy_ajp) (introduced in AJP 1.2.8) Has to be built and
Support large AJP packets maintained separately from
No need to compile a Apache HTTPD )
separate module, comes
default with Apache 2.2
and higher
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
and a hardware balance (besides price), is resources. any platform is with the application itself. If you don’t
profile your application, you can’t answer questions like:
An HLB has dedicated hardware resources (RAM),
processor, network adapters, etc. This allows hardware • Do you know how much time your web application
balancers to perform at a much more efficient rate, while spends in garbage collection?
providing more features. This is also an infinitely more • Do you know how many threads your app uses?
expensive method, as you can find many free open • Do you know what its longest running query is?
source load balancing solutions. In most cases, these
• How many disk operations does your app perform
software-based open source solutions are perfectly
at any given time?
acceptable. However, some use cases may require a
dedicated hardware solution. Applications that see • Do you know the answers to these questions when
sustained high usage, (e.g., with concurrent users in you have 10 concurrent users? How about 100?
In addition to ultra-high usage profiles, applications that to be one of the most overlooked aspects of application
require global load balancing would need to consider development we run into.
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
Building an application profile does not only have to Wondering what your JVM memory sizing should
occur when doing load/performance testing (though be? Wondering if your application would benefit
it should; more on that later). We can build a real-world from enabling compressibleMimeType? Load and
application profile over time by making sure we are performance testing will tell you.
collecting the right data and metrics.
We all know how tempting it is (and how often it
By making sure we are collecting garbage collection happens) to skip load and performance testing, but just
logs, capturing regular thread dumps, performing access like the previous tip, this should be baked into every
logging, gathering database call times, etc., we can release cycle. With modern CI/CD tools like Jenkins and
make sure we have the data to review over time to get GitLab, automating your load and stress testing to have
a better understanding of how applications are running them run in your CI/CD pipeline is easier than ever.
and what they look like in a real-world scenario.
NEEDS BASED TUNING
Of course, there is not much value in collecting this data
Most Tomcat tuning is “needs” based tuning; in other
if no one is looking at it. A review of an application’s
words, we are tailoring the Tomcat configuration to the
performance profile should be baked into every
given application it is running. Now that we know what
release cycle.
our application and performance profiles look like we can
AUTOMATE YOUR LOAD AND start building out environments to match these profiles.
STRESS TESTING Does the application churn CPU? Does it need a ton of
memory for a large JVM? Does it have a lot of disk I/O?
As mentioned in the previous section, you don’t have to
Now that we know these things, we know how to build
do load testing to build an accurate application profile,
out the compute environment in a way that matches the
but you should.
needs of the application and the projected users that are
In a perfect world, we would have a 1 to 1 replica of our going to be using the system.
production environment to perform load and stress
We can also start making fact-based decisions on specific
testing, but rarely do we find this to be the case. Granted
Tomcat settings like number of threads, max keepalive
it is a lot easier to spin up a production like environment
times etc. Does your application have a lot of long
these days compared to 20 years ago when we were
running queries? If so, then we may need to set longer
doing this all on bare metal, but you don’t have to have
keepalive timeouts than we normally would. Or, on the
a perfect production replica to perform worthwhile load
other end of the spectrum, does the application perform
and performance stress testing.
a lot of short duration queries but fire off a lot of them at
Taking a scaled-down model of your production one time? In that case we would want to consider short
environment and pushing it to its breaking point with “keepalive” timeouts and a larger number of connections
a utility like JMeter will still provide your team with in the connection pool.
invaluable information and create a baseline that the rest
of your performance tuning can be based upon.
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
Following known security best practices for any piece of DISABLE SSLV3 PROTOCOLS
software is important. And, as we show below, many of
POODLE was a well-publicized attack that targeted
the basic security best practices for Tomcat are ones you
the SSLv3 protocols, so you will need to disable that in
can apply to other pieces of software within your web
Tomcat before you get it up and running. Maintaining
applications.
detailed logs is also key to ensuring your Tomcat server
That said, Tomcat does carry some default exposures and environment security. This applies to user access,
that need to be corrected on new deployments. The best application traffic, Tomcat internals, the OS/firewall, etc.
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
is disabled by default, so if you enable it, be sure you servers, you could consider a hybrid cluster. If there are
treat it appropriately. servers available with two or more processors and a large
amount of ram (8 gigabytes or more) this would be ideal
USE REALMS TO CONTROL RESOURCE ACCESS for multiple Tomcat instances. In this configuration you
Realms are another method of controlling access to can set up a hybrid cluster by running multiple instances
resources in Tomcat. Realms are components that access of Tomcat on multiple machines, and multiple instances
databases of users that should have access to a given of Apache Httpd to handle the load of load balancing.
application or group of apps, and the roles and privileges This configuration could look something like this:
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
USE CASE #2: MULTIPLE APPLICATIONS There are many things to take into consideration when
designing and building your cluster. If a large company
Tomcat can also accommodate organizations that want
is relying on you to provide a reliable, highly-available
to deploy multiple applications. How the applications
application implementation, then clustering and load
are divided into Tomcat nodes is up to the user. However,
balancing is the right choice. Regardless of if you are
the configuration of these nodes in Tomcat will be a little
new to clustering, or an old hand, purchase a support
more complicated.
contract. There are companies that will provide open
If an organization has an application that requires heavy source software support for your Tomcat and Apache
processing and substantial amounts of RAM (HPR1,) you Httpd configuration. This will allow you to offer your
can set up this application on two nodes by itself. After customers an extremely reliable, available service while
this, take the remaining applications (GUI) and place at the same time providing someone to turn to if you run
them on two different nodes in the cluster. This will into problems.
prevent the GUI application from being bogged down
when HPR1 is consuming the CPU and RAM. This cluster Cluster Setup and Configuration
might look like this: Overview
This is not a complete step-by-step tutorial on cluster
creation, but we will provide you with the tools you
can use to implement a cluster rapidly and effectively.
Internet - Incoming Request
Whether you have created many clusters in the past or
this is your first attempt, we hope that you will be able
HPR App Requests GUI App Requests to learn something, whether it be basic or advanced,
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
SERVER.XML 1
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
SERVER.XML 2
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
CREATING A TOMCAT CLUSTER duplicated to all other servers in the cluster. If your
application creates session data for a user, and you have
Tomcat clustering is quite simple to set up. However,
a heterogeneous cluster, the session data will still be
if you wish to leverage clustering in your enterprise
replicated across the other nodes.
environment the default configuration is not going to
be the best route for you. To turn on clustering in your A heterogeneous configuration is one that does not have
Tomcat server all you must do is add one line of code to all of the same applications on every node. Therefore,
your server.xml. if application A stores session data for a user, and
application A is running on server A, but not server B,
<Cluster className=”org.apache.catalina. session data will replicate to server B, even though there
<distributable/> DeltaManager”.../>
This tells Tomcat that this application is designed to run The DeltaManager replicates all changed session data
on multiple nodes in this cluster. to all nodes of the cluster. The BackupManager backs
up session data to a specific backup node. For large
SETTING UP SESSION REPLICATION clusters the BackupManager is the option to go with,
The default session replication mode is “All to All,” for smaller clusters it is common to just use the default
meaning any session data created on a server will be DeltaManager.
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
In Tomcat 5, you could not choose the specific session Channel send options control how messages are sent
manager for your application. In Tomcat 10, you can between cluster nodes. Are these messages sent
define a manager in the cluster configuration, as you synchronously? Or, in basic terms, does the thread
could in earlier versions, but you can also define a that sends the message need to wait until the message
manager in a web application’s context. has sent before continuing to work, in turn, potentially
making the users request wait on this message to be
Defining the Manager in your clustering configuration
sent? Sending the messages asynchronously is when the
provides a default setting for applications that do not
thread generates and sends the message but does not
provide their own Manager configuration. For instance,
stop and wait for this to happen. Instead, it does this by
the following code will set all applications in your cluster
spawning a worker thread.
to use the BackupManager for session replication.
As you can see, this is just one aspect of the channel send
options, and it is a lot of information. To go over channel
<Manager
send options in detail will require a whole default channel
className=”org.apache.catalina.ha.session.
send mode that is asynchronous.
BackupManager”.../>
WRAPPING UP
CHANNEL SEND OPTIONS Tomcat clustering is a powerful tool that can provide the
After setting the Manager, you might need to apply a high availability, reliability, and dependability that your
non-default channel send options value. Channel send company requires and all of it can be set up with little
options is a setting specified on the cluster object. For effort. That said, this paper barely scratches the surface of
channelSendOptions=”6”>
expect minor problems along the way, like nodes not
joining a cluster, session information being lost, random
node crashes, and configuration issues.
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)
WHITE PAPER
About Perforce
Perforce powers innovation at unrivaled scale. Perforce solutions future-proof competitive advantage by driving quality, security, compli-
ance, collaboration, and speed – across the technology lifecycle. We bring deep domain and vertical expertise to every customer, so nothing
stands in the way of success. Our global footprint spans more than 80 countries and includes over 75% of the Fortune 100. Perforce is trust-
ed by the world’s leading brands to deliver solutions to even the toughest challenges. Accelerate technology delivery, with no shortcuts.
Get the Power of Perforce.
www.openlogic.com OpenLogic by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0520TKP23)