InfoSphere Guardium S-TAP Performance V9 GPU 50

Usage: To be given directly to customers as requested. Do not distribute widely.

For questions about this document, contact Kalpana Doddamreddy (kalpana@us.ibm.com) or Kathy
Zeidenstein (krzeide@us.ibm.com)

The information contained in this document has not been submitted to any formal IBM test and is
distributed AS IS. The use of this information or the implementation of any of these techniques is a
customer responsibility and depends on the customer’s ability to evaluate and integrate them into the
customer’s operational environment. While IBM may have reviewed each item for accuracy in a specific
situation, there is no guarantee that the same or similar results will be obtained elsewhere. Anyone
attempting to adapt these techniques to their own environments does so at their own risk.

Any performance data contained in this document were determined in various controlled laboratory
environments and are for reference purposes only. Customers should not adapt these performance numbers
to their own environments as system performance standards. The results that may be obtained in other
operating environments may vary significantly. Users of this document should verify the applicable data for
their specific environment.

InfoSphere Guardium S-TAP Performance V9 GPU 50 .................................................... 1

Introduction......................................................................................................................... 2
Test Environment................................................................................................................ 2
Database servers and operating systems ......................................................................... 2
Workload......................................................................................................................... 2
Policies............................................................................................................................ 3
Results................................................................................................................................. 4
Observations ....................................................................................................................... 4
Factors affecting performance ............................................................................................ 5
Notes ................................................................................................................................... 7
Notices ................................................................................................................................ 7

IBM InfoSphere Guardium S-TAP CPU performance (Do not distribute) 1

Introduction
InfoSphere® Guardium® uses a lightweight, host-based agent that monitors all database
activities on a server, no matter what database type and no matter what type of
connection is used to access the database (TCP, shared-memory, Oracle Bequeath, named
pipes, TLI and IPC connections, etc.). This agent is known as software TAP (S-TAP) and
is responsible for monitoring all database activities in a way that is non-intrusive to the
database.

The S-TAP relays database activity to a Guardium server, known as a collector, for
processing. The S-TAP uses patented technology to ensure that very little resources are
used on the server so that it can be used even on very highly-loaded servers and so that it
does not impact the availability of applications.

You can use policies not only to determine what needs to be logged to meet the security
and audit compliance requirements of your organization but also to determine what NOT
to log, which is important from a performance perspective as well as to reduce overall
noise.1

The most important characteristic for the S-TAP is its nonintrusiveness. One part of non-
intrusiveness is the S-TAPs ability to monitor all activity without requiring changes to the
database configuration and without requiring native auditing or traces. But the second
important characteristic is its low resource consumption. An important design goal for the
S-TAP is that it must require negligible resources from the operating system where it is
installed so that it does not have an impact on the database environment. In the field, S-
TAP resource utilization has been observed to be not more than 5%, no matter what the
auditing policy, and in most cases well below 2%.

This document presents CPU results for Guardium S-TAP Version 9, GPU 50 on the S-
TAP, which bears out results that we have seen in the field.

Test Environment
Database servers and operating systems
To represent typical production customer environments, we ran our tests on three
different database servers (DB2®, Oracle, and Microsoft SQL Server), different
operating systems, and even different processor sets (PowerPC® and Intel Xeon). Note
that if you use older, slower processor than used in our tests, you may see slightly higher
percentage of S-TAP CPU.

Workload
In our testing, we ran a mixed SQL workload on the database servers. Our goal was to
keep driving traffic up to a point somewhere below where we would see dropped
messages on the collector, which would indicate that the S-TAP is working faster than

IBM InfoSphere Guardium S-TAP CPU performance (Do not distribute) 2

the collector appliance can keep up. Note that the hardware and software configurations
are different among all the databases so there are differences in the amount of traffic we
were able to drive through.

 For DB2, we ran 100 concurrent sessions with 2 ms delay between SQL
commands. We pushed approximately 15K SQL commands per second.
 For Oracle, we ran 50 concurrent sessions with no delay and pushed
approximately 21K SQL commands per second.
 For Microsoft SQL Server we ran 60 concurrent sessions with 1 ms delay. We
pushed approximately 16.5K SQL commands per second.

We measured the CPU overhead for the S-TAP on the database server while running the
workloads. The workload consisted of 60% short SQL, 30% medium SQL, and 10% long
SQL as measured by length in bytes:
Short: 100 bytes
Medium: 500 byes
Long: 1000 byes

We ran the workloads for 5 minutes and repeated the tests at least three times to get an
average.

Note: Since this paper is focused on S-TAP performance, not throughput or collector
performance, we did not use options such as workload balancing, which can be used to
spread session workload across multiple collector appliances and improve performance
on the collector.

Policies
Policy rules play an important role in enforcing security policies as well as tuning
performance. The more filtering that can be done on the database server side, the less the
impact on the collector. The S-TAP overhead itself is improved as well mainly in the
reduction of cost of managing the network traffic overhead.

The following policies actions were used in our measurements.

 Ignore S-TAP session - This policy only logs session information (login / logout
info) to the collector. All other activity within sessions is not sent to the collector.
This is a very useful option to consider to filter network traffic and to reduce
storage and processing on the InfoSphere Guardium collector.

Hint: To see the number of bytes that were not sent to the collector because of
use of this policy rule, you can create an S-TAP statistic report with the column
Total Bytes Ignored.

 Allow All – This policy rule sends all activity to the collector. On the collector
side only unique constructs (SQL templates) are logged and aggregated within a
specified access period (usually 1 hour), and all values are redacted so no

IBM InfoSphere Guardium S-TAP CPU performance (Do not distribute) 3

 sensitive information is stored on the appliance.

 Log Full Details- This policy sends all activity to the collector. On the collector
side, all statements, including string values, are logged, with the exact timestamp.

Results
Even with these high workloads, the S-TAP CPU overhead was well under 4%, and was
generally under 2%. The slightly higher CPU percentages on Microsoft SQL Server are
typical of what we see and are due to internal implementation interactions between the S-
TAP and the operating system.

Table 1.Summary of S-TAP performance results

avg
STAP
*
Database OSType Processors (cores) Memory Policy Rule CPU%

32
Linux Red Intel® Xeon® CPU
Oracle Hat 5.9 E5-2665 @ 2.40GHz 64 Ignore 100% <0.1
Allow All 0.7
Log Full Details 0.6
8
PowerPC_POWER6
DB2 AIX 6.1 @ 4.6 GHz 32 Ignore 100% <0.1
Allow All 1.3
Log Full Details 1.3

32
MS SQL Intel® Xeon® CPU
2008 MS 2008 E5-2665 @ 2.40GHz 64 Ignore 100% 0.43
Allow All 3.45
Log Full Details 3.33
*
The nmon performance tool for Linux and AIX only returns data to one decimal place. Any differences betweeen
Allow All and Log Full Detail S-TAP CPU overhead are negligible.

Observations
As you can see from our results, there was minimal difference between Allow All and
Log Full Details policies. As expected, you can see the most gain with use of the
IGNORE S-TAP SESSION policy action, and this should be considered for trusted
connections only. Remember, with this option only session login and logout information
is logged, so you cannot go back later and do forensic analysis on any detailed commands
for the session. However, by using connection profiling reports to analyze connections,
you may over time be able to filter out more and more such trusted connection traffic.

IBM InfoSphere Guardium S-TAP CPU performance (Do not distribute) 4

With some exceptions,2 the S-TAP process is single-threaded, so is utilized on only one
core of a multicore processor. So the most CPU that could ever be consumed on a
multicore system is 100% of one core; this situation is to be avoided since this could
cause performance degradation on the server.

For Windows, the S-TAP spawns multiple threads for various tasks. It generally can
spawn anywhere from 4 to 10 threads. More threads will be used if there are multiple
appliance connections and multiple instances of MSSQL.

Determining what can be considered excessive CPU depends at least partly on your
configuration. For example, on a Linux system if the S-TAP CPU utilization is
consistently running at 5% on a system with 16 cores, it indicates that S-TAP is
consistently consuming 80% of one core. If S-TAP is consistently running that high, it
leaves very little room to accommodate any additional spikes in traffic.3

Factors affecting performance

The Deployment Guide for InfoSphere Guardium Redbook3 includes a good discussion
on S-TAP performance factors. Items such as encryption, firewalls (blocking) and so on
can increase S-TAP CPU usage, so use of these options should be implemented with
proper planning and thought. For example, blocking should be implemented only for
specific connections, such as privileged users or likely hackers. If you accidentally turn
on blocking for your production application traffic, this could lead to serious performance
degradations.

Encryption in the form of Transport Level Security (TLS) between the S-TAP and the
collector has a significant impact on performance. Configuring the S-TAP with TLS
requires extra encryption time that might affect performance on the database server where
the S-TAP agent is installed. The appliance (collector) also requires time to decrypt this
traffic. If this configuration makes sense in your environment, you will need to plan in
advance for this. 2,3

Network bandwidth is an issue we see in our customer deployments as well. The next
section goes into this in more detail.

There are also S-TAP parameters that can help improve performance in the right
circumstances, and the Redbook goes into detail on those as well. For example, for Linux
and UNIX operating systems, one recommendation is to set ktap_fast_tcp_verdict=1,
which preloads TCP port information into K-TAP when S-TAP starts up and can reduce
the likelihood of experiencing database performance degradation. Please be sure to see
the Redbook for limitations on using this option.

IBM InfoSphere Guardium S-TAP CPU performance (Do not distribute) 5

Network bandwidth and filtering

S-TAP performance can be adversely affected by insufficient network bandwidth to

accommodate the volume of data being sent by the S-TAP to the Guardium collector.
This issue is most prevalent when the Guardium collector and host database server are
not in the same datacenter or LAN, but can occur otherwise as well.

This performance test did not focus on the network bandwidth required by S-TAP. The
reason is that the network utilization for S-TAP is directly proportional to the network
utilization of the database and the audit policies defined within the Guardium server; it is
so directly proportional that it is almost possible to compute these numbers. As a result,
we believe it is better to explain this matter so that customers can easily predict and make
choices related to network overhead.

The Guardium S-TAP intercepts the client-to-server communication and the server-to-
client communication and forwards this data to the Guardium collector appliances.
Barring any filtering, network activity will double the database traffic. For example, if the
database server has an average of 10Mbps of incoming queries and an average of
20Mbps of outgoing result sets, then the S-TAP will send an average of 30Mbps from the
database server to the Guardium system (assuming extrusion rules are being used). Note
that this “doubling” is only of real database activity.

If the goal is to reduce network traffic there are appropriate controls to filter out
information, which is beyond the scope of this paper. For example, "Ignore result sets"
would ignore virtually 20Mbps of traffic in the example from the above paragraph.
Another control is to "Ignore S-TAP session" which is typically used for "authorized
connections" which have appropriate auditing controls, so all traffic from this session
would not be sent to the appliance. The S-TAP architecture is very mature and placed in
the largest organizations in the world under real world conditions. Please talk with a
Guardium representative if you have concerns about network overhead.

Other network packets that do not involve on-line database activity (including non-
database activities such as SSH, SCP, file copies, backups etc.) are not sent from the S-
TAP to the Guardium server.

Policies can be applied to reduce this network overhead by specifying in a

very granular manner, which connections need to be monitored and whether result sets
need to be monitored (to enforce extrusion rules). For example, if the project only
requires that privileged users be monitored and these connections only account for 5% of
the total database activity then the network packets sent from the S-TAP to the Guardium
server will only add 1.5Mbps.

IBM InfoSphere Guardium S-TAP CPU performance (Do not distribute) 6

Notes
1. 4-Part educational video series on Data Flow and Policy Rules. IBM Guardium
Community on developerWorks.
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W
f32fc3a2c8cb_4b9c_83e4_09b3c6f60e46/page/InfoSphere%20Guardium%20Poli
cies%20Deep%20Dive
2. The DB2_Exit and the connection_pool configuration option are both threaded.
Connection_pool is a UNIX S-TAP configuration option that opens up more than
the default level of 1 connection to the collector and can be used in some use
cases where there is more than one processor core that can be utilized. For
example, this may be useful for TLS connections, but the potential to raise overall
database server CPU overhead also increases with the additional thread usage.
3. Barkai, Boaz, et al. Deployment Guide for InfoSphere Guardium. IBM Redbook.
2014. http://www.redbooks.ibm.com/redpieces/abstracts/sg248129.html?Open

Notices
© Copyright IBM Corp. 2014. IBM, the IBM logo, DB2, PowerPC, InfoSphere, Guardium, and ibm.com® are trademarks
or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other
product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available
on the Web at “Copyright and trademark information” (www.ibm.com/legal/copytrade.shtml)

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States
and other countries.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States,
other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.

IBM InfoSphere Guardium S-TAP CPU performance (Do not distribute) 7