Splunk

ITSI Sandbox Guidebook

START HERE .................................................................................................................................................................................................................................... 2
1 - Fly Over the Product .............................................................................................................................................................................................................. 3
2 - Prepare for the Journey: Core Concepts ....................................................................................................................................................................... 4
3 - Tour the Glass Tables ......................................................................................................................................................................................................... 13
4 - Troubleshooting Tour with Glass Tables and Deep Dives ................................................................................................................................. 16
5 - Dive in to Deep Dive ........................................................................................................................................................................................................... 25
7 - Tour Multi-KPI Alerts ......................................................................................................................................................................................................... 28
6 - Dive in to the Notable Events Review ......................................................................................................................................................................... 34
7 - Dive in to the Service Analyzer ...................................................................................................................................................................................... 36
8 - Side Trip to OS Host Details ............................................................................................................................................................................................ 37
7b - And Back to Service Analyzer ...................................................................................................................................................................................... 37



Document Revision History
Date Notes
2016Apr18 Added Multi-KPI Alerts, tweaked various chapters, edited for consistency ..dmillis
2016Apr08 Updated for 2.2 ..lsnow
2015Dec07 Completed "tour" chapters ..jlebaugh, dmillis
2015Dec03 Filled out first 4 chapters ..dmillis
2015Nov29 Initial version ..dmillis

1
START HERE
Welcome to the ITSI Sandbox Playbook! It is intended as a travel guide to help you explore the features, capabilities and
possibilities of IT Service Intelligence, using your new Splunk ITSI Online Sandbox. If you do not already have an ITSI Sandbox,
go to the ITSI Homepage (http://www.splunk.com/itsi) and click the green "Free Online Sandbox" button. It only takes a few
minutes!

The playbook contains a series of chapters, or exercises, to facilitate the exploration of ITSI and illustrate how it could be
useful in actual "real world" environments. The student should already have a basic understanding of core Splunk, especially
how to create searches and reports. This playbook should not be considered "real training"; please see Splunk Education
(http://www.splunk.com/view/education/SP-CAAAAH9) for in-depth courses on ITSI and other topics.

"Fly-Over" and "Tour" chapters show features and capabilities, in less detail and more detail, respectively. "Dive in" chapters
go into the most detail about how to set up and configure. Other chapters cover how to create new components, how to use
ITSI to troubleshoot problems quickly, and how to mock-up visualizations for your own high-value services.

Although the ITSI Sandbox is not set up to allow outside machine-data to be brought in, it does contain an event generator to
simulate the events which might be seen in a typical IT environment, including failure scenarios. It also contains a number of
pre-built KPIs, services, Glass Tables and other goodies to make the journey more interesting.

Generally, the chapters are laid out with the more basic concepts and exercises first, and more advanced topics later. Students
can skip chapters and jump around as they care to; each chapter lists the recommended pre-requisite chapters.

Ultimately, the purpose of this playbook is allow students to work with and understand the full capabilities of IT Service
Intelligence, and explore how ITSI could help solve actual, useful, high-value challenges in their own IT environments.

2
1 - Fly Over the Product
For the traveller who is in a hurry, who wants the 30,000-foot view, this is the section for you! It is also the best place to begin,
for the student who is largely unfamiliar with IT Service Intelligence.

Instructions
1. After logging into Splunk, click on "Product Tour"

2. Click through the slides to preview services, entities, KPIs, thresholding, Deep Dives, Multi-KPI Alerts, Notable Events
and the Service Analyzer
3. These topics, and more, are covered in more detail in the following chapters

3
2 - Prepare for the Journey: Core Concepts
Before we begin the journey, it is helpful to understand a few core concepts of IT Service Intelligence.

ITSI Core Concepts – Services

Technical Services Business Services
Requests
DNS
Responses Customer Requests
Transac6ons Responses
Requests
Auth Responses
Requests
Support Desk
Requests Responses
Web
Responses

Conceptually, a Service is a “black box” which we send requests and expect responses.
Includes technical (lower-level) and business (higher-level)

ITSI Core Concepts – Services

Customer Transac4ons
API Services

Partner Portal

Middleware
Mobile API/
Web Services

RDBMSs

Hypervisor and Hosts

DNS
Storage Tier
Packet Network

Services can encompass mulCple Cers of the IT domain.

Services may also depend upon other services

5
 ITSI Core Concepts – KPIs & Health Scores
Customer Requests
Requests
DNS Transac:ons Responses
Responses
KPI: Number of requests KPI: Number of transac9ons
KPI: Error rate KPI: Error rate
KPI: Average response 9me KPI: Average response 9me
KPI: Server CPU load KPI: Count of Incident Tickets
KPI: Server network I/F errors KPI: Synthe9c Transx Health
A Key Performance Indicator (KPI) is a Splunk saved search that produces a
metric like CPU %, Avg Response Time, Error Rate, etc. KPIs are contained
within Services. A Health Score is a score from 0-100 that helps determine
the health of a service. It is calculated based on importance and status
(e.g., green, orange, red) of all KPIs, once every minute
12

6
 ITSI Core Concepts – Service Analyzer

Service Analyzer is an auto-

generated, filterable, 8led view of
Services and KPIs. It is a launching
point for exploring Service and
En8ty Health in detail, as well as
crea8ng ad-hoc Deep Dives

13

7
 IT Service Intelligence – Core Concepts

A Glass Table is a customizable free form

drawing dashboards to view Health scores and
KPIs of choice with visual tools to create
context with live widgets

Go Deeper to a
Deep Dive View

27

8
 IT Service Intelligence – Core Concepts

Deep Dive – Swim lane analysis

dashboard to show KPI indicators
over :me for inves:ga:ons

28

9
 IT Service Intelligence – Core Concepts

Mul5 KPI Alerts – Visual tool to create

correla0on searches based on KPIs

29

10


11
ITSI represents a new way of dealing with IT Service challenges:

• Data-driven approach uses ALL IT Data - events, metrics, logs, structured,

unstructured, from-the-device, from-the-wire, etc.

• Service-awareness provides actionable insights into high-visibility services

• Customized contextual visualizations can be tailored for any person or group:

highly technical to business-oriented

• Mitigate problems before they impact customers

12
3 - Tour the Glass Tables
Glass Tables are a new type of dashboard, which allow ITSI services, KPIs and health scores to be visualized in highly
customizable ways. Glass Tables can be tailored to show very detailed technical views, or higher-level business views with
customer/revenue-relevant metrics. From the technical "soldiers in the trenches" to executive management, Glass Tables can
be crafted to show services, service relationships, transaction flows, health scores, key business metrics and other content
which are relevant to the users. And they're a lot of fun to build, too!

This section shows a number of example Glass Tables.
Instructions
1. Navigate to the Glass Table list by clicking on 'Glass Tables' in the top menu bar

2. From the list of Glass Tables, click on a Title to view that Glass Table

13
 3. Select Buttercup Games Business Process
This Glass Table shows the high-level business process status Buttercup Games. It could be used by service owners, executive
management or others who need to quickly understand the "big picture".

4. Select On Line Transaction Service
This Glass Table shows a detailed view of a customer-facing service, including transaction flow, component relationships and
dependencies, and critical health scores and metrics of key service points along the way. It makes excellent use of a pre-
existing drawing, with live ITSI "widgets" placed strategically on top. This Glass Table would helpful for NOC, Tier 1&2 and
similar support personnel who need to understand the complex relationships of all the service components supporting an
important business service.

5. Select Buttercup Games Online Store
This Glass Table shows a streamlined view of Buttercup Games' customer-facing service-- the "online store" summarized in
the "Buttercup Games Business Process" Glass Table. This view provides more detail of the underlying technical services, their
dependencies, and the overall transaction flow. It uses native Glass Table drawing tools, as well as service and KPI widgets,
which display health and metric values live (updating over time). These widgets have configurable drill-down capabilities,
including the ability to navigate to other, even-more-detailed Glass Tables. For example, if you click on the widget next to Web
Tier, you will navigate to ...

6. Web Tier
This Glass Table represents a more detailed visualization of the KPIs, overall Web Tier health score, and the health score of its
dependent service, Middleware. Such Glass Tables allow technical personnel to quickly troubleshoot problems by being able
to drill down to the detailed technical metrics which matter.

7. Select Buttercup Games Online Store (again)
Several drill-down options are available when a widget is clicked. Click on the widget next to Database; this will navigate to a
Deep Dive.

14

Glass Tables allow services, dependencies, health scores, KPIs and other critical
information to be visualized in a contextual way that is truly meaningful to the
targeted audience. This allows users to quickly size-up service delivery health and
when necessary, efficiently isolate problems.

15
4 - Troubleshooting Tour with Glass Tables and Deep Dives
This section describes a possible problem scenario, and how ITSI could be used to efficiently troubleshoot to find root cause.
This would typically be driven by a NOC or Tier 1 or Tier 2 support person.

We're going to "set up" the failure scenario and first see how Glass Tables can accelerate the troubleshooting process, then
continue isolating root cause with Deep Dives.
Pre-Requisites
You should already be familiar with:
• Core Concepts (Ch. 2)
• Glass Tables (Ch. 3)

About the event generator...

In order to make the ITSI Sandbox more interesting to play in, an event generator is included which continuously generates
a simulated stream of realistic machine events, including web access, database, Linux metrics (from the *nix Technology
Add-on) and others. Included in this stream of events are two failure scenarios, showing a sequence of failures and
resulting service degradations, each scenario repeating hourly. Typically, the initial failures for each scenario occur at the
top of the hour, and reset back to "OK" around the top of the next hour. However, the event generator (eventgen) timing
may not be precise. The failure scenarios may occur at slightly different times from hour to hour, and may vary from
sandbox to sandbox. Thus, within the Sandbox, it is impossible to predict exactly how the health scores and KPIs will
appear, during any specific hour. This makes it difficult to setup a "clean" failure simulation. Please pardon any eventgen
inconsistencies. We decided to put most of our effort into developing ITSI-- not an event generator.

16
Instructions
1. Navigate to the Glass Table called, Buttercup Games Online Store:
a. Click on Glass Tables in the upper menu bar to navigate to the page, Saved Glass Tables
b. Click on Buttercup Games Online Store to navigate to this Glass Table
2. Modify the view time by clicking on the time picker in the upper right corner. In the pop-up window, type in an explicit
time from the past, such as XX:15.0 from the previous hour (or the hour before that, etc). Be sure to use the correct
HH:MM:SS.sss format (example: "10:15:00.0")

17

3. In a few seconds, the colors of the widgets will change, to indicate their states at that particular time in the past. As noted
earlier, the two different failure scenarios toggle each hour. Try different even & odd hours in the past to see this.
4. For the purposes of this troubleshooting exercise, imagine that your Glass Table looks like the following:

18
5. The scenario: Customer Care has informed us that customers are calling to complain when they try to purchase through
the Online Store; they are seeing slow response and occasional errors. The problems seem to be affecting both web-based
and mobile-based customers.
6. Based on just the reports that the customer-facing web-based service is having problems, most support persons would
begin troubleshooting "from the top"-- the web and mobile tiers in this case. If no obvious problems were found, they
would proceed down the service dependency tree-- to the middleware tier, etc.
7. But using a Glass Table such as "Buttercup Games Online Store" provides instant and context-relevant visibility into service
health scores and important KPIs, all in one place. In the above example, which supporting tier seems to be in distress?
(Database) By being able to visualize the relevant services and their health scores, we have the ability to immediately
focus our troubleshooting on the areas that are degraded. This can save huge amounts of time and greatly reduce the time
required to find root cause.
8. On your Sandbox Glass Table, click on the widget beneath Database to drill down into the Database tier to continue the
troubleshooting exercise. (Select Leave This Page if prompted)

19
(Now in DB Deep Dive)

9. Click on the > next to Focus to collapse the service tree navigator panel on the right side; we will explore this feature later.
10. Change the Primary Time Range to Last 2 Hours by clicking on the time picker in the lower left corner:
a. In the Relative section, type in "2" and select Hours Ago
b. Click Apply

20
(The Deep Dive should now displays some "mostly green" and some "mostly red" – your screen may not look exactly like the
below, but you should see a point where things go from green to red and red to green)

11. We are looking at the Aggregated Health Score (top swimlane) and KPIs for the DB Service, across a time range which
shows the service moving from "healthy" to "not healthy".
12. Slowly mouse over the swim lanes to compare values at various points in time.
13. Click the checkbox in the upper left to select all swimlanes, and use the “Bulk Actions” menu to “Show State Thresholds” or
"Hide Thresholds", toggling to compare the swim lanes with and without the threshold colors/states overlaid.
14. Note that the ServiceHealthScore in the top swimlane is an aggregation of the service's KPIs and dependent services,
ranging from 100-0. When did the health score begin to deteriorate, and which KPI(s) may have been part of the root
cause?

21
15. Click on the name-box for Storage Free Space: % System, then drag it upwards to reposition this swimlane.
16. A few of the swimlanes are continuously green, indicating that they are not particularly helpful in our troubleshooting
exercise ("CPU Utilization", "Memory Free", etc). Click on the checkbox in the upper left corner to unselect all swimlanes,
then select the checkbox for CPU Utilization: % User and Memory Free. Select Bulk Actions -> Delete to (temporarily)
remove this swimlane from our Deep Dive.

17. Click on the darker blue tile within the DB Service Errors swimlane to reveal "raw errors" from the underlying Splunk
search. Click on Hide Events to dismiss.

22
18. Mouse over the Storage Free Space: % System swimlane, in the place where it goes from green to red. Note the high &
low metric values shown for the swimlane, and that this metric has gone to 0%, indicating that a filesystem is full.
19. Click anywhere within the Storage Free Space: % System swimlane to reveal an options popup. Select Add Overlay as
Lane.

(Three new swimlanes are added at the bottom, representing the separate KPI values for the individual entities (hosts) which
comprise this KPI)

20. Which host/server is suffering from a filesystem-full condition? (mysql-02)

23
Overall service health can be effectively and efficiently visualized in Glass Tables,
allowing support personnel to quickly find likely hot spots. They can drill down to more
detailed layouts and ultimately compare and correlate KPI & Service trends in parallel
swimlanes within Deep Dive. Faster root cause analysis (RCA) leads to substantial
reductions in Mean Time To Repair (MTTR).

24
5 - Dive in to Deep Dive
Deep Dives allow KPI metrics and health scores to be compared in side-by-side swimlanes, which allows trends and
correlations to be more easily and quickly discovered.

This chapter explores Deep Dives and how they can be used.

Pre-Requisites
You should already be familiar with:
• Core Concepts (Ch. 2)
• Troubleshooting (Ch. 4) also goes into Deep Dives

Instructions
1. Navigate to the Deep Dive called, DB Deep Dive:
a. Click on Deep Dives in the upper menu bar to navigate to the page, Saved Deep Dives
b. Click on DB Deep Dive to navigate to this Deep Dive
2. Click on the > next to Focus to collapse the service tree navigator panel on the right side; we will explore this feature later.
3. Select an arbitrary time range by clicking on the Primary Time Range menu option at the bottom right; it functions like a
standard Splunk search bar time picker
4. Zoom in to a tighter time range in the current view by click-holding anywhere in the swimlanes, then dragging horizontally
to select the range.
5. Toggle the threshold health score colors by clicking on the checkbox in the upper left corner to select all swimlanes, then
Bulk Actions -> Show State Thresholds/Show Level Thresholds/Hide Thresholds.
6. Click on the > next to Focus to open the service tree navigator panel on the right side.
a. Click on a service node to navigate up and down the dependency tree of services
b. After clicking on a service node, note that those service's KPIs are listed below.
c. Click on the + on a listed KPI to add it to the current swimlanes
d. Click on the > next to Focus to collapse the service tree navigator panel on the right side
7. Mouse-over the name-box for any swimlane to reveal the "options wheel", then select it to view available options:

25


8. The student is encouraged to explore these options, which are covered in more detail at
http://docs.splunk.com/Documentation/ITSI/latest/User/DeepDives
9. Click-hold on the name-box for any swimlane, and then drag it vertically to reposition this swimlane.
10. Click on the darker blue tile within the DB Errors (or any "event"-style) swimlane to reveal "raw errors" from the
underlying Splunk search. Click on Hide Events to dismiss.
11. To save a Deep Dive after modifying the layout and/or visualization options, click on the Edit menu option in the upper
right corner, then select Save
12. To compare the current time range against a different time range, click on Compare to ... in the lower left corner, then
select a comparison time range. This causes each KPI to display twin swimlanes: primary time range above comparison
time range. Note that when mousing over the swimlanes, the time display at the top now shows both times.
13. To dismiss the "twin" lanes display, deselect the checkbox next to Compare to ... in the lower left corner

26
Deep Dive allows any KPIs and Services to be compared and correlated in a side-by-
side fashion, across multiple time ranges, using a variety of visualizations. It is
intended to greatly enhance and streamline the troubleshooting process for finding
root cause, significantly decreasing Mean Time To Repair (MTTR).

27

7 - Tour Multi-KPI Alerts
Multi-KPI Alerts are Correlation Searches which can combine any KPIs to create meaningful, actionable alerts, using
multiple correlation factors such KPI threshold indications, length of time in this state, time-of-day, and others. Multi-KPI
alerts can find not just "failures", but early "canary in the coal mine" indications that the service is becoming unstable; it is
possible to find problems BEFORE they impact customer-facing services. When a Multi-KPI Alert fires, it creates a Notable
Event; it could also execute a script and/or send email.
Pre-Requisites
You should already be familiar with:
• Core Concepts (Ch. 2)
• Troubleshoot with Glass Tables and Deep Dives (Ch. 4)

Instructions
1. Navigate to the Deep Dive called, DB Deep Dive:
a. Click on Deep Dives in the upper menu bar to navigate to the page, Saved Deep Dives
b. Click on DB Deep Dive to navigate to this Deep Dive
2. Click on the > next to Focus to collapse the service tree navigator panel on the right side.
3. Change the Primary Time Range to Last 2 Hours by clicking on the time picker in the lower left corner:
a. In the Relative section, type in "2" and select Hours Ago
b. Click on Apply

28


(The Deep Dive should now displays some "mostly green" and some "mostly red" – your screen may not look exactly like the
below, but you should see a point where things go from green to red and red to green)

29


4. We are looking at the Aggregated Health Score (top swimlane) and KPIs for the DB Service, across a range of time which
shows the service moving from "healthy" to "not healthy".
5. Click/drag across a narrower range of time when the service transitions from green to yellow/orange.
6. Click on the checkbox in the upper left to unselect all swimlanes, then select the checkboxes next to the KPI swimlanes
which were involved in this outage (turned red) during this period, such as Storage Free Space, DB Service Queries & DB
Service Response Time.
7. In the upper left, select Bulk Actions -> Create Multi KPI Alert

30


(This will open the Multi KPI Alert configuration workflow page)

31


ITSI provides a sophisticated array of options for setting up Multi-KPI Alerts, also known as Correlation Searches. The goal
is to allow the creation of useful alerts based on correlations of several KPIs-- fewer "noise" alerts, more actionable alerts.
Here are some of the features and capabilities:
• Control the range of time to correlate the KPIs across (time-picker in the upper right corner)
• Add KPIs from any service
• Create a KPI based on the aggregate health score of the KPIs, or on Status over time (upper right corner)
• Re-weight the KPIs using the Importance sliders (lower right corner)

32
 • Control alert actions, suppression, and other details (later in the creation workflow, after hitting 'Save' in the lower
right corner)
One of the most important things which happens when a Multi-KPI Alert fires, is the creation of a Notable Event. Notable
Events are explored in a later chapter.

Existing sample Multi-KPI alerts can be examined by clicking on Configure -> Correlation Searches, then selecting a
correlation search from the list.

More details are available here: http://docs.splunk.com/Documentation/ITSI/latest/User/CreateMulti-KPIAlerts






Multi-KPI Alerts can combine any KPIs to create useful, actionable alerts (less alert
"noise"). "Canary in the coal mine" problems can be detected early, potentially before
they affect customers, revenue or SLAs.

33
6 - Dive in to the Notable Events Review
When a Multi-KPI Alert fires, it creates a Notable Event. The Notable Events Review is Splunk's next-generation event
management console. Notable Events Review provides a quick way view, sift and organize events, allowing us to triage,
manage and streamline workflow more effectively.

It has the ability filter Notable Events and events from other event management sources, based on various criteria, such as
Severity, Status, Service and others. It also allows events to be modified, to change Owner, Severity, Status, and/or add
comments. Events can also have workflow actions associated with them, to allow an operator the ability to quickly hit
troubleshooting options, execute mitigation scripts, or open a "real" Incident Management trouble-ticket.

Pre-Requisites
You should already be familiar with:
• Core Concepts (Ch. 2)
• Tour Multi-KPI Alerts (Ch. 7)

Instructions
1. Navigate to the Notable Events Review by clicking on Notable Events Review in the upper menu bar
2. Click on Show Timeline to reveal the timeline
3. See details for an event: Click on any event to open the Details panel on the right.
Details include which KPIs contributed, and which services might be affected, as well as the ability to examine these in
more detail in a Deep Dive. Severity, Status and Assignment can also be changed directly.
4. Modify Severity for an event: Click on the Severity dropdown at upper left of the Details panel, choose a different Severity
5. Choose a workflow action: Click on </> icon in upper right corner of Details panel to reveal the workflow options
Custom workflow actions can be created for each type of Notable Event, to streamline workflow actions. These can be
additional troubleshooting or mitigation scripts, or something as basic as opening a 'real' incident ticket.
6. Dismiss Details: Click on the X in the upper right corner of Details panel to dismiss
7. Filter the Notable Events by Severity: Click on the "gear wheel" in the upper right corner, then choose Edit Filter Settings.
Click Add Filter, and then Severity. Click in the Severity box to see and choose from a list of the available severity levels.

34
8. Filter by Status, Owner, Service, Time Range, Name ("Title") or freeform search criteria by adding other filters to your filter
settings.
9. Click Done to (re)apply search filter criteria
10. Change view options: Click on the "gear wheel" in the upper right corner, then choose Edit View Settings. Select Viewing
Option -> Prominent and Deduplication -> On, then Done
An Event Count column has now been added for deduplicated events, and Severity color is now more 'prominent'
11. Add, remove or re-order columns: Click on the "gear wheel" in the upper right corner, then choose Edit View Settings. In
Columns Shown, click X to remove a column, click + Add Column to add a column, or click/drag a column to re-order how
it is viewed.
12. To sort the event rows: Click on the V chevron next to Sort By: (left side, above rows), then select a column to sort by.
Toggle the sort order (ascending/descending) by clicking on the vertical arrow next to Sort By:
More details are available here: http://docs.splunk.com/Documentation/ITSI/latest/User/NotableEventsReview







The Notable Events Review allows an operator to:
• Quickly and effectively find, deduplicate and manage just the events they want
• Tie workflow actions to events, to streamline operations
• Manage ITSI Notable Events and events from other sources

35
7 - Dive in to the Service Analyzer
The Service Analyzer is a "Big Picture" view of all services, and the "most interesting" KPIs (i.e., KPIs with degraded health
scores). It is "no frills", designed for NOCs, Tier 1 or 2 support, and others who need a high level view of all services/KPIs, or a
subset. It also provides a launching point for exploring Services, KPIs and Entities in more detail.

Pre-Requisites
You should already be familiar with:
• Core Concepts (Ch. 2)

Instructions
1. Navigate to the Service Analyzer by clicking on Service Analyzer in the upper menu bar, then choosing Default Service
Analyzer.
2. Click on Middleware Service to navigate to its service health page. Here you can see the service tree on the left, the KPIs
in the center, and the entities associated with a selected KPI on the right.
3. Click on DB Service in the left service tree panel to navigate to this service
4. From Service Health, you can also navigate to a deep dive containing the KPIs for that service using the link at the top of
the KPI table in the center of the page.
a. Notice the deep dive has been built for you on the fly, containing all the KPI’s associated with that service
5. Click on Storage Free Space: % System and notice that you now have a table on the right that shows the entities
associated with this KPI.
6. Click on mysql-02 in the entity list to navigate to its Entity Health page.
a. This is an entity-centric view, showing information about a specific entity, including which services and KPIs it
supports.
b. Clicking on a service name will navigate to that service health page

36
8 - Side Trip to OS Host Details
7. If you are using one or more ITSI modules, relevant module dashboards for this entity will show up in the left-side Modules
panel. In this case, "OS Host Details" is listed. More details about modules are available here:
http://docs.splunk.com/Documentation/ITSI/latest/IModules/AboutITSIModules
8. Click on OS Host Details to navigate to this page.
a. The OS Host Details section offers several dashboards with detailed status, performance and event reports.
b. OS Host Details can also be accessed in Deep Dive.
c. More details about the Operating System (OS) Module are available here:
http://docs.splunk.com/Documentation/ITSI/latest/IModules/AbouttheOperatingSystemModule

7b - And Back to Service Analyzer

9. Navigate back to Service Analyzer
10. Click in the Select service(s) to monitor box to select & show only certain services
11. Click on the "Option Wheel" next to Top ... Services to control how many services are shown
12. Click on the "Option Wheel" next to Top ... KPIs to control how many KPIs are shown, and to select which KPIs are shown
13. To create an ad-hoc Deep Dive:
a. Mouse over one or more Service or KPI tiles, then select the checkbox in the upper right corner of the tile
b. Click Drilldown to Deep Dive


Service Analyzer provides a "Big Picture" view of all services and the "most
interesting" (not green) KPIs. It is also a launching point for exploring Services, KPIs
and Entities in more detail, as well as for creating ad-hoc Deep Dives with selected
KPIs.

37