Splunk

Development
Day 1
- Vikram Yadav (VY)

Your attendance & progress of the course will be calculated based on

the completion of the assignments and quiz.
Agenda – Day 1
• Machine Data
• Why we need a Splunk?
• Introduction to Splunk
• What is Splunk?
• Benefits of using Splunk?
• Features & Use cases
• Architecture and components
• Lab
 About me !
• 3+ years of Splunk experience

• Splunk Developer & Administrator at Avotrix

• Enterprise Security & ITSI

• IOT & AWS

• Leader of Splunk Mumbai User Group

• Watching movies, learning new technology

- thevikramyadav
Data Everywhere

• What is Machine data?

• How big it is?

Why do we need Splunk ?
Introduction to Splunk
• What is Splunk?
• Splunk is a software platform to search, analyze and visualize the machine-generated
data gathered from the websites, applications, sensors, devices etc. which make up
your IT infrastructure and business.

• Big Data Tool, Log Analytics Platform

• Machine-generated data is information automatically generated by a computer

process, application, or other mechanism without the active intervention of a human
• Complex to understand
• In an unstructured format
• Not suitable for making analysis / visualization
Got Data?
Visuals - Reports and Dashborads
Introduction to Splunk
Splunk in Various Area

Network QoS & KPI Analytics

Android Logs Analytics
Operation Intelligence & Visualization
Mobile Device Product Performance Visibility
Customer Loyalty & Churn Analysis
Field Test Data Analytics
Network Planning & Optimization
QXDM Logs Analysis
Fault Prediction & Alerts

Media &
Network
Entertainment
Infrastructure

Performance Monitoring
Operational Health Visualization STB & CMTS Process Logs Analytics
Security Intelligence Content & Search Analytics
Storage &
Alert Monitoring Customer Usage Pattern Insights
Computing
Resource Utilization Analytics
 Benefits of Using Splunk
• Your input data can be in any format for e.g. .csv, or json or other human readable
formats
• You can configure Splunk to give Alerts / Events notification at the onset of a machine
state
• You can accurately predict the resources needed for scaling up the infrastructure
• You can create knowledge objects for Operational Intelligence
• No need of Data Mapping or any structure specific conversion
• Works on any Machine Generated Data
• One Tool for Data Uploading, Data Searching & Visualization
• Supports Machine Learning, Prediction & Forecasting
• On Premise & Cloud Solution
Splunk Features
Analyze system
Analyze system
performance
performance
Store and retrieve data for Troubleshoot any failure
Store and retrieve
use data for Troubleshoot
later conditionany failure
later use condition

SPLUNK
SPLUNK
Create dashboard to
Create dashboard to Monitor business metrics
visualize & analyze results Monitor business metrics
visualize & analyze results

Search and investigate

Search andoutcome
particular investigate
particular outcome
 Use Cases 1: Hospital Service
• Reporting health conditions in real time

• Delve deeper into the patient’s health record and

analyse patterns

• Alarms / Alerts to both the doctor and patient when

the patient’s health degrades
Use Case 2 : Data Electrified - Porsche
Use Case 3: Dominos Story
• Lot of unstructured data was generated because:
• They had an omni-channel presence for driving sales
• They had a huge customer base
• They had several touch points for customer service
• They provided multiple systems for delivery: Order food in-store, order via
telephone, via  their website and through cross-platform mobile applications
• They upgraded their mobile apps with a new tool to support ‘voice ordering’ and
enable tracking of their orders

• The excess data generated gave rise to the following problems:

• Manual searches being tedious and error prone
• Less visibility into how customer need/preference varies
• Unpreparedness and thus working in reactive mode to fix any problem
Dominos Story
Business Scenario

• Domino’s had no clear visibility into which offer works best – in

terms of:
• Offer type (Whether their customers preferred a 10% discount or a
flat $2 discount?)
• Cultural differences at a regional level (Do cultural differences play a
role in offer choice?)
• Device used for buying products (Do devices used for ordering play a
role in offer choices?)
• Time of Purchase (What is the best time for the order to be live?)
• Order revenue (Will offer response change wrt to order revenue size?)
Introduction to Splunk
Indexers

Search head
Offline orders

Mobile orders

Forwarder
Different Stages In Data Pipeline
Data Processing Stages

• Different Stages In Data Pipeline with respective Splunk

Components
• There are 3 main components in Splunk:
• Splunk Forwarder
• Splunk Indexer
• Search Head

DATA INPUT PARSING INDEXING SEARCHING

Forwarder Indexer Search Head

 Architecture and components

• Universal Forwarder

• Heavy Forwarder
 Heavy Forwarder

Indexer
Heavy Forwarder

DATA Parsing Indexing Routing

Architecture and components

• Splunk Indexer
Architecture and components

• Splunk Indexer ( Event Processing)

• Separating the data stream into individual, searchable events
• Creating or identifying timestamps
• Extracting fields such as host, source, and sourcetype
• Performing user-defined actions on the incoming data
• identifying custom fields
• masking sensitive data
• writing new or modified keys
• applying breaking rules for multi-line events
• filtering unwanted events
• routing events to specified indexes or servers
Architecture and components

• Splunk Search Head

 Architecture and components

Universal Forwarder Heavy Forwarder Indexer Search Head

Architecture and components
Example of Splunk Deployment
 Splunk Architecture - simple
Distributed Search and
Summary Indexing Tier

Indexing Tier
License Capacity: 500 MB/day … x5

Forwarders or
Forwarding Tier

Data Sources

desktops laptops servers/VMs proxy applications syslog firewall config

28
Architecture and components
Key components of Splunk

One Splunk
Many Uses
Organizations Using Splunk
Products of splunk

Core Products
• Splunk Enterprise
• Splunk Cloud
• Splunk Light

Premium Application
• Splunk Enterprise Security (ES)
• Splunk IT Service Intelligence(ITSI)
• Splunk User Behavior Analytics(UBA)
 LAB
• Download Splunk Package
• Install Splunk in Windows/MAC Env
• Operations: Start, Stop and Restart

Please submit an email with a screenshot of the assignment before the commencement of the next class, Also in order to
move forward with the course, it is important to complete this assignment. If you are unable to complete the assignment or
need any assistance regarding the same, please email me at vikram@avotrix.com
Question & Answer.....