This guide is not intended to be a tutorial on how

to install/prepare Splunk. This guide is intended

mainly to teach you how to use Splunk to parse
and extract data in different cyber attacks
scenarios. Additionally it's now being expanded to
include notes for data analytics purposes as well.

The variables, names, data sources, source-types

and other identifiers used in this guide may need
to be changed if you want to apply them in your
environment.

Definitions
Host
It's the name of the physical or virtual device
where an event originates. It can be used to find all
data originating from a specific device.

Source
It's the name of the file, directory, data stream, or
other input from which a particular event
originates.
Sources are classified into source types, which
can be either well known formats or formats
defined by the user. Some common source types
are HTTP web server logs and Windows event
logs.

Tags
A tag is a knowledge object that enables you
to search for events that contain particular field
values. You can assign one or more tags to any
field/value combination, including event types,
hosts, sources, and source types. Use tags to
group related field values together, or to track
abstract field values such as IP addresses or ID
numbers by giving them more descriptive names.

Indexes
When data is added, Splunk software parses
the data into individual events, extracts the
timestamp, applies line-breaking rules, and
stores the events in an index. You can create new
indexes for different inputs. By default, data is
stored in the “main” index. Events are retrieved
from one or more indexes during a search.
Splunk Forwarder
A Splunk instance that forwards data to another
Splunk instance is referred to as a forwarder.

Splunk Indexer
An indexer is the Splunk instance that indexes
data.
The indexer transforms the raw data into events
and stores the events into an index. The indexer
also searches the indexed data in response to
search requests. The search peers are indexers
that
fulfill search requests from the search head.

Splunk Search Head

In a distributed search environment, the search
head is the Splunk instance that directs search
requests to a set of search peers and merges
the results back to the user. If the instance
does only search and not indexing, it is usually
referred to as a dedicated search head.
Search Processing Language
It's the language used to perform search
operations in Splunk. SPL or Splunk processing
language consists of keywords, quoted phrases,
Boolean expressions, wildcards,parameter/value
pairs, and comparison expressions. Unless you’re
joining two explicit Boolean expressions, omit
the AND operator because Splunk assumes the
space between any two search terms to be AND

Basic Search Commands

eval
Calculates an expression
*
search through all available logs
rare
Used to find uncommon events
top
Top most common events
timechart
Track occurrences of events over time
stats
Gather general statistical information about a
search
head
Include only the first few values found within my
search
tail
Include only the last few values found within my
search
transaction
The transaction command gives us an easy way to
group events together based on one or more fields
and returns a field called duration that calculates
the difference between the first and last event in a
transaction.
dedup
Removes duplicates from a search result.
head/tail N
Returns the first/last N results.
sort
Sorts the search results by the specified fields.
stats
Provides statistics, grouped optionally by fields.
See COMMON STATS FUNCTIONS.
table
Specifies fields to keep in the result set. Retains
data in tabular format.
rex
Extract fields according to specified regular
expression(s)

Log Monitoring
Continuous Log Monitoring and From
A Log File
We can setup Splunk to import logs from a specific
log file and then set it up to continuously import
the logs from the specified log file. An example
would be constantly monitoring and importing web
server logs from to Splunk from the web server
logs.

1. Log in to your Splunk server.

2. From the home launcher in the top-right
corner, click on the Add Data button.
3. In the Choose a Data Type list, click on A file or
directory of files.
4. Click on Next in the Consume any file on this
Splunk server option.
5. Select Preview data before indexing and enter
the path to the logfile (/var/log/apache2/) and
click on Continue.
6. Select Start a new source type and click on
Continue.
 7. Assuming that you are using the provided file
or the native /var/log/messages file, the data
preview will show the correct line breaking of
events and timestamp recognition. Click on the
Continue button.
8. A Review settings box will pop up. Enter
apache_logs as the source type and then,
click on Save source type.
9. A Sourcetype saved box will appear. Select
Create input.
10. In the Source section, select Continuously
index data from a file or directory this Splunk
instance can access and fill in the path to your
data. If you are just looking to do a one-time
upload of a file, you can select Upload and
Index a file instead. This can be useful to index
a set of data that you would like to put into
Splunk, either to backfill some missing or
incomplete data or just to take advantage of
its searching and reporting tools.
11. Ignore the other settings for now and simply
click on Save. Then, on the next screen, click
on Start searching. In the search bar, enter the
following search over a time range of All time
`sourcetype=apache_logs
Continuous Log Monitoring Through
Network Ports
Sending data to Splunk over network ports doesn't
need to be limited to network devices. Applications
and scripts can use socket communication to the
network ports that Splunk is listening on. This can
be a very useful tool in your back pocket, as there
can be scenarios where you need to get data into
Splunk but don't necessarily have the ability to
write to a file.

1. Log in to your Splunk server.

2. From the home launcher in the top-right
corner, click on the Add Data button.
3. In the Or Choose a Data Source list, click on
the From a UDP port link or TCP port link.
4. Specify the port that you want Splunk to listen
on.
5. In the Source type section, select From list
from the Set sourcetype drop-down list, and
then, select syslog from the Select source type
from list drop-down list.
6. Click on Save, and on the next screen, click on
Start searching. Splunk is now configured to
listen on the port you assigned in step 4. Any
 data sent to this port now will be assigned the
syslog source type. To search for the syslog
source type, you can run the following search
sourcetype=syslog

Continuous Log Monitoring Through

Splunk Universal Forwarder
Splunk Universal Forwarder (UF) can be installed
on your remote endpoint
servers and used to forward data back to Splunk to
be indexed. The Universal Forwarder is similar to
the Splunk server in that it has many of the same
features, but it does not contain Splunk web and
doesn't come bundled with the Python executable
and libraries. Additionally, the Universal Forwarder
cannot process data in advance, such as
performing line breaking and timestamp extraction.

To obtain the Universal Forwarder software, you

will need to go to www.
splunk.com/download and register for an account if
you do not already have one. Then, either
download the software directly to your server or
download it to your laptop or workstation and
upload it to your server via a file-transfer process
such as SFTP.
1- On the server with the Universal Forwarder
installed, open a command prompt if you are a
Windows user or a terminal window if you are a
Unix user.

2- Change to the [$SPLUNK_HOME/bin] directory,

where [$SPLUNK_HOME] is the
directory in which the Splunk forwarder was
installed.. For Unix, the default installation directory
will be /opt/splunkforwarder/bin and for
Windows, it will be C:\Program
Files\SplunkUniversalForwarder\bin.

3- Start the Splunk forwarder if not already

started, using the following command: ./splunk
start

4- Accept the license agreement.

5- Enable the Universal Forwarder to autostart,

using the following command: ./splunk enable
boot-start

6- Set the indexer that this Universal Forwarder

will send its data to. Replace the host value with
the value of the indexer as well as the username
and password for the Universal Forwarder
./splunk add forward-server <host>:port -
auth <username>:<password>. The username and
password to log in to the forwarder (default is
admin:changeme) is username:password.
Additional receiving indexers can be added in the
same way by repeating the command in the
previous step with a different indexer host or IP.
Splunk will automatically load balance the
forwarded data if more than one receiving indexer
is specified in this manner. Port 9997 is the default
Splunk TCP port and should only be changed if it
cannot be used for some reason.

7- Log in to your receiving Splunk indexer server.

From the home launcher, in the top-right corner
click on the Settings menu item and then select the
Forwarding and receiving link

8- Click on the Configure receiving link.

9- Click on New.

10 - Enter the port you chose in step 6 in the

Listen on this port field.

11- Click on Save and restart Splunk. The Universal

Forwarder is installed and configured to send data
to your Splunk server, and the Splunk server is
configured to receive data on the default Splunk
TCP port the port you chose in step 6.

Operational Notes
Its vastly important to determine the source of
your data to be able to correctly pull the
logs/events. Specify this source as
index=name-of-the-datasource. If you don't
know you can then retrieve all data from all
sources index=* but this will retrieve huge
amounts of events that you don't necessarily
need to parse.
When we are dealing with Windows hosts and
looking at processes starting, we can view
both Windows event logs and Sysmon for
more information
Remember that when searching for
backslashes to add a second backslash,
because a single backslash is an escape
character and will cause your search to not run
correctly.
When we analyze a ransomware attack, one of
the questions we should ask is what
connections are the impacted systems making
to other systems? File shares can be leveraged
to inflict greater damage by encrypting shared
data thus increasing the impact of the
ransomware beyond the individual host.
Registry data from Windows systems as well
as Microsoft Sysmon will provide insight into
file shares
Oftentimes we will need to determine which
destinations an infected system
communicated with. The first time and last
time it occurred are important pieces of
information in your investigation as well
In a host-centric environment, we use
hostnames more frequently than IP addresses.
As we start looking at our events from a
network-centric approach, we need to be
aware that we may need to search by IP as
well
IDS/IPS or Malware Signatures may already
exist for threats that we need to deal with.
Understanding what signatures fired is
important to understand when the threat was
seen, where in the network it was seen, when
it was seen, what technology identified the
signature and the nature of the threat.
Stream is a free app for Splunk that collects
wire data and can focus on a number of
 different protocols, including but not limited to
smtp, tcp, ip, http and more

Using Splunk For Incident

Response
Parsing Suricate IDS Logs
When parsing Suricate IDS events, we mostly aim
to extract data from a specific malware/Trojan that
triggered an alert hence we are interested in
displaying the IDS alerts. A key field you want to
focus on is alert.signature
Scenario One
Say you want to see all IDS [suricata] alerts
triggered by the machine infected with Advanced
Persistent Threat. Infected machine ip is
[40.80.148.42]

index=dataset domain.com
src=40.80.148.42 sourcetype=suricata

Looking for executable files caught in Suricata IDS

logs
 index=* sourcetype=suricata .exe

Looking through endpoints to which executable

files were downloaded or uploaded. Make sure to
change ip values

index=botsv1 sourcetype=suricata
(dest="ip1" OR dest_ip="ip2") .exe

Looking through endpoints to which executable

files were uploaded through http POST request.
Make sure to change ip values by specifying the
endpoints using IP and domain

index=botsv1 sourcetype=suricata
(dest=domain.com OR dest="ip1")
http.http_method=POST .exe

#TIP
Using the stats command, we can get a count for
each combination of signature and signature_id.
Because the signature and the id are likely unique,
that may not be necessary in this case, but it
provides a good way to see the description and the
associated ID of the signature accompanied by the
count.
Example

index=dataset sourcetype=suricata
alert.signature=*keyword* | stats
count by alert.signature
alert.signature_id | sort count

Scenario 2
This scenarios focuses on the point we mentioned
earlier about extracting IDS alerts.
We can display all IDS alerts

index=dataset sourcetype=suricate |
stats values(alert.signature)

After we display alerts, we narrow down our efforts

on a specific alert that was triggered by a malware
infection.
 index=dataset sourcetype=suricate
"alert.signature"="ET MALWARE
Win32/Trickbot Data Exfiltration"

And then we can extract relevant details and IOCs

about the malware such as the source and
destination IPs

index=dataset sourcetype=suricate
"alert.signature"="ET MALWARE
Win32/Trickbot Data Exfiltration" |
table src, dest

Parsing http traffic

Looking through http traffic with domain name as a
keyword

index=dataset domain.com
sourcetype=stream:http

Pasrsing http traffic for specific source ip

 index=botsv1 src=192.168.250.70
sourcetype=stream:http

Investigating http protocol with http responses

being equal to [200] and count the number of hits
per URL

index=* dest=192.168.250.70
sourcetype=stream:http status=200 |
stats count by uri | sort - count

Investigating IIS logs

index=* sourcetype=iis

Investigating HTTP post requests and displaying

authentication information

- index=* sourcetype=stream:http
http_method=POST
form_data=*username*passwd*
| table form_data
Using regular expressions to display usernames
and passwords in http requests

index=* sourcetype=stream:http
form_data=*username*passwd*
| rex field=form_data "passwd=(?
<userpassword>\w+)"
| head 10
| table userpassword

Using regular expressions to display usernames

and passwords in http requests and display the
count of the password length

index=* sourcetype=stream:http
form_data=*username*passwd*
| rex field=form_data "passwd=(?
<userpassword>\w+)"
| eval lenpword=len(userpassword)
| table userpassword lenpword

Using lowercase and uppercase on the password

 index=* sourcetype=stream:http
form_data=*username*passwd*
| rex field=form_data "passwd=(?
<userpassword>\w+)"
| eval lenpword=len(userpassword)
| search lenpword=6
| eval password=lower(userpassword)

Sorting by time

index=* sourcetype=stream:http
form_data=*username*passwd*
dest_ip=192.168.250.70
src=40.80.148.42
| rex field=form_data "passwd=(?
<userpassword>\w+)"
| search userpassword=*
| table _time uri userpassword

Searching for specific password

[1]
 index=* sourcetype=stream:http | rex
field=form_data "passwd=(?
<userpassword>\w+)" | search
userpassword=batman | table _time
userpassword src

[2]

index=* sourcetype=stream:http
| rex field=form_data "passwd=(?
<userpassword>\w+)"
| search userpassword=batman
| transaction userpassword
| table duration

Looking for executable files transferred through

[http]
[1]

index=* sourcetype=stream:http .exe

[2]
 index=* sourcetype=stream:http
uri=*.exe | stats values(uri)

Looking through get requests for a specific domain

index="dataset" domain.com
sourcetype="stream:http"
http_method=GET

Looking through visited websites from a source ip

with an exclusion list.

index="dataset" ip
sourcetype="stream:HTTP" NOT
(site=*.microsoft.com OR
site=*.bing.com OR site=*.windows.com
OR site=*.atwola.com OR site=*.msn.com
OR site=*.adnxs.com OR
site=*.symcd.com OR
site=*.folha.com.br)
| dedup site
| table site
Displaying URL values between a client and a
server. This is useful if you are looking for patterns
of web application attacks or OWASP attacks such
as SQL injection, IDOR, File inclusion or directory
traversal. All these attacks happen over URLs

index=* src=10.0.0.10 dest=10.0.0.11

sourcetype=http | stats values(uri)

Parsing general network traffic

Counting the number of requests initiated from the
network [domain.com] and sort them by IP
Address.

index=dataset domain.com
sourcetype=stream* | stats
count(src_ip) as Requests by src_ip |
sort - Requests

Looking for an image transferred between two ips

 index=dataset dest=23.22.63.114
"image.jpeg" src=192.168.250.70

Looking through specific file extensions on a host.

Useful for investigating what file(s) was encrypted
by a ransomware

index="dataset" host="MACLORY-AIR13"
(*.ppt OR *.pptx)

Parsing Sysmon events

Looking for an executable name [name] in sysmon
events

index=dataset name.exe
sourcetype=XmlWinEventLog:Microsoft-
Windows-Sysmon/Operational

Looking for process creation events with a named

executable
 index=dataset name.exe
sourcetype=XmlWinEventLog:Microsoft-
Windows-Sysmon/Operational EventCode=1

Sorting by MD5 hashes and

index=dataset name.exe
CommandLine=name.exe
| stats values(MD5)

searching with the hostname of a PC. Replace

hostname and its value

index=botsv1 hostname
sourcetype=XmlWinEventLog:Microsoft-
Windows-Sysmon/Operational

Looking through scheduled tasks activity.

 index="dataset" schtasks.exe
sourcetype="XmlWinEventLog:Microsoft-
Windows-Sysmon/Operational"
| dedup ParentCommandLine
| dedup CommandLine
| table ParentCommandLine CommandLine

Parsing Fortinet Firewall logs

Looking for hits to malicious websites and
providing an ip address

index=dataset sourcetype=fgt_utm
"192.168.250.70" category="Malicious
Websites"

Looking though process that changed file creation

time. Useful to look for files left by a malicious
executable such as [ransomware]. Remember to
add double [\] in the path.
 index=dataset
sourcetype=XmlWinEventLog:Microsoft-
Windows-Sysmon/Operational
host=hostname EventCode=2
TargetFilename="pathtofile" | stats
dc(TargetFilename)

USB attacks
USB records can be found in the windows registry.
You can also use somekeywords if you have info
about the USB

index=dataset sourcetype=winregistry
keyword

Sorting by the host and object

index=dataset sourcetype=winregistry
keyword | table host object data

Finding Sysmon Events for The Infected System on

an External Drive and show oldest events first. The
external drive letter is [d:]

index=dataset
sourcetype=XmlWinEventLog:Microsoft-
Windows-Sysmon/Operational
host=targetpcname "d:\\" | reverse

Refining Our Search To Find [D:] ONLY in

Command and Parent Command Line. we use the
parenthesis with OR so that we can look for a
reference to [D:] in either the CommandLine field
or the ParentCommandLine field. We then can take
our results and table both fields along with _time
and then sort oldest to newest.

index=dataset
sourcetype=XmlWinEventLog:Microsoft-
Windows-Sysmon/Operational
host=targetpcname
(CommandLine="*d:\\*" OR
ParentCommandLine="*d:\\*") | table
_time CommandLine ParentCommandLine |
sort _time
Finding the vendor name of a USB inserted into a
host

index="dataset" host=hostname usb

File sharing
File sharing events can be found in sysmon

index=dataset
sourcetype=XmlWinEventLog:Microsoft-
Windows-Sysmon/Operational
host=hostname

If we have information about the file server

address domain name we can use it

index=dataset
sourcetype=XmlWinEventLog:Microsoft-
Windows-Sysmon/Operational
host=hostname src="filserver.com"
By adding the stats command we can quickly see
the number of network connections the host has
made.

index=dataset
sourcetype=XmlWinEventLog:Microsoft-
Windows-Sysmon/Operational
host=hostname src="filserver.com" |
stats count by dest_ip | sort - count

Identifying the Hostname of the File Server

We can do that by specifying its ip address.

index=dataset
sourcetype="XmlWinEventLog:Microsoft-
Windows-Sysmon/Operational" ip

Parsing DNS
Scenario[1]
Its useful to find C2C domains when investigating a
compromised network or machine.
By using conditions like AND OR and NOT to
narrow down search criteria, we can quickly
exclude domains that we don't have an interest in.
Keep in mind AND is implied between conditions

Rather than writing our search such that the query

does not equal x and the query does not equal y,
we can use parenthesis to group a number of
conditions in this manner (query=x OR query=y)
and then place a NOT statement in front of the
parenthesis to apply to the entire group of query
values. Either syntax is acceptable.

Example is below. We can look through A records

for a specific ip and ignore a list of domains.

index=dataset sourcetype=stream:DNS
src=192.168.250.100 record_type=A NOT
(query{}=*.microsoft.com OR
query{}=*.bing.com OR
query{}=*.bing.com OR
query{}=*.windows.com OR
query{}=*.msftncsi.com) | stats count
by query{} | sort - 10 count
By tabling the result set and then using the reverse
command we get the earliest time and query at the
top of the list.

index=dataset sourcetype=stream:DNS
src=192.168.250.100 record_type=A NOT
(query{}=*.microsoft.com OR
query{}=*.waynecorpinc.local OR
query{}=*.bing.com OR query{}=isatap
OR query{}=wpad OR
query{}=*.windows.com OR
query{}=*.msftncsi.com) | table _time
query{} src dest | reverse

#TIP
To understand the relationship between processes
started and their parent/children, we can use the
table command to see the time along with the
process that was executed, its associated parent
process command, the process ID and parent
process ID. Using the reverse command moves the
first occurrence to the top. The IDs provide the
linkage between these different processes. While
Sysmon can show the immediate parent process
that spawned another process in a single event, it
cannot show an entire chain of processes being
created.

Scenario [2]
Sometimes we want statistical information about
the DNS connections. One of which is the round
trip time or RTT. This metric measures the time
spent from when the DNS query was sent until the
answers is received.
Say we want to calculate RTT sent to a specific
destination IP

index=* sourcetype=dns
dest=10.10.10.10 | stats avg(rtt) as
avgrtt | eval avgrtt=round(avgrtt,5)

You can use RTT for network troubleshooting as

well to detect delay in DNS responses. We can sort
rtt in reverse order and then use _time to check
the highest delay in minutes.
 index=* sourcetype=dns
dest=10.10.10.10 | sort - rtt | table
_time, rtt

Or we can group RTT by one minute slot and then

calculate the average.

index=* sourcetype=dns
dest=10.10.10.10 | bucket _time
span=1m | stats avg(rtt) by _time

Email Activity
Examining smtp events with keywords such as
email address and domain name

index="dataset" sourcetype=stream:smtp
email@email.com domain.com

Finding zip attachments

 index="dataset"
sourcetype="stream:smtp" *.zip

We can also list emails by subject and sender

index="dataset"
sourcetype="stream:smtp" | table
mailfrom, subject

Then we can examine the emails/events generated

by a specific sender

index="dataset"
sourcetype="stream:smtp"
mailfrom=test@test.com

FTP events
Looking through downloaded files
 index="botsv2" sourcetype="stream:ftp"
method=RETR
| reverse

AWS Events
Listing out the IAM users that accessed an AWS
service (successfully or unsuccessfully)

index="dataset"
sourcetype="aws:cloudtrail" IAMUser
| dedup user
| table user

Looking through AWS API activity that has

occurred without MFA (multi-factor authentication)

index="dataset"
sourcetype="aws:cloudtrail" NOT
tag::eventtype="authentication"
"userIdentity.sessionContext.attribute
s.mfaAuthenticated"=false
Looking through events related to n S3 bucket
publicly accessible.

index="dataset"
sourcetype="aws:cloudtrail"
eventType=AwsApiCall
eventName=PutBucketAcl
| reverse

Looking through files that were successfully

uploaded into the S3 bucket

index="dataset"
sourcetype="aws:s3:accesslogs" *PUT* |
reverse |

Looking through full process usage information on

aws instance.Useful to find coin mining activity
 index="dataset"
sourcetype="PerfmonMk:Process"
process_cpu_used_percent=100
| table _time host process_name
process_cpu_used_percent

Finding IAM user access key that generates the

most distinct errors when attempting to access
IAM resources

index="dataset"
sourcetype="aws:cloudtrail"
user_type="IAMUser"
errorCode!="success"
eventSource="iam.amazonaws.com"
| stats dc(errorMessage) by
userIdentity.accessKeyId

Given a key, we can look through the unauthorized

attempts to create a key for a specific resource.
 index="dataset"
sourcetype="aws:cloudtrail"
userIdentity.accessKeyId="AKIAJOGCDXJ5
NW5PXUPA" eventName=CreateAccessKey

Given a key, we can look through the unauthorized

attempts to describe an account.

sourcetype="aws:cloudtrail"
userIdentity.accessKeyId="AKIAJOGCDXJ5
NW5PXUPA"

eventName="DescribeAccountAttributes"

Symantec endpoint protection events

Investigating coin mining attacks and findig the
signature ID
[1]
 index="dataset"
sourcetype="symantec:*" *coin*
| table _time CIDS_Signature_ID

[2]

index="dataset"
sourcetype="symantec:*" *coin*

O365 Events
Looking through file upload events to onedrive

index="dataset"
sourcetype="ms:o365:management"
Workload=OneDrive
Operation=FileUploaded
| table _time src_ip user object
UserAgent
WIN event logs
Finding antivirus alerts. In the example below we
used symantec antivirus. Useful to find malicious
exes

index="dataset"
source="WinEventLog:Application"
SourceName="Symantec AntiVirus"
*Frothly*

Looking through users created

index="dataset"
source="wineventlog:security"
EventCode=4720

Finding groups a user assigned to

index="dataset"
source="wineventlog:security" svcvnc
"EventCode=4732"
Listing URLs accessed through [powershell]

index="dataset"
source="WinEventLog:Microsoft-Windows-
PowerShell/Operational" Message="*/*"
| rex field=Message "\$t\=[\'\"](?
<url>[^\'\"]+)"
| table url

Linux events
Looking through users added along with their
passwords

index="dataset" (adduser OR useradd)

source="/var/log/auth.log"

Osuery
Looking through users added along with their
passwords [Linux]
 index="botsv3"
sourcetype="osquery:results" useradd

Finding information about a process listening to

[1337] port

index="dataset" 1337
sourcetype="osquery:results"
"columns.port"=1337

SSH Events
When parsing SSH events, we mainly aim to detect
failed SSH logins which indicate possible brute
force attacks or successful SSH logins for the
purpose of auditing.

Below will list successful logins from 10.0.0.10

index=* src=10.0.0.10 sourcetype=ssh

auth_success=true
Likewise, for failed logins you can change
auth_success to false in the query above.

Detecting PrintNightmare
vulnerability
Identifies Print Spooler adding a new
Printer Driver

source="WinEventLog:Microsoft-Windows-
PrintService/Operational"
EventCode=316 category = "Adding a
printer driver" Message =
"*kernelbase.dll,*" Message =
"*UNIDRV.DLL,*" Message = "*.DLL.*"
| stats count min(_time) as firstTime
max(_time) as lastTime by OpCode
EventCode ComputerName Message
Detects spoolsv.exe with a child
process of rundll32.exe

| tstats count min(_time) as firstTime

max(_time) as lastTime from
datamodel=Endpoint.Processes where
Processes.parent_process_name=spoolsv.
exe
Processes.process_name=rundll32.exe by
Processes.dest Processes.user
Processes.parent_process
Processes.process_name
Processes.process
Processes.process_id
Processes.parent_process_id
Suspicious rundll32.exe instances
without any command-line arguments

| tstats count FROM

datamodel=Endpoint.Processes where
Processes.process_name=spoolsv.exe by
_time Processes.process_id
Processes.process_name Processes.dest
| rename "Processes.*" as *
| join process_guid _time
[| tstats count min(_time) as
firstTime max(_time) as lastTime FROM
datamodel=Endpoint.Filesystem where
Filesystem.file_path="*\\spool\\driver
s\\x64\\*"
Filesystem.file_name="*.dll" by _time
Filesystem.dest
Filesystem.file_create_time
Filesystem.file_name
Filesystem.file_path
| rename "Filesystem.*" as *
| fields _time dest
file_create_time file_name file_path
process_name process_path process]
| dedup file_create_time
 | table dest file_create_time,
file_name, file_path, process_name

Detects when a new printer plug-in has

failed to load

source="WinEventLog:Microsoft-Windows-
PrintService/Admin"
((ErrorCode="0x45A" (EventCode="808"
OR EventCode="4909"))
OR ("The print spooler failed to load
a plug-in module" OR
"\\drivers\\x64\\"))
| stats count min(_time) as
firstTime max(_time) as lastTime by
OpCode EventCode ComputerName Message

Using Splunk for Data Analytics

and Statistical Operations
Below are most common functions for
computational operations
Calculating the absolute value
#Syntax
abs(X)
absolute value of X
abs(number)

Case Function
Takes pairs of arguments X and Y, where X
arguments are Boolean expressions. When
evaluated to TRUE, the arguments return the
corresponding Y argument

case(X,"Y",…)
#Example
case(id == 0, "Amy", id == 1,"Brad", id ==
2, "Chris")

Ceiling of a number
ceil(X)

IP Address Identification
#Syntax
cidrmatch("X",Y)
Identifies IP addresses that belong to a particular
subnet
#example
cidrmatch("133.155.22.0/25",ip)

Coalese Function
#Syntax
coalesce(X,…)
The first value that is not NULL
#example
coalesce(null(), "Returned val", null())

Cosine Function
#Syntax
cos(X)

Exact Function
#Syntax
exact(X)
Evaluates an expression X using double precision
floating point arithmetic

exact(3.14+num)
Exp Function
#Syntax
exp(X)
e (natural number) to the power X (eX)

exp(3)

IF Function
#Syntax
if(X,Y,Z)
If X evaluates to TRUE, the result is the second
argument Y. If X evaluates to FALSE, the result
evaluates to the third argument Z

`if(error==200, "OK", "Error")

In Function
#Syntax
in(field,valuelist) TRUE if a value in valuelist
matches a value in field. You must use the in()
function embedded inside the if() function

`if(in(status, "404","500","503"),"true","false")
Boolean Functions

isbool(X)
TRUE if X is Boolean
isbool(field)
isint(X)
TRUE if X is an integer
isint(field)
isnull(X)
TRUE if X is NULL
isnull(field)
isstr(X)
TRUE if X is a string
isstr(field)

Len Function
#Syntax
len(X)
Character length of string X

len(field)
Like Function
#Syntax
like(X,"Y")
TRUE if and only if X is like the SQLite pattern in Y

like(field, "addr%")

Logarithm Function
#Syntax
log(X,Y)
Logarithm of the first argument X where the
second argument Y is the base. Y defaults to 10
(base-10 logarithm)

log(number,2)

Lower Function
#Syntax
lower(X)
Lowercase of string X

lower(username)
Ltrim Function
#Syntax
ltrim(X,Y)
X with the characters in Y trimmed from the left
side. Y defaults to spaces and tabs

ltrim(" ZZZabcZZ ", " Z")

Match Function
#Example
match(X,Y)
TRUE if X matches the regular expression pattern Y

match(field, "^\d{1,3}\.\d$")

Max Function
#Syntax
max(X,…)
The maximum value in a series of data X,…

max(delay, mydelay)
Hashing
#Syntax
md5(X)
MD5 hash of a string value X

md5(field)

Min Function
#Syntax
min(X,…)
The minimum value in a series of data X,…

min(delay, mydelay)

Mv Count Function
#Syntax
mvcount(X)
Number of values of X

mvcount(multifield)
Mv Filter Function
#Syntax
mvfilter(X)
Filters a multi-valued field based on the Boolean
expression X

mvfilter(match(email, "net$"))

mvindex Function
#Syntax
mvindex(X,Y,Z) Returns a subset of the multi-
valued field X from start position (zero-based) Y to
Z (optional)

mvindex(multifield, 2)

mvjoin Function
#Syntax
mvjoin(X,Y)
Joins the individual values of a multi-valued field X
using string delimiter Y

mvjoin(address, ";")
now() Function
#Syntax
now()
Current time as Unix timestamp

now()

null() Function
#Syntax
null()
NULL value. This function takes no arguments.

null()

nullif Function
#Syntax
nullif(X,Y)
X if the two arguments, fields X and Y, are
different. Otherwise returns NULL.

nullif(fieldX, fieldY)
random() Function
#Syntax
random()
Pseudo-random number ranging from 0 to
2147483647

random()

relative_time Function
#Syntax
relative_time (X,Y)
Unix timestamp value of relative time specifier Y
applied to Unix timestamp X

relative_time(now(),"-1d@d")

replace Function
#Syntax
replace(X,Y,Z)
A string formed by substituting string Z for every
occurrence of regex string Y in string X
The example swaps the month and day numbers of
a date.
 #Example
`replace(date, "^(\d{1,2})/(\d{1,2})/", "\2/\1/")

round Function
#Syntax
round(X,Y)
X rounded to the number of decimal places
specified by Y, or to an integer for omitted Y

round(3.5)

rtrim Function
#Syntax
rtrim(X,Y)
X with the characters in (optional) Y trimmed from
the right side. Trim spaces and tabs for unspecified
Y.

rtrim(" ZZZZabcZZ ", " Z")

split Function
#Syntax
split(X,"Y")
X as a multi-valued field, split by delimiter Y.
split(address, ";")

sqrt Function
#Syntax
sqrt(X)
Square root of X

sqrt(9) # 3

strftime Function
#Syntax
strftime(X,Y)
Unix timestamp value X rendered using the format
specified by Y

strftime(time, "%H:%M")

strptime Function
#Syntax
strptime(X,Y)
Value of Unix timestamp X as a string parsed from
format Y
strptime(timeStr, "%H:%M")

substr Function
#Syntax
substr(X,Y,Z)
Substring of X from start position (1-based) Y for
(optional) Z characters

substr("string", 1, 3)

time Function
#Syntax
time()
Current time to the microsecond.

time()

tonumber Function
#Syntax
tonumber(X,Y)
Converts input string X to a number of numerical
base Y (optional, defaults to 10)
tonumber("FF",16)

tostring Function
#Syntax
tostring(X,Y)
Field value of X as a string. If X is a number, it
reformats it as a string. If X is a Boolean value, it
reformats to "True" or "False" strings. If X is a
number, the optional second argument Y is one
of:"hex": convert X to hexadecimal,"commas":
formats X with commas and two decimal places,
or"duration": converts seconds X to readable time
format HH:MM:SS.

tostring(500, "duration")

typeof Function
#Syntax
typeof(X)
String representation of the field type

This example returns "NumberBool":

| makeresults | eval n=typeof(12) +
typeof(1==2)

urldecode Function
#Syntax
urldecode(X) URL X, decoded.

urldecode("http%3A%2F%2Fwww.site.com%2Fvie
w%3Fr%3Dabout")

validate Function
#Syntax
validate(X,Y,…)
For pairs of Boolean expressions X and strings Y,
returns the string Y corresponding to the first
expression X which evaluates to False, and
defaults to NULL if all X are True.

validate(isint(N), "Not an integer", N>0, "Not

positive")
Splunk Config and
Troubleshooting
Restart Splunk

splunk stop
splunk enable webserver
splunk start

List Splunk Configs

splunk btool inputs list

splunk btool check #check configs

License Management
Show current licenses

splunk list licenses

User Management
Search for all users who are admins
 splunk _internal call
/services/authentication/users -
get:search admin

See which users could edit indexes

splunk _internal call

/services/authentication/users -
get:search indexes_edit

Reload authentication configurations for Splunk 6.x

splunk _internal call

/authentication/providers/services/_re
load