Professional Documents
Culture Documents
OSSEC Log Mangement With Elasticsearch
OSSEC Log Mangement With Elasticsearch
$ whoami
Software Architect for Trend Micro Data Analytics Group
Blogger for Trend Micro Security Intelligence and Simply
Security
Email: vichargrave@gmail.com
Website: vichargrave.com
Twitter: @vichargrave
LinkedIn: www.linkedin.com/in/vichargrave
syslog
commercial or
open source
SIEM
Syslog
Syslog
commercial
SIEM
Logstash
Kibana
Elasticsearch
Open source, distributed, full text search engine
Based on Apache Lucene
Stores data as structured JSON documents
Supports single system or multi-node clusters
Easy to set up and scale just add more nodes
Provides a RESTful API
Installs with RPM or DEB packages and is controlled
with a service script.
Elasticseach Elements
Index contains documents, table
Document contains fields, row
Field contains string, integer, JSON object, etc.
Shard smaller divisions of data that can be stored
across nodes
Replica copy of the primary shard
Logstash
Log aggregator and parser
Supports transferring parsed data directly to
Elasticsearch
Controlled by a configuration file that specifies input,
filtering (parsing) and output
Key to adapting Elasticsearch to other log formats
Run logstash in logstash home directory as follows:
bin/logstash conf <logstash config file>
10
OSSEC logstash.conf
input {
#
stdin{}
udp {
}
}
filter {
if [type] == "syslog" {
grok {
# SEE NEXT SLIDE
mutate {
}
}
}
output {
# stdout {
#
codec => rubydebug
# }
elasticsearch_http {
grok { }
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_host}
%{DATA:syslog_program}: Alert Level: %{NONNEGINT:Alert_Level};
Rule: %{NONNEGINT:Rule} - %{DATA:Description};
Location: %{DATA:Location}; (srcip: %{IP:Src_IP};%{SPACE})?
(dstip: %{IP:Dst_IP};%{SPACE})?
(src_port: %{NONNEGINT:Src_Port};%{SPACE})?
(dst_port: %{NONNEGINT:Dst_Port};%{SPACE})?
(user: %{USER:User};%{SPACE})?%{GREEDYDATA:Details}"
}
add_field => [ "ossec_server", "%{host}" ]
12
Kibana
General purpose query UI
Javascript implementation
Query Elasticsearch without coding
Includes many widgets
Run Kibana in browser as follows:
http://<web server ip>:<port>/<kibana path>
13
Kibana config.js
/** @scratch /configuration/config.js/5
* ==== elasticsearch
*
* The URL to your elasticsearch server. You almost certainly don't
* want +http://localhost:9200+ here. Even if Kibana and Elasticsearch
* are on the same host. By default this will attempt to reach ES at the
* same host you have kibana installed on. You probably want to set it to
* the FQDN of your elasticsearch host
*/
elasticsearch: http://+"<elasticsearch node IP>"+":9200",
14
15
16
17
18
19
20
Back to Reality
Free
21
22
Further Information
Elasticsearch
http://www.elasticsearch.org
Logstash
http://logstash.net
Kibana
http://www.elasticsearch.org/overview/kibana/
ElasticHQ
http://elastichq.org
23
Any questions?
24