The Bro Network Security Monitor

Broverview
Bro Workshop 2011
NCSA, Urbana-Champaign, IL
Bro Workshop 2011

Outline

Bro Workshop 2011 2

Outline

Philosophy and Architecture
A framework for network traffic analysis.

Bro Workshop 2011 2

Bro Workshop 2011 2 . Outline Philosophy and Architecture A framework for network traffic analysis. History From research to operations.

History From research to operations. logs. scripts. Bro Workshop 2011 2 . Architecture Components. cluster. Outline Philosophy and Architecture A framework for network traffic analysis.

What is Bro? Bro Workshop 2011 3 .

What is Bro? Packet Capture Bro Workshop 2011 3 .

What is Bro? Packet Capture Traffic Inspection Bro Workshop 2011 3 .

What is Bro? Packet Capture Traffic Inspection Attack Detection Bro Workshop 2011 3 .

What is Bro? Packet Capture Traffic Inspection Attack Detection NetFlow Log Recording syslog Bro Workshop 2011 3 .

What is Bro? Packet Capture Traffic Inspection Attack Detection NetFlow Log Recording syslog Flexibility Abstraction Data Structures Bro Workshop 2011 3 .

What is Bro? Packet Capture Traffic Inspection Attack Detection NetFlow Log Recording syslog Flexibility Abstraction Data Structures Bro Workshop 2011 3 .

What is Bro? Packet Capture Traffic Inspection Attack Detection NetFlow Log Recording syslog Flexibility Abstraction Abstraction Data Data Structures Structures Bro Workshop 2011 3 .

What is Bro? Packet Capture Traffic Inspection Attack Detection “Domain-specific Python” NetFlow Log Recording syslog Flexibility Abstraction Abstraction Data Data Structures Structures Bro Workshop 2011 3 .

What is Bro? Packet Capture Sum is mo re than the pieces Traffic Inspection Attack Detection “Domain-specific Python” NetFlow Log Recording syslog Flexibility Abstraction Abstraction Data Data Structures Structures Bro Workshop 2011 3 .

Philosophy Bro Workshop 2011 4 .

Bro Workshop 2011 4 . Reset your idea of an IDS before starting to use Bro. Philosophy Fundamentally different from other IDS.

Bro Workshop 2011 4 . Real-time network analysis framework. Reset your idea of an IDS before starting to use Bro. but many use it for general traffic analysis. Primarily an IDS. Philosophy Fundamentally different from other IDS.

Can accommodate a range of detection approaches. Philosophy Fundamentally different from other IDS. Bro Workshop 2011 4 . Primarily an IDS. Real-time network analysis framework. Reset your idea of an IDS before starting to use Bro. but many use it for general traffic analysis. Policy-neutral at the core.

Can accommodate a range of detection approaches. Policy-neutral at the core. Highly stateful. Bro Workshop 2011 4 . Tracks extensive application-layer network state. Philosophy Fundamentally different from other IDS. Primarily an IDS. Reset your idea of an IDS before starting to use Bro. but many use it for general traffic analysis. Real-time network analysis framework.

Extensively logs what it sees. Philosophy Fundamentally different from other IDS. Tracks extensive application-layer network state. but many use it for general traffic analysis. Reset your idea of an IDS before starting to use Bro. Highly stateful. Bro Workshop 2011 4 . Real-time network analysis framework. Policy-neutral at the core. Supports forensics. Primarily an IDS. Can accommodate a range of detection approaches.

Target Audience Bro Workshop 2011 5 .

Effective also with liberal security policies. Target Audience Large-scale environments. Bro Workshop 2011 5 .

Network-savvy users. Effective also with liberal security policies. Bro Workshop 2011 5 . Target Audience Large-scale environments. Requires understanding of your network.

Bro Workshop 2011 5 . Command-line based. Effective also with liberal security policies. fully customizable. Requires understanding of your network. Target Audience Large-scale environments. Network-savvy users. Unixy mindset.

Bro History 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Vern writes 1st line of code Bro Workshop 2011 6 .

Bro History 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Vern writes 1st line of code LBNL starts using Bro operationally Bro Workshop 2011 6 .

4 LBNL starts HTTP analysis Signatures BinPAC DHCP/BitTorrent using Bro Scan detector SMTP IRC/RPC analyzers HTTP entities operationally IP fragments IPv6 support 64-bit support NetFlow Linux support User manual Sane version Bro Lite Deprecated numbers v0.9aX v1.6 v0.4 v0.2 v1.1/v1.8a37 v1.2 v0.5 Bro 2.8aX v1.7a90 v0.7a48 0.3 Consistent Communication Ctor expressions CHANGES Persistence GeoIP Namespaces Conn Compressor Log Rotation Bro Workshop 2011 6 . Bro History 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Vern writes 1st v0.0 line of code 1st CHANGES RegExps Profiling SSL/SMB when Stmt BroControl entry Login analysis State Mgmt STABLE releases Resource BroLite tuning Bro Waters Broccoli DPD v0.8aX/0.7a175/0.0 v1.

2 v0.8aX/0.8aX v1.4 v0.4 LBNL starts HTTP analysis Signatures BinPAC DHCP/BitTorrent using Bro Scan detector SMTP IRC/RPC analyzers HTTP entities operationally IP fragments IPv6 support 64-bit support NetFlow Linux support User manual Sane version Bro Lite Deprecated numbers v0.1/v1.6 v0.5 Bro 2.7a48 0. 2nd Path Autotuning 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Vern writes 1st v0.0 v1.7a90 v0.2 v1. State Shunt Anonymizer Parallel Prototype BinPAC Stepping Stone Active Mapping DPD USENIX Paper Detector Context Signat.9aX v1. Bro Cluster Independ.3 Consistent Communication Ctor expressions CHANGES Persistence GeoIP Namespaces Conn Compressor Log Rotation Bro Workshop 2011 6 .0 line of code 1st CHANGES RegExps Profiling SSL/SMB when Stmt BroControl entry Login analysis State Mgmt STABLE releases Resource BroLite tuning Bro Waters Broccoli DPD v0.8a37 v1. Bro History Host Context Time Machine Academic Enterprise Traffic Publications TRW State Mgmt.7a175/0.

Research Heritage Bro Workshop 2011 7 .

Bridging gap between academia and operations. Research Heritage Much of Bro is coming out of research projects. Bro Workshop 2011 7 .

However. Bridging gap between academia and operations. Research Heritage Much of Bro is coming out of research projects. Bro Workshop 2011 7 . documentation. We were lacking resources for development. polishing. that meant limited engineering resources.

Office of Cyberinfrastructure Bro Workshop 2011 7 . Research Heritage Much of Bro is coming out of research projects. NSF now funding Bro development at ICSI and NCSA. that meant limited engineering resources. documentation. We were lacking resources for development. Bridging gap between academia and operations. Full-time engineers working 3 years on capabilities & user experience. polishing. However.

polishing. We were lacking resources for development. Bridging gap between academia and operations. that meant limited engineering resources. Office of Cyberinfrastructure Bro Workshop 2011 7 . Objective is a sustainable development model. Full-time engineers working 3 years on capabilities & user experience. However. Research Heritage Much of Bro is coming out of research projects. Aiming to create a larger user and development community. documentation. NSF now funding Bro development at ICSI and NCSA.

Deployment Internal Internet Network Bro Workshop 2011 8 .

Deployment Ta Internal Internet Network Bro Bro Workshop 2011 8 .

Supports FreeBSD/Linux/OS X. Bro Workshop 2011 8 . Deployment Ta Internal Internet Network Bro Runs on commodity platforms. ! Standard PCs & NICs.

Architecture Packets Network Bro Workshop 2011 9 .

Architecture Events Protocol Decoding Event Engine Packets Network Bro Workshop 2011 9 .

Architecture Logs Notification Analysis Logic Policy Script Interpreter Events Protocol Decoding Event Engine Packets Network Bro Workshop 2011 9 .

Architecture Logs Notification “User Interface” Analysis Logic Policy Script Interpreter Events Protocol Decoding Event Engine Packets Network Bro Workshop 2011 9 .

Script Example: Matching URLs Task: Report all Web requests for files called “passwd”. Bro Workshop 2011 10 .

# Decoded URL. # HTTP method. # Connection. method: string. # Requested URL. event http_request(c: connection.. unescaped_URI: string.).*passwd/ ) NOTICE(.. version: string) # HTTP version. Script Example: Matching URLs Task: Report all Web requests for files called “passwd”. # Alarm. { if ( method == "GET" && unescaped_URI == /. } Bro Workshop 2011 10 . original_URI: string.

Script Example: Scan Detector Task: Count failed connection attempts per source address. Bro Workshop 2011 11 .

event connection_rejected(c: connection) { local source = c$id$orig_h. # Increase counter.. # Alarm. NOTICE(. Script Example: Scan Detector Task: Count failed connection attempts per source address.. local n = ++attempts[source]. # Get source address.). global attempts: table[addr] of count &default=0. } Bro Workshop 2011 11 . if ( n == SOME_THRESHOLD ) # Check for threshold.

Distributed Scripts Bro Workshop 2011 12 .

000 lines of script code. Prewritten functionality that’s just loaded. Distributed Scripts Bro comes with >10. Bro Workshop 2011 12 .

000 lines of script code. Amendable to extensive customization and extension. Distributed Scripts Bro comes with >10. Bro Workshop 2011 12 . Prewritten functionality that’s just loaded. Scripts generate alarms and logs.

Example Logs Bro Workshop 2011 13 .

.] > cat conn. Example Logs > bro -i en0 [ ..log Bro Workshop 2011 13 .. wait ..

186.5597 192.186.150.118.150.6063 192.186.169 53090 198.169 53116 82.. wait .437460 8661 63663 1144876596.6102 192...151.169 53075 207.02667 3027 11761 1144876745.orig_h id.169 53082 198.] > cat conn.186.186.7.189.143 80 tcp http 0.597711 337 5146 1144876741.255.82 80 tcp http 4.14929 435 66363 1144876612.94.004346 422 1637 1144876605.218 80 tcp http 16.227.169 53115 82.129 80 tcp http 0.203.150.resp_p proto service duration obytes rbytes [.186.186.150.resp_h id..1198 192.237.237.150.189.218 80 tcp http 16.] 1144876741. Example Logs > bro -i en0 [ .169 53117 66.7789 192.372440 461 753 1144876606..029663 347 1011 Bro Workshop 2011 13 .255.169 53051 193..150.73 80 tcp http 0.6847 192.orig_p id.102.150.99 80 tcp http 1.4693 192.log #fields ts id.94.

169 53115 82.14929 435 66363 1144876612.372440 461 753 1144876606.99 80 tcp http 1.004346 422 1637 1144876605.94.94.029663 347 1011 > cat http.203.] > cat conn.186.102.218 80 tcp http 16.150..129 80 tcp http 0.log Bro Workshop 2011 13 .169 53051 193.] 1144876741.186.150..resp_p proto service duration obytes rbytes [.150.169 53075 207.150..169 53082 198.118.73 80 tcp http 0.189...150.186.143 80 tcp http 0.227.186.255.437460 8661 63663 1144876596.186.82 80 tcp http 4.186.151.150.255.169 53090 198.7.orig_p id..7789 192.237.597711 337 5146 1144876741.5597 192.169 53117 66. Example Logs > bro -i en0 [ .4693 192.1198 192.169 53116 82. wait .6102 192.189.orig_h id.02667 3027 11761 1144876745.log #fields ts id.186.6063 192.237.150.218 80 tcp http 16.resp_h id.6847 192.

.] 1144876741.186.169 53116 docs.png 304 Mozilla/5.150.73 80 tcp http 0.0 1144876742.6144 192.169 53051 193.189.82 80 tcp http 4.google.186.186.150.1198 192.png 304 Mozilla/5.372440 461 753 1144876606.0 1144876742..169 53082 198.004346 422 1637 1144876605.6102 192.html 200 Mozilla/5.] > cat conn.150.5597 192.3337 192.0 1144876742.029663 347 1011 > cat http.150.python..255.437460 8661 63663 1144876596.3337 192.150.186.org /lib/lib.0 1144876742.94.0 1144876741.150..3338 192.3337 192.118.218 80 tcp http 16...94.169 53116 82.150.203.02667 3027 11761 1144876745.png 304 Mozilla/5.python.] 1144876741. wait .169 53117 www.186.6063 192.2838 192.png 304 Mozilla/5.237.org /icons/previous.102.169 53090 198.218 80 tcp http 16.169 53116 docs.resp_h id.150.186.237.python.99 80 tcp http 1.orig_p id.186.log #fields ts id.169 53116 docs.150.org /icons/modules.7789 192.169 53115 82.169 53116 docs.python.3337 192.169 53115 docs.150.python.150.png 304 Mozilla/5.169 53117 66.151.150.org /icons/next.169 53116 docs.186.png 304 Mozilla/5.0 1144876742.python.log #fields ts id.7.4693 192.0 1144876745.189.css 200 Mozilla/5.] host uri status_code user_agent [.org /icons/up.143 80 tcp http 0.6335 192..orig_h id.186.169 53075 207.6847 192.0 Bro Workshop 2011 13 .186.python.597711 337 5146 1144876741.python.150.1687 192.orig_h id.227.186.org /icons/contents.186.150.169 53116 docs.com / 200 Mozilla/5.0 1144876742.org /icons/index.resp_p proto service duration obytes rbytes [.255..150..186. Example Logs > bro -i en0 [ .orig_p [.org /lib/lib.186.186.14929 435 66363 1144876612.150.129 80 tcp http 0..169 53116 docs.186.

Bro Ecosystem Tap Internal Internet Network Bro Bro Workshop 2011 14 .

Bro Ecosystem Tap Internal Internet Network Bro Control Output BroControl User Interface Bro Workshop 2011 14 .

Bro Ecosystem Tap Internal Internet Network Contributed Functionality Scripts Bro Control Output BroControl User Interface Bro Workshop 2011 14 .

Bro Ecosystem Tap Internal Internet Network Contributed Functionality Events Scripts Bro State Other Bros Control Output BroControl User Interface Bro Workshop 2011 14 .

Bro Ecosystem

Tap
Internal
Internet Network

Contributed Functionality Events
Scripts Bro State
Other Bros

Control Output
Events
Bro Client Communication Library

BroControl
Broccoli
User Interface

Bro Workshop 2011 14

Bro Ecosystem

Tap
Internal
Internet Network

Contributed Functionality Events
Scripts Bro State
Other Bros

Control Output
Events
Bro Client Communication Library

BroControl Broccoli Python

Broccoli Broccoli Ruby

User Interface
(Broccoli Perl)

Bro Workshop 2011 14

Bro Ecosystem

Tap
Internal
Internet Network

Contributed Functionality Events
Scripts Bro State
Other Bros

Control Output
Events
Bro Client Communication Library
bro-aux BinPAC capstats

BroControl Broccoli Python
trace-
BTest
summary
Broccoli Broccoli Ruby

User Interface
(Broccoli Perl)

Bro Workshop 2011 14

0. Bro Ecosystem Bro Distribution Tap Internal Internet bro-2.tar.gz Network Contributed Functionality Events Scripts Bro State Other Bros Control Output Events Bro Client Communication Library bro-aux BinPAC capstats BroControl Broccoli Python trace- BTest summary Broccoli Broccoli Ruby User Interface (Broccoli Perl) Bro Workshop 2011 14 .

bro-ids.gz Network Contributed Functionality Events Scripts Bro State Other Bros Control Output Events Bro Client Communication Library bro-aux BinPAC capstats BroControl Broccoli Python trace- BTest summary Broccoli Broccoli Ruby User Interface (Broccoli Perl) http:://www.org Bro Workshop 2011 14 . Bro Ecosystem Bro Distribution Tap Internal Internet bro-2.0.bro-ids.tar.org/download git://git.

Bro Cluster Ecosystem Tap Internal Internet Network Contributed Functionality Events Scripts Bro State External Bro Control Output Events Bro Client Communication Library bro-aux BinPAC capstats BroControl Broccoli Python trace- BTest summary Broccoli Broccoli Ruby User Interface (Broccoli Perl) Bro Workshop 2011 15 .

Bro Cluster Ecosystem Tap Internal Internet Network Contributed Functionality Events Scripts Bro State External Bro Control Output Events Bro Client Communication Library bro-aux BinPAC capstats BroControl Broccoli Python trace- BTest summary Broccoli Broccoli Ruby User Interface (Broccoli Perl) Bro Workshop 2011 15 .

Bro Cluster Ecosystem Tap Internal Internet Network Load- Balancer Contributed Functionality Events Scripts Bro State External Bro Control Output Events Bro Client Communication Library bro-aux BinPAC capstats BroControl Broccoli Python trace- BTest summary Broccoli Broccoli Ruby User Interface (Broccoli Perl) Bro Workshop 2011 15 .

Bro Cluster Ecosystem Tap Internal Internet Network Load- Balancer Packets Contributed Functionality Bro Bro Bro Events Bro Scripts Bro State External Bro Control Output Events Bro Client Communication Library bro-aux BinPAC capstats BroControl Broccoli Python trace- BTest summary Broccoli Broccoli Ruby User Interface (Broccoli Perl) Bro Workshop 2011 15 .

Bro Cluster Ecosystem Tap Internal Internet Network Load- Balancer Packets Contributed Functionality Bro Bro Bro Events Bro Scripts Bro State External Bro Control Control Output Output Events Bro Client Communication Library bro-aux BinPAC capstats BroControl Broccoli Python trace- BTest summary Broccoli Broccoli Ruby User UserInterface Interface (Broccoli Perl) Bro Workshop 2011 15 .

Bro Cluster Ecosystem Tap Internal Internet Network Load- Balancer Packets “Frontend” Contributed Functionality Bro Bro Bro Events Bro Scripts Bro “Workers” State External Bro Control Control Output Output Events “Manager” Bro Client Communication Library bro-aux BinPAC capstats BroControl Broccoli Python trace- BTest summary Broccoli Broccoli Ruby User UserInterface Interface (Broccoli Perl) Bro Workshop 2011 15 .

“The Bro Team” Vern Paxson Gregor Maier Jim Barlow Jonathan Siwek Gilbert Clark Adam Slagell Seth Hall Robin Sommer Christian Kreibich Daniel Thayer Hui Lin Matthias Vallentin Bro Workshop 2011 16 .