Logging: WTH? Dr. Anton Chuvakin WRITTEN: 2008 DISCLAIMER: Security is a rapidly changing field of human endeavor.

Threats we face literally change every day; moreover, many security professionals consider the rate of change to be accelerating. On top of that, to be able to stay in touch with such ever-changing reality, one has to evolve with the space as well. Thus, even though I hope that this document will be useful for to my readers, please keep in mind that is was possibly written years ago. Also, keep in mind that some of the URL might have gone 404, please Google around. The what? System logs, audit trails, network logs, IDS and IPS alerts – and all other data that information systems are spewing at us at an ever-increasing rate. Soon more log sources will be added to today’s mix of servers, applications, firewalls and security appliances – all the way to mobile devices that log and then possibly internet-connected appliance. A typical large company already has gigabytes of logs produced each week. Why is it important? System logs are the source of three types of insight: 1. Security – logs are used to detect attacks; they are also indispensable for incident investigations and forensics 2. Regulatory compliance – many regulations in USA, UK and other countries mandate the collection and review of logs; auditors may ask organizations for a proof of log collection and review 3. IT operational efficiency – sysadmins reach for logs when troubleshooting the system problems; logs are also used to measure network and application performance. How does it work? Many piece of IT infrastructure product heaps of logs by default (e.g. Unix servers create syslog records and Windows systems log to Even Log); others need to be configured to produce larger volumes of logs to make them useful for security, compliance and operations. However, some technologies, such as databases, come with almost no logging configured by default and those who

deploy them need to change configurations and enable logging settings so that logs are created and can be used for the above purposes. Who needs it? IT administrators, IT managers, security analysts, incident responders, IT directors and even CIOs and compliance officers will either look at logs or at reports based on them. Given the diverse range of uses, it is not surprising that both system admins and CIO will use logs to accomplish their goals. Here are the recent poll results on log use that shows it:

How do they get it? The best way to take control of the logs is to deploy a log management system. More advanced organization use a log data warehouse approach. Such system centralizes the collection, storage and access of log data across the enterprise, freeing organizations from a device-by-device approach to log management, which is inefficient and heavily relies on manual tasks . Providing a central repository for all log data, log data warehous enables you to easily query and report on this data with unparalleled speed and efficiently manage the massive amounts of log data generated through network devices, security gear, operating systems, network servers, databases, and more. It also goes beyond simple storage, allowing users to discover and act on relationships between data from these heterogeneous data sources. Additional: What’s wrong with what they have now? Today many organizations employ an ad hoc approach to logs: a situation when you have a security team "owning" network IDS logs, network team having firewall and router logs (as well as all SNMP traps) and a system admins possessing the logs from servers and desktop is not only sad, counterproductive, inefficient and wasteful, but also dangerous. Where does such approach to logs (when they are divided by both technical and political chasms!) breaks down most painfully? In case of an incident response,

of course. This is where instead of one query across all logs and all log sources (or whatever needed subset of logs or log sources), people end up with having run around, beg for logs, connect, wait, wait, download logs, dig in many places at once – overall, waste time and energy in massive amounts. All of the above instead of connecting to a log management system and running a few reports, drilldowns and searches across the relevant logs! Where else does it break down? Compliance, of course! Most regulations and mandates don't call out logs by the type of the log source, but apply to all logs across. Thus one system to verify the compliance status is much more productive compared to digging in many systems. ABOUT THE AUTHOR: This is an updated author bio, added to the paper at the time of reposting in 2009. Dr. Anton Chuvakin (http://www.chuvakin.org) is a recognized security expert in the field of log management and PCI DSS compliance. He is an author of books "Security Warrior" and "PCI Compliance" and a contributor to "Know Your Enemy II", "Information Security Management Handbook" and others. Anton has published dozens of papers on log management, correlation, data analysis, PCI DSS, security management (see list www.info-secure.org) . His blog http://www.securitywarrior.org is one of the most popular in the industry. In addition, Anton teaches classes and presents at many security conferences across the world; he recently addressed audiences in United States, UK, Singapore, Spain, Russia and other countries. He works on emerging security standards and serves on the advisory boards of several security start-ups. Currently, Anton is developing his security consulting practice www.securitywarriorconsulting.com/ , focusing on logging and PCI DSS compliance for security vendors and Fortune 500 organizations. Dr. Anton Chuvakin was formerly a Director of PCI Compliance Solutions at Qualys. Previously, Anton worked at LogLogic as a Chief Logging Evangelist, tasked with educating the world about the importance of logging for security, compliance and operations. Before LogLogic, Anton was employed by a security vendor in a strategic product management role. Anton earned his Ph.D. degree from Stony Brook University.