Enabling Grids for E

E-sciencE
sciencE

Logging Proposal
for JRA1 Code

JRA1 All-Hands,
All H d H Helsinki,
l i ki 18-20
18 20 J
Jun, 2007

Steve Fisher/RAL and Ales Krenek/Cesnet

www.eu-egee.org
www eu egee org
www.glite.org

INFSO-RI-508833
 Overview
Enabling Grids for E-sciencE

• Why the logging discussion – again?

• Aims

• Specific recommendations

• Next steps

INFSO-RI-508833 Steve Fisher/RAL 2

 Why discuss logging again?
Enabling Grids for E-sciencE

• Have discussed on several occasions:

– Each time the answer was to adopt some kind of logging
framework
• Re-triggered by David Groep’s document: Middleware
Security Audit Logging Guidelines (version 0.8)
– This document says what should be logged (anything security
related) and how it should be logged (via syslog)
– It is also the basis of some ad-hoc OGF agreement with OSG
• We were asked to produce a recommendation
– This was sent last week to all JRA1 members - so far it has been
remarkably quiet.
– Aim to produce final version next week

INFSO-RI-508833 Steve Fisher/RAL 3

 Why another document
Enabling Grids for E-sciencE

• David’s document matches its title.

– He is onlyy concerned about security
y audits.
• Logging serves multiple purposes:
– developers
p to check p
program
g logic
g
– developers to understand problems on a running system
– sys-admins to check for correct functioning
– sys-admins to investigate possible security incidents

INFSO-RI-508833 Steve Fisher/RAL 4

 Aims
Enabling Grids for E-sciencE

• Unify configuration of logging

– This is the main requirement,
q , it is felt to be unavoidable for
smooth deployment of the services
– Other requirements follow from it
• Follow the spirit of the security audit logging
guidelines document
– This
Thi iis a good
dddocumentt with
ith valid
lid concerns
• Uniformity of what is logged
– makes life easier for site administrators
– improves communication between developers when interaction
between middleware components
p is analysed.
y
• Minimise number of solutions
– With diverse developer
p communities we are unlikely
y to find a
solution which suits everybody
INFSO-RI-508833 Steve Fisher/RAL 5
 Common Configuration Mechanism
Enabling Grids for E-sciencE

• Sys-admins determine what is logged at their site

• If sys-admin
sys admin suspects a problem he may want to
increase the logging
• Should be easy y to do this without intimate knowledge
g
of the middleware

All JRA1 components MUST use a common

configuration file format modeled on the log4j
one … (The
(Th configuration
fi i file
fil should
h ld beb in
i XML
XM
as it is more powerful that a properties file).

INFSO-RI-508833 Steve Fisher/RAL 6

 Specific libraries to consider
Enabling Grids for E-sciencE

• Java
– either log4j
g j or Jakarta commons
• C++
– Log4cxx
g
– Should try HEAD version and perhaps contact developers
• C
– log4c
• Python
– log4py

INFSO-RI-508833 Steve Fisher/RAL 7

 Set Of Loggers
Enabling Grids for E-sciencE

A dedicated securityy logger

gg MUST be used for securityy
related messages
A dedicated access logger MUST be used to log initial
access to a service
A dedicated control logger MUST be used for service
startup, configuration and shutdown
Other loggers SHOULD be named in a shallow
hi
hierarchy
h - nott simply
i l one logger
l per class.
l

• Finer severity levels (e

(e.g.
g as defined by syslog) may
be used when appropriate.
• If you use tomcat this handles the access log

INFSO-RI-508833 Steve Fisher/RAL 8

 Format of Message String
Enabling Grids for E-sciencE

• Brief but include as much relevant information as practical.

• Only log at most 768 bytes
– Some syslog implementations cannot deal with messages of more than
1024 bytes,
– Restriction only applies to the loggers control, security and access to
ensure that
th t they
th can beb sentt to
t syslog.
l
• Any key-value pairs should be separated by a comma (which is
preferred) or a space; and the key and value should be separated
by an "="
• White space should be used as necessary for readability
• Only
O y characters
c a acte s fromo tthee se
seven-bit
e b t ASCII
SC set with t dec
decimala values
a ues
in the range 32-126 should be used in the message.
– It can be assumed that the syslog appender can translate or escape
anything
a y g nasty.
as y
• Messages must be self-contained.
– If your message is too long devise a way to split it into multiple stand-
alone messages

INFSO-RI-508833 Steve Fisher/RAL 9

 What to log
Enabling Grids for E-sciencE

Service start up,p, configuration

g and termination
To control logger
Authorization logging
To security logger
Initial service access, session establishment and
termination
To access logger
gg

• Other Logging
– as the developer sees fit

INFSO-RI-508833 Steve Fisher/RAL 10

 Message Context
Enabling Grids for E-sciencE

• If the request processing etc. is related to an externally

identifiable entity
y (j
(jobid,, file guid,
g , etc.)) this identifier
should be logged.
• Whenever there is a logical notion of session, request,
etc., and multiple messages related to one such entity
instance can be logged, it is recommended that a
unique id is assigned to such an entity and it is logged
in all the messages consistently.

INFSO-RI-508833 Steve Fisher/RAL 11

 Next Steps
Enabling Grids for E-sciencE

• We hope we can agree to go forward with this proposal

today
y
– Perhaps with minor changes
• Then need a timescale for implementation
– Otherwise, once more, nothing will actually happen

Comments ???

INFSO-RI-508833 Steve Fisher/RAL 12