TRACEABLE LOG

FILES
PROBLEM STATEMENT
Given unstructured log files of different servers,
configuration file and blue print generated merge
convert the unstructured log files into traceable
logs. Traceable logs indicate the logs along with
the information about the application and server
from which the logs have been generated.
SOLUTION
 Step 1: Traverse through the blue print using a graph traversal
algorithm.
 Step 2: During traversal locate the details in the configuration file for
each application and get the log file associated with that application.
 Step 3: Open the log file. Use regular expressions in order to bring
out the essential information and append the details in the
configuration file and bring into a uniform format.
OR
Use Apache pig after appending the server and the application
information (from the configuration file) in order to bring into a
uniform format. Apache pig is high level open source data flow
engine in order to handle unstructured data.
 Step 4: Place the newly generated traceable logs into another file for
further processing.
FLOW CHART
CONFIGURATION FILE
 While traversing through Blue Print Configuration file is
taken which is in the form of key value pairs. Each
application is identified by its name or ID field that acts
as a search field to obtain information about the
application.