LOG ANALYTICS SOLUTION WITH APACHE HADOOP OPEN STACK

OBJECTIVE - Log Analytics is a technique for automatically understanding the

meaningful patterns from heterogeneous log data. Valuable information is stored on
log data which can be extracted and stored into Big Data platforms. By using
apache Hadoop open source technologies we can automate the process of analysing
the logs generated in real time environment.
METHODOLOGY Solution supports stream processing to aggregate and analyse
log data in order to dynamically recognize data patterns and detect outliers. The
real-time, high volume event processing for immediate action and response to
streaming is supported through an integrated hybrid architecture. This hybrid
architecture is built based upon Apache open source tool stack Nifi, Solr and
Banana. This architecture supports both streaming and batch with fast indexing
search facility.
Benefits:
1 The open stack Apache Nifi , Solr and Banana can be embed in complex
event processing architecture.
2 Capable of handing both stream and batch processing
3 To process real time data of low latency and high frequency
4 ELT strategy; Transformation of data while loading
5 Simple and Complex data flows creation
6 Alert and Monitoring mechanism at real time
7 Fast indexing and searching with low latency
8 Visualizations at Dashboard level and Search level
9 Visualizations for Trends and correlations