Agent for collecting both logs and metrics

Source
System Metrics(CPU,
memory disk)

Infrastructure
Metrics(AWS
CloudWatch)
Metrics
Sources Web tracking
METRICBEAT L
scripts(Google
Analytics)
I
Application B transform
Agents(APM, error B logstash elasticsearch
tracking)
E
System logs(syslog)
A
T
Application logs(log4j,
log4net)
Logs
Sources Server logs(Apache, FILEBEAT
MySQL)

Platform logs(AWS
CloudTrail
Metricbeat: Metricbeat is a lightweight agent that can be installed on target servers to periodically collect metric data
from various target servers and systems. It can be configured to send output to elasticsearch directly or to logstash if you
need to transform the data. The configurations related to Metricbeat are stored in a configuration file
named metricbeat.yml, and it uses YAML syntax. It is written in Go language.

Filebeat: Filebeat plays the role of the logging agent—installed on the machine generating the log files, tailing them,
and forwarding the data to either Logstash for more advanced processing or directly into Elasticsearch for indexing.
To configure Filebeat, you edit the configuration file. The default configuration file is called filebeat.yml. It is also written in
Go language.

The libbeat library, written entirely in Go, offers the API that all Beats use to ship data to Elasticsearch, configure the
input options, implement logging, and more.

Logstash: Logstash is a log aggregator that collects and processes(transforms) data from multiple sources, converts,
and ships it to stash, such as Elasticsearch. Logstash can unify data from different sources and normalize the data into
your desired destinations. It allows you to cleanse and democratize all your data for analytics and visualization of use
cases. Logstash is written using JRuby(Java and Ruby). It runs on the JVM, hence you can run Logstash on different
platforms.

ElasticSearch: The transformed data from Logstash is stored, searched, and indexed. Elasticsearch is a NoSQL
database. It is based on Lucene search engine, and it is built with RESTful APIs. It is equipped with a rich and powerful
HTTP RESTful API that enables you to perform fast searches in near real-time. Elasticsearch is developed in Java,
supporting clients in many different languages, such as Java, JavaScript(Node.js), Go, .NET(C#), PHP, Perl, Python and
Ruby.