APACHE KAFKA

Presentation by Arta Abbasi

SAD | 2023 Zanjan University

 THE NECESSITY OF
MESSAGE BROKERS

Message brokers are crucial in distributed systems as they facilitate

seamless communication between different components, ensuring
reliability, scalability, and decoupling. They help manage
asynchronous communication, handle varying message formats,
and enable efficient data transfer between services, enhancing
system flexibility and maintainability.

1/10 1/10
 MESSAGE BROKERS
COMPARISON

Kafka
RabbitMQ
Redis

1/10 2/10
 KAFKA VS. RMQ VS. REDIS

KAFKA RMQ REDIS

Designed for
distributed streaming. Traditional message in-memory data store
queuing system
Provides persistence
and real-time data
streams.

3/10
DEMYSTIFYING

Apache Kafka, born at LinkedIn and now an

Apache Software Foundation project:
• Is a distributed streaming platform.
• Thrives on fault tolerance and high throughput.
• Excels in handling real-time data streams and
event sourcing.

4/10
KEY KAFKA
CONCEPTS

5/10
 PRACTICAL
APPLICATIONS

• Log aggregation for large-scale systems.

• Real-time stream processing for analytics and monitoring.
• Event sourcing for reconstructing data histories.
• Building data pipelines for data integration.
• IoT applications for handling massive data influx.

6/10
 KAFKA IN ACTION

1. Establish and configure a Kafka cluster.

2. Create Kafka topics that act as data channels.
3. Producers publish data to these topics.
4. Consumers subscribe to topics, consuming and processing data.
5. Implement real-time data processing and analytics.

7/10
 THE POWER OF PROTOCOL BUFFERS

Efficiency in data encoding and Language-agnostic data

transmission. interchange.

8/10
 USE CASES

• Facilitating microservices communication.

• Efficient data storage for optimized resource utilization.

• Data exchange across diverse programming languages

with ease.

9/10
 INTEGRATING PROTO
BUFFERS WITH KAFKA

1. Define your data structures in a .proto file.

2. Use the Proto Buffers compiler to generate language-specific code.
3. Serialize and deserialize messages with generated code.
4. Publish and consume Proto Buffers messages through Kafka.

10/10