1 Kafka Architecture
Kafka is used for building real-time data pipelines and streaming apps. It is horizontally scalable,
fault-tolerant, wicked fast, and runs in production in thousands of companies.
Producer: the producer of messages, responsible for sending messages to the Kafka cluster;
Consumer: the message consumer, who actively pulls messages from the Kafka cluster.
Consumer Group: Each consumer belongs to a specific consumer group. When creating a new
consumer, you need to specify the corresponding consumer group ID.
Broker: A service instance in a Kafka cluster, also called a node. Each Kafka cluster contains one
or more brokers (a broker is a server or node).
Message: the object entity transmitted through Kafka cluster, which stores the information to be
transmitted.
Topic: the category of messages. It is mainly used to distinguish messages logically. Each message
sent to Kafka cluster needs to have a designated topic. Consumers consume the specified
messages according to the topic.