Kafka’s Key Property Which Tells Interact of Consumers

Kafka’s architecture supports a broker for producers and consumers. The producers can be any production data, and the consumers can be data analysis systems or tools.

How Consumers share data in Kafka

The question you may face in interviews is how to say which consumers participated in sharing the data.

Consumers are receivers of data, and producers are senders of data to Kafka as input. So the output is the consumption by consumers.

Here’s Kafka’s basic architecture.

Consumer properties

The group-id plays a key role. When you group all the consumers under the same group, then the consumers can share the data.

Properties kaProperties = new Properties();
kaProperties.put("group.id", "kinaction_team0group"); 

Use cases

Let’s think of two use cases for data that come from a human resource system. One team wonders about the number of hires from specific states, and the other team is more interested in the data for the impact on travel budgets for interviews.

Would anyone on the first team care about what the other team is doing or would either of the teams want to consume only a portion of the messages? Likely not! How can we keep this separation? The answer is to assign a separate group.id to each application.

Each consumer that uses the same group.id as another consumer will be considered to be working together to consume the partitions and offsets of the topic as one logical application.

Related