#
JAVA: Create a Kafka Producer with a Key
This tutorial explains to you how to create a Kafka producer which send a key to the topic. This tutorial has an example written in Java.
A nice job to do is to write messages from an application, agent, etc to a Kafka topic. In this tutorial, I will explain to you how to write a message from a Java application which contains also a key.
This key is useful because when a Kafka topic receive a key
, all the messages with the same key will go
to the same partition. All the messages in a partition are FIFO list/queues. The order is guaranteed.
After the message is sent a callback function could be called automatically, and you can get some useful information from Kafka server.
First of all you must install a Kafka server in order to test it. For this example you don't need to install a Kafka cluster, but this example works better on clusters.
So, on my environment you have some prerequisites in order to test the Java code below.
PREREQUISITES:
From the machine you run the Java client you must see the private Kafka server sockets (
hostname:IP
)You must have a Kafka topic named my-topic1 (this topic has 2 partitions !!!)
-- List all topics (prints only topic names)
kafka-topics.sh --list --zookeeper zookeeper1:2181/kafka
And now look at the code.
In pom.xml you must have :
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>2.8.0</version>
</dependency>
And the class which create the Kafka producer with key looks like this (in the picture you can see how the record is created) :
This class is almost the same as the one from a consumer without callback & key.
When you run the Java class, you must see something like this:
Info
In the partitions, each of the messages will have a specific ID attached to it, called the Offset
.
As you can see, you can read some information from Kafka server/ Kafka broker, and also you can see that the messages with the same key go to the same partition (Partition 1 in my case).
However, the topic has 2 partitions: