Integrating Kafka with Spring Boot 3

Introduction

Apache Kafka is a powerful distributed streaming platform that allows you to build real-time data pipelines and applications. Spring Boot 3, the latest version of the popular framework, offers a simplified approach to developing and deploying microservices. In this blog post, we will explore how to integrate Kafka with Spring Boot 3 using code examples.

Prerequisites

• Basic knowledge of Java and Spring Boot
• An installed and running Kafka cluster
• Apache Maven or Gradle for building the project

Step 1

Creating a Spring Boot Project First, create a new Spring Boot project using the Spring Initializr (https://start.spring.io/). Select the following dependencies:

• Web
• Apache Kafka

Download the generated project and extract it. Open the project in your favorite IDE.

Step 2

Configuring Kafka Properties In the src/main/resources/application.properties file, add the following Kafka configurations:

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=my-group
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

Replace localhost:9092 with your Kafka cluster address.

Step 3

Creating a Kafka Producer Create a new class called MessageProducer in the com.example.kafkademo package:

package com.example.kafkademo;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;

@Service
public class MessageProducer {

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    public void sendMessage(String topic, String message) {
        kafkaTemplate.send(topic, message);
    }
}

Step 4

Creating a Kafka Consumer Create a new class called MessageConsumer in the com.example.kafkademo package:

package com.example.kafkademo;

import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Service;

@Service
public class MessageConsumer {

    @KafkaListener(topics = "${app.topic.example}", groupId = "${spring.kafka.consumer.group-id}")
    public void listen(String message) {
        System.out.println("Received message: " + message);
    }
}

Don’t forget to add the topic name to the application.properties file:

app.topic.example=example-topic

Step 5

Creating a REST Endpoint Create a new class called MessageController in the com.example.kafkademo package:

package com.example.kafkademo;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/messages")
public class MessageController {

    @Autowired
    private MessageProducer messageProducer;

    @Value("${app.topic.example}")
    private String topicName;

    @PostMapping
    public void sendMessage(@RequestBody String message) {
        messageProducer.sendMessage(topicName, message);
    }
}

Step 6

Running the Application Now you can run your Spring Boot application, and it will start listening to messages on the example-topic. To send messages, make a POST request to http://localhost:8080/messages with a message body.

Conclusion

In this blog post, we have demonstrated how to integrate Kafka with Spring Boot 3 using a step-by-step guide with code examples. With this setup, you can now build scalable, real-time data pipelines and applications using the powerful combination of Kafka and Spring Boot. As you continue to explore these technologies, you may want to look into more advanced features such as error handling, partitioning, and securing your Kafka communication. Happy coding!