Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#2673: Microservice pattern: Polling publisher #3243

Open
wants to merge 20 commits into
base: master
Choose a base branch
from

Conversation

quantdevv
Copy link

Pull Request Template

What does this PR do?

This PR introduces a microservice-based architecture that includes a polling-publisher and a subscriber-service communicating via Apache Kafka. It demonstrates an in-memory data flow mechanism using Kafka topics.

The PR includes:

  • Kafka producer REST endpoint for sending messages
  • Kafka consumer that listens automatically and processes messages
  • Spring Boot configuration for both microservices
  • Initial testing and local setup for message flow validation

Fixes #2673

Type of change

  • New feature (non-breaking change)
  • Documentation update

How has this been tested?

  • Manually tested Kafka message publishing via /send endpoint
  • Verified automatic listening and processing in subscriber microservice
  • Logs and terminal output confirm end-to-end communication

Additional context

Both microservices are designed for local Kafka communication using port 9092. Make sure the Kafka broker is running before testing

Copy link

github-actions bot commented Apr 6, 2025

PR Summary

This PR introduces a microservice-based architecture with a polling publisher and subscriber service communicating via Apache Kafka. It demonstrates an in-memory data flow mechanism using Kafka topics, including a Kafka producer REST endpoint, a Kafka consumer, Spring Boot configuration for both microservices, and initial testing.

Changes

File Summary
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/App.java Updated license header in the App.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/Cart.java Updated license header in the Cart.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/CartController.java Updated license header in the CartController.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/CartRepository.java Updated license header in the CartRepository.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/InMemoryCartRepository.java Updated license header in the InMemoryCartRepository.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/InMemoryOrderRepository.java Updated license header in the InMemoryOrderRepository.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/InMemoryProductRepository.java Updated license header in the InMemoryProductRepository.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/Order.java Updated license header in the Order.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/OrderController.java Updated license header in the OrderController.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/OrderRepository.java Updated license header in the OrderRepository.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/Product.java Updated license header in the Product.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/ProductRepository.java Updated license header in the ProductRepository.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/ShoppingCartService.java Updated license header in the ShoppingCartService.java file.
clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/package-info.java Updated license header in the package-info.java file.
clean-architecture/src/test/java/com/iluwatar/cleanarchitecture/AppTest.java Updated license header in the AppTest.java file.
clean-architecture/src/test/java/com/iluwatar/cleanarchitecture/CartControllerTest.java Updated license header in the CartControllerTest.java file.
polling-publisher/README.md New file: README for the polling publisher project, describing the project structure, tech stack, and setup instructions.
polling-publisher/etc/polling-publisher.urm.puml New file: Empty plantuml file for the polling publisher.
polling-publisher/polling-service/README.md New file: README for the polling service, explaining its functionality and structure.
polling-publisher/polling-service/etc/polling-service.urm.puml New file: Empty plantuml file for the polling service.
polling-publisher/polling-service/pom.xml New file: POM file for the polling service module, defining dependencies and build configurations.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/App.java New file: Main application class for the polling service, using Spring Boot.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/DataRepository.java New file: Repository class for managing in-memory data, using a HashMap.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/DataSourceService.java New file: Service class for interacting with the data source, adding, retrieving, and removing data.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/KafkaProducer.java New file: Producer class for sending messages to Kafka, using Spring Kafka.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/PollingController.java New file: Controller class for handling REST endpoints, including a health check and a message sending endpoint.
polling-publisher/polling-service/src/main/java/com/iluwatar/polling/PollingScheduler.java New file: Scheduler class for periodically polling the data source and publishing updates to Kafka.
polling-publisher/polling-service/src/main/resources/application.yml New file: Configuration file for the polling service, specifying server port and Kafka connection details.
polling-publisher/polling-service/src/test/java/com/iluwatar/polling/AppTest.java New file: Test class for the polling service application, verifying the main method execution.
polling-publisher/polling-service/src/test/java/com/iluwatar/polling/DataRepositoryTest.java New file: Test class for the DataRepository, covering save, findById, delete, and findAll methods.

autogenerated by presubmit.ai

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Review Summary

Commits Considered (19)
  • 059f292: added polling-publisher module in pom.xml
  • 84d08f0: added scope before class definition in Test classes & removed duplicate dependency from pom.xml & added required one
  • 5e3c951: synced with remote branch
  • 24b87bb: added description on both application.yaml file
  • 4eeb011: updated all README.md
  • 5aa46ac: added new listener for other topic
  • a59362c: Fixed topic name
  • 1b3d933: Fixed subscriber-service & updated pom.xml
  • 46abbfb: Fixed Kafka bug & added a controller to send message using API
  • a7cc2c3: implemented subscriber microservice using kafkaConsumer
  • a7497c4: implemented Data repository, DataSource Service & added unittest-case & implemented polling-service microservice
  • 78f1ea9: resolved dependency & structured classes
  • 931ab84: added unit test-case for DataSourceService class
  • bb065d3: added entry point for subscriber service & unit test-case
  • 4988d2e: subscriber-service implementation
  • a2b8743: fixed checkstyle errors
  • 0051d3f: polling-publisher service implementation
  • c4e9276: added two service polling service & publisher service to handle two different operation & prepared initial skeleton
  • faf0a12: added new microservice pattern polling-publisher, created folder structure, README.md & App.java
Files Processed (30)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/App.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/Cart.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/CartController.java (2 hunks)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/CartRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/InMemoryCartRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/InMemoryOrderRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/InMemoryProductRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/Order.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/OrderController.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/OrderRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/Product.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/ProductRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/ShoppingCartService.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/package-info.java (1 hunk)
  • clean-architecture/src/test/java/com/iluwatar/cleanarchitecture/AppTest.java (1 hunk)
  • clean-architecture/src/test/java/com/iluwatar/cleanarchitecture/CartControllerTest.java (1 hunk)
  • polling-publisher/README.md (1 hunk)
  • polling-publisher/etc/polling-publisher.urm.puml (1 hunk)
  • polling-publisher/polling-service/README.md (1 hunk)
  • polling-publisher/polling-service/etc/polling-service.urm.puml (1 hunk)
  • polling-publisher/polling-service/pom.xml (1 hunk)
  • polling-publisher/polling-service/src/main/java/com/iluwatar/polling/App.java (1 hunk)
  • polling-publisher/polling-service/src/main/java/com/iluwatar/polling/DataRepository.java (1 hunk)
  • polling-publisher/polling-service/src/main/java/com/iluwatar/polling/DataSourceService.java (1 hunk)
  • polling-publisher/polling-service/src/main/java/com/iluwatar/polling/KafkaProducer.java (1 hunk)
  • polling-publisher/polling-service/src/main/java/com/iluwatar/polling/PollingController.java (1 hunk)
  • polling-publisher/polling-service/src/main/java/com/iluwatar/polling/PollingScheduler.java (1 hunk)
  • polling-publisher/polling-service/src/main/resources/application.yml (1 hunk)
  • polling-publisher/polling-service/src/test/java/com/iluwatar/polling/AppTest.java (1 hunk)
  • polling-publisher/polling-service/src/test/java/com/iluwatar/polling/DataRepositoryTest.java (1 hunk)
Actionable Comments (0)
Skipped Comments (0)

@quantdevv quantdevv changed the title Issue 2673 #2673: Microservice pattern: Polling publisher Apr 6, 2025
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Review Summary

Commits Considered (1)
  • 8c50536: Merge branch 'master' into issue-2673
Files Processed (16)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/App.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/Cart.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/CartController.java (2 hunks)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/CartRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/InMemoryCartRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/InMemoryOrderRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/InMemoryProductRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/Order.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/OrderController.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/OrderRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/Product.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/ProductRepository.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/ShoppingCartService.java (1 hunk)
  • clean-architecture/src/main/java/com/iluwatar/cleanarchitecture/package-info.java (1 hunk)
  • clean-architecture/src/test/java/com/iluwatar/cleanarchitecture/AppTest.java (1 hunk)
  • clean-architecture/src/test/java/com/iluwatar/cleanarchitecture/CartControllerTest.java (1 hunk)
Actionable Comments (0)
Skipped Comments (0)

Copy link

sonarqubecloud bot commented Apr 6, 2025

Quality Gate Failed Quality Gate failed

Failed conditions
2 Security Hotspots
64.1% Coverage on New Code (required ≥ 80%)
E Security Rating on New Code (required ≥ A)
D Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@@ -0,0 +1,47 @@
# Polling Publisher-Subscriber Microservice Pattern
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +42 to +60
// Start a separate thread to add data every 3 seconds
new Thread(
() -> {
Random random = new Random();
while (true) {
try {
Thread.sleep(3000); // Add data every 3 seconds
int id = random.nextInt(100); // Random ID
String value = "Auto-Data-" + id;
this.addData(id, value);
System.out.println("🔵 Data Added: " + id + " -> " + value);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
break;
}
}
})
.start();
}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid busy-waiting loops. Perhaps use ScheduledExecutorService or similar...

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Lombok's @slf4j for logging

Comment on lines +44 to +47
<properties>
<java.version>21</java.version>
<spring.boot.version>3.4.4</spring.boot.version>
</properties>
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove these. Use the versions from parent pom.xml.

Comment on lines +50 to +69
<!-- https://mvnrepository.com/artifact/org.junit.jupiter/junit-jupiter-api -->
<!-- <dependency>-->
<!-- <groupId>org.junit.jupiter</groupId>-->
<!-- <artifactId>junit-jupiter-api</artifactId>-->
<!-- <version>5.10.1</version>-->
<!-- <scope>test</scope>-->
<!-- </dependency>-->
<!-- <dependency>-->
<!-- <groupId>org.junit.jupiter</groupId>-->
<!-- <artifactId>junit-jupiter-engine</artifactId>-->
<!-- <version>5.9.2</version>-->
<!-- <scope>test</scope>-->
<!-- </dependency>-->
<!-- <dependency>-->
<!-- <groupId>org.springframework.boot</groupId>-->
<!-- <artifactId>spring-boot-dependencies</artifactId>-->
<!-- <version>${spring.boot.version}</version>-->
<!-- <type>pom</type>-->
<!-- <scope>import</scope>-->
<!-- </dependency>-->
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove commented code

Comment on lines +111 to +121
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-inline</artifactId>
<version>5.2.0</version>
<scope>test</scope>
</dependency>
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check if Spring Boot already contains a version of mockito...

Comment on lines +130 to +139
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>1.5.18</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.5.18</version>
</dependency>
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check if Spring Boot already contains a version of logback...

Comment on lines +145 to +165
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<!-- <artifactId>maven-assembly-plugin</artifactId>-->
<version>3.8.1</version>
<configuration>
<source>21</source>
<target>21</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
</plugin>
</plugins>
</pluginManagement>
</build>
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need these plugins specified here. They should come from the parent pom.xml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Microservice pattern: Polling publisher
2 participants