Skip to content

Commit ce2e5f7

Browse files
committed
docs: update backpressure description
1 parent 32c9031 commit ce2e5f7

File tree

22 files changed

+78
-174
lines changed

22 files changed

+78
-174
lines changed

backpressure/README.md

+38-103
Original file line numberDiff line numberDiff line change
@@ -1,65 +1,46 @@
11
---
2-
title: "Backpressure Pattern in Java: controlling data streams from producers to consumer inorder to prevent overwhelming the consumer"
2+
title: "Backpressure Pattern in Java: Gracefully regulate producer-to-consumer data flow to prevent overload."
33
shortTitle: Backpressure
4-
description: "Explore the Backpressure design pattern in Java with detailed examples. Learn how it helps by preventing system overload, ensuring stability and optimal performance by matching data flow to the consumer’s processing capability."
4+
description: "Dive into the Backpressure design pattern in Java through practical examples, discovering how it prevents overload while ensuring stability and peak performance by aligning data flow with consumer capacity."
55
category: Concurrency
66
language: en
77
tag:
8-
- Decoupling
8+
- Asynchronous
99
- Event-driven
1010
- Reactive
11+
- Resilience
1112
---
1213

13-
## Intent of the Backpressure Design Pattern
14+
## Also known as
1415

15-
The Backpressure Design Pattern is a strategy used in software systems (especially in data streaming, reactive programming, and distributed systems)
16-
to handle situations where a fast producer overwhelms a slow consumer. The intent is to prevent system instability, resource exhaustion, or crashes by managing the flow of data between components.
16+
* Flow Control
17+
* Rate Limiting Mechanism
1718

18-
## Detailed Explanation of Backpressure Pattern with Real-World Examples
19+
## Intent of the Backpressure Design Pattern
1920

20-
### Real-world examples
21+
Control the rate of data production so downstream consumers are not overwhelmed by excessive load.
2122

22-
#### 1. Real-Time Data Streaming (Reactive Systems)
23-
- **Stock Market Data**
24-
- High-frequency trading systems generate millions of price updates per second, but analytics engines can't process them all in real time.
25-
- Backpressure mechanisms (e.g., in RxJava, Akka, Reactor) throttle or drop excess data to avoid overwhelming downstream systems.
26-
- **IoT Sensor DataQ**
27-
- Thousands of IoT devices (e.g., smart factories, wearables) send continuous telemetry, but cloud processing has limited capacity.
28-
- Reactive frameworks apply backpressure to buffer, drop, or slow down data emission.
23+
## Detailed Explanation of Backpressure Pattern with Real-World Examples
2924

30-
#### 2. Message Queues (Kafka, RabbitMQ)
31-
- **E-Commerce Order Processing**
32-
- During flash sales (e.g., Black Friday), order requests spike, but payment and inventory systems can’t keep up.
33-
- Message queues like Kafka and RabbitMQ use, Limited queue sizes to drop or reject messages when full or Consumer acknowledgments to slow producers if consumers lag.
34-
- **Log Aggregation**
35-
- Microservices generate massive logs, but centralized logging (E.g.: ELK Stack) can’t ingest them all at once.
36-
- Kafka applies backpressure by pausing producers when consumers are slow.
25+
Real-world example
3726

38-
#### 3. Stream Processing (Apache Flink, Spark)
39-
- **Social Media Trends (Twitter, TikTok)**
40-
- Viral posts create sudden spikes in data, but trend analysis is computationally expensive.
41-
- Backpressure in Spark Streaming prioritizes recent data and discards older, less relevant updates.
42-
- **Fraud Detection in Banking**
43-
- Millions of transactions flow in, but fraud detection models take time to analyze each one.
44-
- slow down ingestion if processing lags (Throttling), save progress to recover from backpressure-induced delays (Checkpointing).
27+
> Imagine a busy coffee shop where multiple baristas brew drinks (producers), and a single barista is responsible for carefully crafting latte art (consumer). If drinks are brewed faster than the latte-art barista can decorate them, they pile up, risking quality issues or discarded drinks. By introducing a pacing system—only sending new cups once the latte-art barista is ready—everyone stays synchronized, ensuring minimal waste and a consistently enjoyable customer experience.
4528
46-
### In plain words
29+
In plain words
4730

48-
The Backpressure design pattern is a flow control mechanism that prevents overwhelming a system by regulating data production based on the consumer’s processing capacity.
31+
> The Backpressure design pattern is a flow control mechanism that prevents overwhelming a system by regulating data production based on the consumer’s processing capacity.
4932
50-
### Wikipedia says
33+
Wikipedia says
5134

52-
Back pressure (or backpressure) is the term for a resistance to the desired flow of fluid through pipes. Obstructions or tight bends create backpressure via friction loss and pressure drop.
35+
> Back pressure (or backpressure) is the term for a resistance to the desired flow of fluid through pipes. Obstructions or tight bends create backpressure via friction loss and pressure drop. In distributed systems in particular event-driven architecture, back pressure is a technique to regulate flow of data, ensuring that components do not become overwhelmed.
5336
54-
In distributed systems in particular event-driven architecture, back pressure is a technique to regulate flow of data, ensuring that components do not become overwhelmed.
37+
Sequence diagram
5538

56-
### Architectural Diagram
57-
![backpressure](./etc/backpressure.png)
39+
![Backpressure sequence diagram](./etc/backpressure-sequence-diagram.png)
5840

5941
## Programmatic Example of Backpressure Pattern in Java
6042

61-
First we will create a publisher that generates a data stream.
62-
This publisher can generate a stream of integers.
43+
This example demonstrates how backpressure can be implemented using Project Reactor. We begin by creating a simple publisher that emits a stream of integers, introducing a small delay to mimic a slower production rate:
6344

6445
```java
6546
public class Publisher {
@@ -69,11 +50,7 @@ public class Publisher {
6950
}
7051
```
7152

72-
Then we can create a custom subscriber based on reactor BaseSubscriber.
73-
It will take 500ms to process one item to simulate slow processing.
74-
This subscriber will override following methods to apply backpressure on the publisher.
75-
- hookOnSubscribe method and initially request for 10 items
76-
- hookOnNext method which will process 5 items and request for 5 more items
53+
Next, we define a custom subscriber by extending Reactor’s BaseSubscriber. It simulates slow processing by sleeping for 500ms per item. Initially, the subscriber requests ten items; for every five items processed, it requests five more:
7754

7855
```java
7956
public class Subscriber extends BaseSubscriber<Integer> {
@@ -111,7 +88,7 @@ public class Subscriber extends BaseSubscriber<Integer> {
11188
}
11289
```
11390

114-
Then we can create the stream using the publisher and subscribe to that stream.
91+
Finally, in the `main` method, we publish a range of integers and subscribe using the custom subscriber. A short sleep in the main thread allows the emission, backpressure requests, and processing to be fully observed:
11592

11693
```java
11794
public static void main(String[] args) throws InterruptedException {
@@ -122,7 +99,7 @@ public static void main(String[] args) throws InterruptedException {
12299
}
123100
```
124101

125-
Program output:
102+
Below is an example of the program’s output. It shows the subscriber’s log entries, including when it requests more data and when each integer is processed:
126103

127104
```
128105
23:09:55.746 [main] DEBUG reactor.util.Loggers -- Using Slf4j logging framework
@@ -149,73 +126,31 @@ Program output:
149126
23:10:01.437 [parallel-8] INFO reactor.Flux.ConcatMapNoPrefetch.1 -- onComplete()
150127
```
151128

152-
## When to Use the Backpressure Pattern
153-
154-
- Producers Are Faster Than Consumers
155-
- If a producer generates data at a much faster rate than the consumer can handle, backpressure prevents resource overload.
156-
- Example: A server emitting events 10x faster than the client can process.
129+
## When to Use the Backpressure Pattern In Java
157130

158-
- There’s Limited Memory or Resource Capacity
159-
- Without flow control, queues or buffers can grow indefinitely, leading to out-of-memory errors or system crashes.
160-
- Example: Streaming large datasets into a low-memory microservice.
131+
* Use in Java systems where data is produced at high velocity and consumers risk overload.
132+
* Applicable in reactive or event-driven architectures to maintain stability under varying load conditions.
161133

162-
- Building Reactive or Event-Driven Architectures
163-
- Reactive systems thrive on non-blocking, asynchronous flows—and backpressure is a core component of the Reactive Streams specification.
164-
- Example: Using RxJava, Project Reactor, Akka Streams, or Node.js streams.
165-
166-
- Unpredictable Workloads
167-
- If the rate of data production or consumption can vary, backpressure helps adapt dynamically.
168-
- Example: APIs receiving unpredictable spikes in traffic.
134+
## Benefits and Trade-offs of Backpressure Pattern
169135

170-
- Need to Avoid Data Loss or Overflow
171-
- Instead of dropping data arbitrarily, backpressure lets you control flow intentionally.
172-
- Example: Video or audio processing pipelines where dropping frames is costly.
136+
Benefits:
173137

174-
## When to avoid the Backpressure Pattern
138+
* Protects consumers from saturation and resource exhaustion.
175139

176-
- For batch processing or simple linear flows with well-matched speeds.
177-
- If data loss is acceptable and simpler strategies like buffering or throttling are easier to manage.
178-
- When using fire-and-forget patterns (e.g., log shipping with retries instead of slowing the producer).
140+
Trade-offs:
179141

180-
## Benefits and Trade-offs of Backpressure Pattern
181-
182-
### Benefits:
183-
184-
- Improved System Stability
185-
- Prevents overload by controlling data flow.
186-
- Reduces chances of out-of-memory errors, thread exhaustion, or service crashes.
187-
- Efficient Resource Usage
188-
- Avoids excessive buffering and unnecessary computation.
189-
- Enables systems to do only the work they can handle.
190-
- Better Responsiveness
191-
- Keeps queues short, which improves latency and throughput.
192-
- More consistent performance under load.
193-
- Graceful Degradation
194-
- If the system can't keep up, it slows down cleanly rather than failing unpredictably.
195-
- Consumers get a chance to control the pace, leading to predictable behavior.
196-
- Fits Reactive Programming
197-
- It's essential in Reactive Streams, RxJava, Project Reactor, and Akka Streams.
198-
- Enables composing async streams safely and effectively.
199-
200-
### Trade-offs:
201-
202-
- Complexity in Debugging
203-
- Adds logic for flow control, demand signaling, and failure handling.
204-
- More state to manage (e.g., request counts, pause/resume, buffer sizes).
205-
- Harder Debugging & Testing
206-
- Asynchronous flow + demand coordination = trickier to test and debug.
207-
- Race conditions or deadlocks may occur if not handled carefully.
208-
- Potential for Bottlenecks
209-
- A slow consumer can throttle the entire system, even if other parts are fast.
210-
- Needs smart handling (e.g., buffer + drop + retry strategies).
142+
* Introduces possible delays if production must slow down to match consumer capacity.
143+
* Requires careful orchestration in complex systems with multiple concurrent data sources.
211144

212145
## Related Java Design Patterns
213-
* [Publish-Subscribe Pattern](https://github.com/sanurah/java-design-patterns/blob/master/publish-subscribe/): Pub-Sub pattern decouples producers from consumers so they can communicate without knowing about each other. Backpressure manages flow control between producer and consumer to avoid overwhelming the consumer.
214-
* [Observer Pattern](https://github.com/sanurah/java-design-patterns/blob/master/observer/): Both involve a producer (subject/publisher) notifying consumers (observers/subscribers). Observer is synchronous & tightly coupled (observers know the subject). Pub-Sub is asynchronous & decoupled (via a message broker).
215-
* [Mediator Pattern](https://github.com/sanurah/java-design-patterns/blob/master/mediator/): A mediator centralizes communication between components (like a message broker in Pub-Sub). Mediator focuses on reducing direct dependencies between objects. Pub-Sub focuses on broadcasting events to unknown subscribers.
146+
147+
* [Observer Pattern](https://java-design-patterns.com/patterns/observer/): Both patterns involve a producer notifying consumers. Observer is synchronous and tightly coupled (observers know the subject).
148+
* [Publish-Subscribe Pattern](https://java-design-patterns.com/patterns/publish-subscribe/): Both patterns deal with asynchronous data flow and can work together to manage message distribution and consumption effectively.
216149

217150
## References and Credits
218151

152+
* [Backpressure Explained (RedHat Developers Blog)](https://developers.redhat.com/articles/backpressure-explained)
153+
* [Hands-On Reactive Programming in Spring 5](https://amzn.to/3YuYfyO)
154+
* [Reactive Programming with RxJava: Creating Asynchronous, Event-Based Applications](https://amzn.to/42negbf)
155+
* [Reactive Streams in Java](https://amzn.to/3RJjUzA)
219156
* [Reactive Streams Specification](https://www.reactive-streams.org/)
220-
* [Reactive Programming with RxJava by Tomasz Nurkiewicz & Ben Christensen](https://www.oreilly.com/library/view/reactive-programming-with/9781491931646/)
221-
* [RedHat Developers Blog](https://developers.redhat.com/articles/backpressure-explained)
Loading

backpressure/pom.xml

+26
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,30 @@
11
<?xml version="1.0" encoding="UTF-8"?>
2+
<!--
3+
4+
This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
5+
6+
The MIT License
7+
Copyright © 2014-2022 Ilkka Seppälä
8+
9+
Permission is hereby granted, free of charge, to any person obtaining a copy
10+
of this software and associated documentation files (the "Software"), to deal
11+
in the Software without restriction, including without limitation the rights
12+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
13+
copies of the Software, and to permit persons to whom the Software is
14+
furnished to do so, subject to the following conditions:
15+
16+
The above copyright notice and this permission notice shall be included in
17+
all copies or substantial portions of the Software.
18+
19+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
20+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
21+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
22+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
23+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
24+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
25+
THE SOFTWARE.
26+
27+
-->
228
<project xmlns="http://maven.apache.org/POM/4.0.0"
329
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
430
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

backpressure/src/main/java/com/iluwatar/backpressure/Publisher.java

+2
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@
3030
/** This class is the publisher that generates the data stream. */
3131
public class Publisher {
3232

33+
private Publisher() {}
34+
3335
/**
3436
* On message method will trigger when the subscribed event is published.
3537
*

bloc/README.md

-4
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,7 @@ description: "Learn how the Bloc pattern helps manage state changes in Java appl
55
category: Structural
66
language: en
77
tag:
8-
- State Management
98
- Event-driven
10-
- Listener Management
11-
- Object Composition
12-
- Dynamic Behavior
139
---
1410

1511
## Also known as

clean-architecture/README.md

-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ category: Behavioral
66
language: en
77
tag:
88
- Decoupling
9-
- Architectural Style
109
---
1110

1211
## Also known as

facade/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ tag:
1010
- Code simplification
1111
- Decoupling
1212
- Encapsulation
13-
- Gang Of Four
13+
- Gang of Four
1414
- Interface
1515
- Object composition
1616
---

function-composition/README.md

-5
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,7 @@ description: "Learn about the Function Composition design pattern in Java. Disco
55
category: Functional
66
language: en
77
tag:
8-
- Code simplification
98
- Composition
10-
- Decoupling
11-
- Functional decomposition
12-
- Lambda
13-
- Reusability
149
---
1510

1611
## Also known as

map-reduce/README.md

+1-4
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,10 @@
22
title: "MapReduce Pattern in Java"
33
shortTitle: MapReduce
44
description: "Learn the MapReduce pattern in Java with real-world examples, class diagrams, and tutorials. Understand its intent, applicability, benefits, and known uses to enhance your design pattern knowledge."
5-
category: Performance optimization
5+
category: Structural
66
language: en
77
tag:
8-
- Data processing
9-
- Code simplification
108
- Delegation
11-
- Performance
129
---
1310

1411
## Also known as

mediator/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ category: Behavioral
66
language: en
77
tag:
88
- Decoupling
9-
- Gang Of Four
9+
- Gang of Four
1010
- Messaging
1111
- Object composition
1212
---

microservices-client-side-ui-composition/README.md

+2-7
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,10 @@
22
title: "Client-Side UI Composition Pattern: Assembling Modular UIs in Microservices Architecture"
33
shortTitle: Client-Side UI Composition Pattern
44
description: "Learn how the Client-Side UI Composition pattern allows the assembly of modular UIs on the client side, enabling independent teams to develop, deploy, and scale UI components in a microservices architecture. Discover the benefits, implementation examples, and best practices."
5-
category: User Interface
5+
category: Structural
66
language: en
77
tag:
8-
- Micro Frontends
9-
- API Gateway
10-
- Asynchronous Data Fetching
11-
- UI Integration
12-
- Microservices Architecture
13-
- Scalability
8+
- Asynchronous
149
---
1510

1611
## **Intent of Client-Side UI Composition Design Pattern**

microservices-distributed-tracing/README.md

+1-6
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,10 @@
22
title: "Microservices Distributed Tracing Pattern: Enhancing Visibility in Service Communication"
33
shortTitle: Distributed Tracing in Microservices
44
description: "Learn how the Distributed Tracing pattern enhances visibility into service communication across microservices. Discover its benefits, implementation examples, and best practices."
5-
category: Integration
5+
category: Structural
66
language: en
77
tag:
8-
- Distributed tracing
9-
- Microservices architecture
10-
- Service communication
11-
- Performance monitoring
128
- Scalability
13-
- Observability
149
---
1510

1611
## Intent of Microservices Distributed Tracing Design Pattern

microservices-idempotent-consumer/README.md

+1-4
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,10 @@
22
title: "Idempotent Consumer Pattern in Java: Ensuring Reliable Message Processing"
33
shortTitle: Idempotent Consumer
44
description: "Learn about the Idempotent Consumer pattern in Java. Discover how it ensures reliable and consistent message processing, even in cases of duplicate messages."
5-
category: Messaging
5+
category: Structural
66
language: en
77
tag:
8-
- Messaging
9-
- Fault tolerance
108
- Event-driven
11-
- Reliability
129
---
1310

1411
## Also known as

money/README.md

-8
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,6 @@ category: Behavioral
66
language: en
77
tag:
88
- Encapsulation
9-
- Precision handling
10-
- Currency safety
11-
- Value Object
12-
- Financial operations
13-
- Currency
14-
- Financial
15-
- Immutable
16-
- Value Object
179
---
1810

1911
## Also known as

monolithic-architecture/README.md

-4
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,6 @@ category: Architectural
66
language: en
77
tag:
88
- Cohesion
9-
- Simplicity
10-
- Scalability
11-
- Deployment
12-
- Maintainability
139
---
1410

1511
## Monolithic-Ecommerce App

observer/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ language: en
77
tag:
88
- Decoupling
99
- Event-driven
10-
- Gang Of Four
10+
- Gang of Four
1111
- Publish/subscribe
1212
---
1313

0 commit comments

Comments
 (0)