Mastering Fault Tolerance with Resilience4J: Exploring Rate Limiter, Bulkhead and Retry

Introduction

When it comes to building resilient, high-performing web applications, developers need a toolbox that can handle a variety of scenarios. Two such tools offered by the Resilience4J library are the Rate Limiter and the Bulkhead. Both of these modules provide different methods of improving system stability and performance, but it’s essential to understand their differences to use them effectively. Let’s explore these two components, understand how they work, and see how we can implement them in a Spring Boot application.

Understanding Rate Limiter

The Rate Limiter module in Resilience4J is based on the token bucket algorithm, which limits the rate at which operations can be performed. In a nutshell, this means that you can specify the number of operations that can be performed in a specific period. If an operation cannot acquire a ‘token’ from the bucket (because all tokens have been used), it has to wait until a new token becomes available.

Rate limiting is crucial for preventing resource exhaustion, especially when dealing with third-party APIs with usage restrictions or when you want to prevent your own system from being overwhelmed with too many operations at once.

Understanding Bulkhead

The Bulkhead module is based on a different principle. A bulkhead is a partition in a ship that helps to prevent the entire ship from flooding if water enters one area. In a similar fashion, the Bulkhead module in Resilience4J isolates system resources, preventing a failure in one part of the system from cascading to other parts.

Bulkheads limit the number of concurrent executions, allowing systems to continue operating even under high load or failure conditions. This is important for maintaining system stability and ensuring that one malfunctioning component doesn’t bring down the entire system.

Key Differences

The primary difference between the Rate Limiter and Bulkhead lies in their approach to improving system stability. While the Rate Limiter focuses on controlling the frequency of operations, the Bulkhead is more concerned with isolating resources and preventing failures from spreading.

In other words, the Rate Limiter helps your system ‘slow down’ to prevent it from becoming overwhelmed, while the Bulkhead helps your system ‘stand strong’ by isolating failure points.

What about Retry?

Sometimes, a failing operation might succeed if retried. The Retry module provides this functionality. It can be configured to retry an operation a certain number of times or until a certain condition is met.

Here’s a simple Retry configuration:

RetryConfig config = RetryConfig.custom()
    .maxAttempts(3)
    .waitDuration(Duration.ofMillis(100))
    .build();

Retry retry = Retry.of("ServiceName", config);
CheckedRunnable retryProtectedCall = Retry
    .decorateCheckedRunnable(retry, yourRunnableFunction);
Try.of(retryProtectedCall::run)

Implementing Rate Limiter and Bulkhead with Spring Boot

To start, add the Resilience4J dependency to your pom.xml:

<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-spring-boot2</artifactId>
    <version>${resilience4j.version}</version>
</dependency>

Next, let’s implement a simple service that uses both a Rate Limiter and a Bulkhead.

@Service
public class MyService {

    private final RateLimiter rateLimiter;
    private final Bulkhead bulkhead;

    public MyService() {
        rateLimiter = RateLimiter.ofDefaults("myService");
        bulkhead = Bulkhead.ofDefaults("myService");
    }

    public String doSomething() {
        return RateLimiter.decorateSupplier(rateLimiter, Bulkhead.decorateSupplier(bulkhead, this::performOperation)).get();
    }

    private String performOperation() {
        // Your operation logic here.
    }
}

In this example, the doSomething method is decorated with both a Rate Limiter and a Bulkhead. This means that the operation is both rate-limited and isolated by a bulkhead, providing multiple layers of protection for your application.

Configuring the Rate Limiter

The Rate Limiter in Resilience4J is highly configurable. Here’s an example of a custom configuration:

RateLimiterConfig config = RateLimiterConfig.custom()
    .timeoutDuration(Duration.ofMillis(100))
    .limitRefreshPeriod(Duration.ofSeconds(1))
    .limitForPeriod(10)
    .build();

RateLimiter rateLimiter = RateLimiter.of("myService", config);

In this configuration:

• timeoutDuration is the maximum wait time a thread will wait to acquire a permission. If it cannot acquire a permission in this time, it will fail with a RequestNotPermitted exception.
• limitRefreshPeriod is the time window to refresh the permission limit.
• limitForPeriod is the number of permissions available during one limit refresh period.

Configuring the Bulkhead

The Bulkhead is also highly configurable. Here’s an example of a custom configuration:

BulkheadConfig config = BulkheadConfig.custom()
    .maxConcurrentCalls(10)
    .maxWaitDuration(Duration.ofMillis(100))
    .build();

Bulkhead bulkhead = Bulkhead.of("myService", config);

In this configuration:

• maxConcurrentCalls is the maximum number of parallel executions allowed by the bulkhead.
• maxWaitDuration is the maximum amount of time a thread should wait to enter a bulkhead. If it can’t enter within this time, it fails with a BulkheadFullException.

Applying Configurations with Spring Boot

If you’re using Spring Boot, you can apply these configurations using properties in your application.yml or application.properties file. Here’s an example for application.yml:

resilience4j:
  ratelimiter:
    instances:
      myService:
        limitForPeriod: 10
        limitRefreshPeriod: 1s
        timeoutDuration: 100ms
  bulkhead:
    instances:
      myService:
        maxConcurrentCalls: 10
        maxWaitDuration: 100ms

Combining Rate Limiter, Bulkhead, and Retry

In many practical applications, it’s common to use Rate Limiter, Bulkhead, and Retry together to enhance resilience. Here’s an example of how you might combine them:

CheckedRunnable combinedCall = RateLimiter
    .decorateCheckedRunnable(rateLimiter,
        Bulkhead.decorateCheckedRunnable(bulkhead,
            Retry.decorateCheckedRunnable(retry, yourRunnableFunction)
        )
    );
Try.of(combinedCall::run)

In this example, if a call fails, the Retry module will attempt to retry it. The Bulkhead will ensure that only a certain number of calls can be active at once, and the Rate Limiter will control the overall rate of incoming calls.

Conclusion

In conclusion, both the Rate Limiter and Bulkhead provide a range of configurations that allow you to fine-tune their behavior based on the specific requirements of your application. By understanding and correctly applying these configurations, you can greatly enhance the fault tolerance and performance of your application.