In this article we will look at very simple basic example of Resilience4j rate limiter feature & look at runtime behavior of rate limiter. Here is the maven dependency for resilience4j-ratelimiter required for this example.
Rate Limiter Concept
Do not burden service with calls more than it can consume in given period of time. If call load is greater than service’s consumption for given period, then keep calls waiting for reasonable time until next window of period to get turn. Otherwise just timeout the calls & go for alternate recovery path. This will avoid overloading service for given period & also give a graceful way to provide alternate path under heavy load.
Example in this article
Rate Limiter Configuration: Allow 5 calls every 5 seconds. Keep other calls waiting until next 5 second window or maximum of 10 seconds overall.
- Create mock external service which takes 2 seconds to finish its processing.
- Create a service client which calls external service using rate limiter with above configurations.
- Mimic 20 parallel users/executions by calling service client in 20 threads.
- We will observe how rate limiter behaves for all threads.
Mock Service & client
Here is a mock service which takes 2 seconds to finish processing. Also a service client which decorates calls to external service using rate limiter. Rate limiter is configured to allow 5 calls every 5 seconds & keep other calls waiting until maximum of 10 seconds.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
class LowRateServiceCallerClient { private RateLimiter rateLimiter; private ExternalLowRateService externalLowRateService = new ExternalLowRateService(); public LowRateServiceCallerClient() { /* * Create rate limiter to allow 5 calls every 5 seconds & keep other calls * waiting until maximum of 10 seconds. */ RateLimiterConfig config = RateLimiterConfig.custom().limitRefreshPeriod(Duration.ofMillis(5000)) .limitForPeriod(5).timeoutDuration(Duration.ofMillis(10000)).build(); // Create registry RateLimiterRegistry rateLimiterRegistry = RateLimiterRegistry.of(config); // Use registry rateLimiter = rateLimiterRegistry.rateLimiter("externalConcurrentService"); } public void callService() { // Wrap service call in ratelimiter & call service. Runnable runnable = () -> externalLowRateService.callService(); rateLimiter.executeRunnable(runnable); } } class ExternalLowRateService { public void callService() { try { // Mock processing time of 2 seconds. Thread.sleep(2000); System.out.println(LocalTime.now() + " Call processing finished = " + Thread.currentThread().getName()); } catch (Exception e) { e.printStackTrace(); } } } |
Rate Limiter in action
Here we will call service client in 20 parallel threads which might mimic 20 parallel users or 20 parallel executions.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
public class RateLimiterBasics { public static void main(String[] args) throws InterruptedException { LowRateServiceCallerClient callerClient = new LowRateServiceCallerClient(); // Make 20 calls using service client mimicking 20 parallel users. for (int i = 0; i < 20; i++) { System.out.println(LocalTime.now() + " Starting service call = " + i); new Thread(() -> callerClient.callService(), "service-call-" + (i + 1)).start(); Thread.sleep(50); } } } |
Output
Key behavior to look for in output
- Service calls start:
- At first you will see “Starting service call” for all 20 threads.
- 5 calls in 5 seconds window & other waiting:
- Then you will see that service-call-1 till service-call-5 processing finished around 11:59:45 while other threads were still waiting.
- No other processing happens in that 5 seconds window other than above 5 calls.
- Next 5 calls in next 5 second window:
- Then you will see that service-call-6 till service-call-10 processing finished after earlier 5 seconds window is over i.e. around 11:59:50
- Wait time over for last 5 calls:
- Then you will see that service-call-16 till service-call-20 ended up with io.github.resilience4j.ratelimiter.RequestNotPermitted because 10 second wait time for them was over & they did not get chance to execute in any 5 seconds window.
- Processing finished for remaining 5 calls:
- Then you will see that service-call-11 till service-call-15 processing finished around 11:59:55 i.e. after earlier 5 seconds window was over. These logs show after above exception which means these might be under process while above calls failed.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 |
11:59:43.330 Starting service call = 0 11:59:43.381 Starting service call = 1 11:59:43.431 Starting service call = 2 11:59:43.482 Starting service call = 3 11:59:43.532 Starting service call = 4 11:59:43.582 Starting service call = 5 11:59:43.632 Starting service call = 6 11:59:43.683 Starting service call = 7 11:59:43.734 Starting service call = 8 11:59:43.785 Starting service call = 9 11:59:43.836 Starting service call = 10 11:59:43.886 Starting service call = 11 11:59:43.937 Starting service call = 12 11:59:43.988 Starting service call = 13 11:59:44.038 Starting service call = 14 11:59:44.089 Starting service call = 15 11:59:44.139 Starting service call = 16 11:59:44.190 Starting service call = 17 11:59:44.240 Starting service call = 18 11:59:44.291 Starting service call = 19 11:59:45.333 Call processing finished = service-call-1 11:59:45.381 Call processing finished = service-call-2 11:59:45.432 Call processing finished = service-call-3 11:59:45.482 Call processing finished = service-call-4 11:59:45.532 Call processing finished = service-call-5 11:59:50.292 Call processing finished = service-call-8 11:59:50.292 Call processing finished = service-call-6 11:59:50.292 Call processing finished = service-call-9 11:59:50.292 Call processing finished = service-call-7 11:59:50.292 Call processing finished = service-call-10 Exception in thread "service-call-16" io.github.resilience4j.ratelimiter.RequestNotPermitted: RateLimiter 'externalConcurrentService' does not permit further calls at io.github.resilience4j.ratelimiter.RequestNotPermitted.createRequestNotPermitted(RequestNotPermitted.java:37) at io.github.resilience4j.ratelimiter.RateLimiter.waitForPermission(RateLimiter.java:288) at io.github.resilience4j.ratelimiter.RateLimiter.lambda$decorateRunnable$8(RateLimiter.java:253) at io.github.resilience4j.ratelimiter.RateLimiter.executeRunnable(RateLimiter.java:411) at com.itsallbinary.resilience4j.tutorial.LowRateServiceCallerClient.callService(RateLimiterBasics.java:49) at com.itsallbinary.resilience4j.tutorial.RateLimiterBasics.lambda$0(RateLimiterBasics.java:19) at java.lang.Thread.run(Thread.java:745) Exception in thread "service-call-17" io.github.resilience4j.ratelimiter.RequestNotPermitted: RateLimiter 'externalConcurrentService' does not permit further calls at io.github.resilience4j.ratelimiter.RequestNotPermitted.createRequestNotPermitted(RequestNotPermitted.java:37) at io.github.resilience4j.ratelimiter.RateLimiter.waitForPermission(RateLimiter.java:288) at io.github.resilience4j.ratelimiter.RateLimiter.lambda$decorateRunnable$8(RateLimiter.java:253) at io.github.resilience4j.ratelimiter.RateLimiter.executeRunnable(RateLimiter.java:411) at com.itsallbinary.resilience4j.tutorial.LowRateServiceCallerClient.callService(RateLimiterBasics.java:49) at com.itsallbinary.resilience4j.tutorial.RateLimiterBasics.lambda$0(RateLimiterBasics.java:19) at java.lang.Thread.run(Thread.java:745) Exception in thread "service-call-18" io.github.resilience4j.ratelimiter.RequestNotPermitted: RateLimiter 'externalConcurrentService' does not permit further calls at io.github.resilience4j.ratelimiter.RequestNotPermitted.createRequestNotPermitted(RequestNotPermitted.java:37) at io.github.resilience4j.ratelimiter.RateLimiter.waitForPermission(RateLimiter.java:288) at io.github.resilience4j.ratelimiter.RateLimiter.lambda$decorateRunnable$8(RateLimiter.java:253) at io.github.resilience4j.ratelimiter.RateLimiter.executeRunnable(RateLimiter.java:411) at com.itsallbinary.resilience4j.tutorial.LowRateServiceCallerClient.callService(RateLimiterBasics.java:49) at com.itsallbinary.resilience4j.tutorial.RateLimiterBasics.lambda$0(RateLimiterBasics.java:19) at java.lang.Thread.run(Thread.java:745) Exception in thread "service-call-19" io.github.resilience4j.ratelimiter.RequestNotPermitted: RateLimiter 'externalConcurrentService' does not permit further calls at io.github.resilience4j.ratelimiter.RequestNotPermitted.createRequestNotPermitted(RequestNotPermitted.java:37) at io.github.resilience4j.ratelimiter.RateLimiter.waitForPermission(RateLimiter.java:288) at io.github.resilience4j.ratelimiter.RateLimiter.lambda$decorateRunnable$8(RateLimiter.java:253) at io.github.resilience4j.ratelimiter.RateLimiter.executeRunnable(RateLimiter.java:411) at com.itsallbinary.resilience4j.tutorial.LowRateServiceCallerClient.callService(RateLimiterBasics.java:49) at com.itsallbinary.resilience4j.tutorial.RateLimiterBasics.lambda$0(RateLimiterBasics.java:19) at java.lang.Thread.run(Thread.java:745) Exception in thread "service-call-20" io.github.resilience4j.ratelimiter.RequestNotPermitted: RateLimiter 'externalConcurrentService' does not permit further calls at io.github.resilience4j.ratelimiter.RequestNotPermitted.createRequestNotPermitted(RequestNotPermitted.java:37) at io.github.resilience4j.ratelimiter.RateLimiter.waitForPermission(RateLimiter.java:288) at io.github.resilience4j.ratelimiter.RateLimiter.lambda$decorateRunnable$8(RateLimiter.java:253) at io.github.resilience4j.ratelimiter.RateLimiter.executeRunnable(RateLimiter.java:411) at com.itsallbinary.resilience4j.tutorial.LowRateServiceCallerClient.callService(RateLimiterBasics.java:49) at com.itsallbinary.resilience4j.tutorial.RateLimiterBasics.lambda$0(RateLimiterBasics.java:19) at java.lang.Thread.run(Thread.java:745) 11:59:55.291 Call processing finished = service-call-11 11:59:55.291 Call processing finished = service-call-14 11:59:55.292 Call processing finished = service-call-12 11:59:55.292 Call processing finished = service-call-13 11:59:55.292 Call processing finished = service-call-15 |
Further reading
Resilience4j Complete Tutorial | Basics with runtime behavior | Simple examples for beginners
When using the rateLimiterRegistry, the name of the service should be “ExternalLowRateService” instead of “externalConcurrentService”.