No, we don't have a distributed cache for rate limiters.

Is it something worth looking at? Thanks

@storozhukBM Correct me if I'm wrong, but my opinion is that Rate Limiters should be fast and not decrease response times and throughput.
A distributed cache introduces a lot of complexity and latency when caches must be synchronized or replicated.
I think the disadvantage is bigger than the advantage.

Maybe you need a centralized ratelimiter inside of a load balancer or proxy instead.

In lieu of some form of shared state for your rate limiters, what is the suggested approach for implementing rate limiting in an elastic/cloud environment? Using a single load balancer or proxy would create a single point of failure, so I don't think that's going to work for a lot of use cases.

There is no approach for the implementation of our current rate limiter.
But a better solution than rate limiters with a shared state could be adaptive capacity management.
See #201 and please have a look at the awesome talk.

@storozhukBM started with an implementation.

One way I guess it could be done is if a bunch of counters are pre generated for each node and each node gets a range and when it has exhausted it's range and goes back and gets more from the cache...

Something like this: https://apacheignite.readme.io/docs/id-generator

Do you want to implement rate limiting at client-side (source) or server-side (sink)?

If you want to have rate limiting at server-side and have a rate limit per client, you could use an API Gateway like Kong.

Resilience4j-ratelimiter is better used at client-side. A client can also be another server.
If you just want to protect the sink server from overload, I still think that adaptive capacity management (or congestion control at application layer) is a better choice than a distributed rate limiter. Thats why we are working on an initial implementation of an adaptive bulkhead implementation.

I don't think that you should use Resilience4j to reimplement an API gateway your own.

Netflix already implemented an adaptive bulkhead: https://github.com/Netflix/concurrency-limits

Totally agree with @RobWin on his thoughts.
It is better for your "elastic/cloud" distributed application to avoid any type of shared state by all means, especially if this state is shared across multiple nodes in your cluster.

For me adaptive capacity management looks like ideal solution in your case.

If it is not feasible/suitable for you for some reason, I'd recommend you to pre-calculate target throughput per node and configure it statically.

If your cluster is truly elastic and it is shrinking and growing under load it automatically means that you have some coordination solution (service discovery like consul or eureka, maybe some other type of coordination), in this case you already have some unavoidable shared state, so it would be convenient to use it to dynamically reconfigure your rate limiters on each node. In this case, cluster configuration changes it is relatively rare event of disturbance and all other time everything will work without unnecessary state sharing.

There are more scenarios I think where shared state rate limiting is a good option - for example where you don't control the target system and where your source system scales dynamically for other reasons, and where rate limit usage may not be the same between all clients in the cluster.
I've found this interesting: https://github.com/vladimir-bukhtoyarov/bucket4j/blob/master/doc-pages/jcache-usage.md and https://github.com/mokies/ratelimitj

Yes, but this would require a complete revision (overhaul) of our metric calculation and storage components.
We are open for contributions for Resilience4j 2.0.

Haha yes ok understood, but that's not what was being pointed out previously. I'm just on the hunt for such a solution and came across all this stuff, so wanted to link them together for other people to also find :)

@RobWin do you have a general idea of what components we would need to change to support this? I also came across this thread and althoguh there seem to be alternatives, they don't seem as good as resilience4j.