-
Notifications
You must be signed in to change notification settings - Fork 153
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Delay interval predictability #72
Comments
To how many rows does it grow? This plugin is often used as if RabbitMQ was a database. It is not. Beyond a certain number of delayed messages the current design is not going to work and this plugin is not a priority for our team at the moment. |
Another source of variability is the fact that this plugin relies on Erlang timers. After a certain number of long lived timers in the system they begin to compete for scheduler resources and time drift accumulates. |
We do not know how many rows, but we believe it gets to be over a million |
I'd not expect this design to support millions of delayed messages. it wasn't created for heavy duty workloads and should not be used as a secondary (leave alone primary) data store for delayed operations. One day it may be more suitable for those tasks but not today. |
Thanks. we very much appreciate your candor |
What is the best threshold for production, if I limit the growing of delayed message. |
This depends on your environment. Please ask questions on the mailing list in the future. |
I understood it's not a DB but at the same time there should be options to configure if not a clarification in doc. It'll save lots of time in capacity planning. It's a question on integrity of rabbitmq. |
We plan to re-architect this plugin. Delay predictability may or may not improve meaningfully as a result, it is an open ended question right now. The limitations of this plugin are reasonably extensively documented. |
So from what I understand, this plugin is not mature enough for heavy usage, does this mean that the previous solution which was described as: " use a mix of message TTL and Dead Letter Exchanges as proposed by James Carr here." Is more suited for production use with heavy load ? |
@rtista I don't know what "heavy usage" means to you but a combination of TTL and dead-lettering have been used in the community for years, including messaging frameworks such as NServiceBus. |
We have noticed a general issue with the reliability of delayed messages using the x-delay message header and x-delayed-type exchange. Specifically, as the size of the mnesia database grows, delayed messages become gradually more and more delayed. The delayed message plugin does not respect the delay times in any predictable way. We have some messages that delay 2000 milliseconds, some that delay 900000 milliseconds (15 minutes), and a variety in between. We are seeing messages come through the system as much as 3 days after being queued with an x-delay header of 900000 This behavior is similar both inside a 2-node production cluster, and in our test environment which consists of only a single node. We are able to work around the issue by deleting the mnesia tables, and allowing it to be recreated by the system.
The text was updated successfully, but these errors were encountered: