LMAX Disruptor模式中的慢速使用者(eventProcessor)问题的解决方案 [英] Solution to slow consumer(eventProcessor) issue in LMAX Disruptor pattern

问题描述

使用中断器时，可能会有一个或多个使用者落后，并且由于该使用者缓慢，整个应用程序都受到了影响.

请记住，每个生产者(Publisher)和使用者(EventProcessor)都在单个线程上运行，那么解决缓慢的使用者问题的方法是什么?

我们可以在一个使用者上使用多个线程吗?如果没有，什么是更好的选择?

解决方案

一般来说，使用WorkerPool允许多个池化工作线程在单个使用者上工作，如果您的任务独立且具有潜在的可变性，那么这很好持续时间(例如:一些简短的任务，一些更长的时间).

另一种选择是让多个独立的工作程序对事件进行并行处理，但是每个工作程序仅处理模N工作程序(例如2个线程，一个线程处理奇数，一个线程处理偶数事件ID).如果您具有一致的持续时间处理任务，那么此方法非常有用，并且还可以使批处理非常有效地工作.

要考虑的另一件事是，消费者可以进行批处理"，这在审核中尤其有用.如果您的使用者有10个事件等待，而不是将10个事件单独写入审核日志，则可以收集所有10个事件并同时写入它们.以我的经验，这不仅仅涉及运行多个线程的需要.

While using the disruptor, there may be a consumer(s) that is lagging behind, and because of that slow consumer, the whole application is affected.

Keeping in mind that every producer(Publisher) and consumer(EventProcessor) is running on a single thread each, what can be the solution to the slow consumer problem?

Can we use multiple threads on a single consumer? If not, what is a better alternative?

解决方案

Generally speaking use a WorkerPool to allow multiple pooled worker threads to work on a single consumer, which is good if you have tasks that are independent and of a potentially variable duration (eg: some short tasks, some longer).

The other option is to have multiple independent workers parallel process over the events, but each worker only handle modulo N workers (eg 2 threads, and one thread processes odd, one thread processes even event IDs). This works great if you have consistent duration processing tasks, and allows batching to work very efficiently too.

Another thing to consider is that the consumer can do "batching", which is especially useful for example in auditing. If your consumer has 10 events waiting, rather than write 10 events to an audit log independently, you can collect all 10 events and write them at the same time. In my experience this more than covers the need to run multiple threads.

这篇关于LMAX Disruptor模式中的慢速使用者(eventProcessor)问题的解决方案的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！