有没有更好的方法阻止高吞吐量的工作吗？ [英] Is there a better way to throttle a high throughput job?

问题描述

我创建了一个简单的类，显示了我想要做的，没有任何噪音。 欢迎来砸走在我的code。这就是为什么我把它贴在这里。

 公共类节流：IDisposable的
{
    私人只读操作工作;
    私人只读Func键＆LT;布尔＆GT;停止;
    私人只读ManualResetEvent的continueProcessing;
    私人只读定时器throttleTimer;
    私人只读INT throttlePeriod;
    私人只读INT throttleLimit;
    私人诠释totalProcessed;

    公共节流阀（行动工作，Func键和LT;布尔＆GT;停下来，INT throttlePeriod，INT throttleLimit）
    {
        this.work =工作;
        this.stop =停止;
        this.throttlePeriod = throttlePeriod;
        this.throttleLimit = throttleLimit;
        continueProcessing =新的ManualResetEvent（真正的）;
        throttleTimer =新的定时器（ThrottleUpdate，空，throttlePeriod，throttlePeriod）;
    }

    公共无效的Dispose（）
    {
        throttleTimer.Dispose（）;
        （（IDisposable接口）continueProcessing）.Dispose（）;
    }

    公共无效执行（）
    {
        而（！停止（））
        {
            如果（Interlocked.Increment（REF totalProcessed）＆GT; throttleLimit）
            {
                锁定（continueProcessing）
                {
                    continueProcessing.Reset（）;
                }
                如果（！continueProcessing.WaitOne（throttlePeriod））
                {
                    抛出新TimeoutException异常（）;
                }
            }

            工作（）;
        }
    }

    私人无效ThrottleUpdate（对象状态）
    {
        Interlocked.Exchange（参考totalProcessed，0）;
        锁定（continueProcessing）
        {
            continueProcessing.Set（）;
        }
    }
}
 

最新code

 公共类节流
{
    私人只读Func键＆LT;布尔＆GT;工作;
    私人只读ThrottleSettings设置;
    私人只读秒表计时秒表;
    私人诠释totalProcessed;

    公共节流阀（Func键＆LT;布尔＆GT;的工作，ThrottleSettings设置）
    {
        this.work =工作;
        this.settings =设置;
        秒表=新的秒表（）;
    }

    私人无效执行（）
    {
        stopwatch.Start（）;
        而（工作（））
        {
            如果（++ totalProcessed＆GT; settings.Limit）
            {
                VAR的timeleft =（INT）（settings.Period  -  stopwatch.ElapsedMilliseconds）;
                如果（的timeleft大于0）
                {
                    Thread.sleep代码（的timeleft）;
                }
                totalProcessed = 0;
                stopwatch.Reset（）;
                stopwatch.Start（）;
            }
        }
    }
}
 

解决方案

首先，我要彻底摆脱控制线程的，因为它的工作可以打电话给工作之前，很容易做到（） 。

然后，我会做的工作线程从主线程不同，从而疏通执行其他任务的主线程。接下来，我想补充一个函数来取消处理，这将可能设置一个标志检查工作线程。

编辑：
根据该意见，我们的目标是限制数量工作（）在每个 throttlePeriod 蜱调用。我们可以注意到时间，秒表，在 throttleLimit 工作操作比较它，睡觉，其余时间做的更好。这样，我们又不需要一个计时器线程。

编辑：（去除，是不正确的）
编辑：
我们可以做某种平衡：是在 throttlePeriod ，我们计算多少时间做了的工作（）走，所以我们可以估算硬件多时间所有剩余的的工作（） s的要采取和每个的两个<$ C $之间等待C>的工作（）的S剩余时间相等的份额。这将使我们不是在分配的时期，有可能阻止数据库的开始执行所有的的工作（）速度非常快。

I created a simple class that shows what I am trying to do without any noise. Feel free to bash away at my code. That's why I posted it here.

public class Throttled : IDisposable
{
    private readonly Action work;
    private readonly Func<bool> stop;
    private readonly ManualResetEvent continueProcessing;
    private readonly Timer throttleTimer;
    private readonly int throttlePeriod;
    private readonly int throttleLimit;
    private int totalProcessed;

    public Throttled(Action work, Func<bool> stop, int throttlePeriod, int throttleLimit)
    {
        this.work = work;
        this.stop = stop;
        this.throttlePeriod = throttlePeriod;
        this.throttleLimit = throttleLimit;
        continueProcessing = new ManualResetEvent(true);
        throttleTimer = new Timer(ThrottleUpdate, null, throttlePeriod, throttlePeriod);
    }

    public void Dispose()
    {
        throttleTimer.Dispose();
        ((IDisposable)continueProcessing).Dispose();
    }

    public void Execute()
    {
        while (!stop())
        {
            if (Interlocked.Increment(ref totalProcessed) > throttleLimit)
            {
                lock (continueProcessing)
                {
                    continueProcessing.Reset();
                }
                if (!continueProcessing.WaitOne(throttlePeriod))
                {
                    throw new TimeoutException();
                }
            }

            work();
        }
    }

    private void ThrottleUpdate(object state)
    {
        Interlocked.Exchange(ref totalProcessed, 0);
        lock (continueProcessing)
        {
            continueProcessing.Set();
        }
    }
}

Latest Code

public class Throttled
{
    private readonly Func<bool> work;
    private readonly ThrottleSettings settings;
    private readonly Stopwatch stopwatch;
    private int totalProcessed;

    public Throttled(Func<bool> work, ThrottleSettings settings)
    {
        this.work = work;
        this.settings = settings;
        stopwatch = new Stopwatch();
    }

    private void Execute()
    {
        stopwatch.Start();
        while (work())
        {
            if (++totalProcessed > settings.Limit)
            {
                var timeLeft = (int)(settings.Period - stopwatch.ElapsedMilliseconds);
                if (timeLeft > 0)
                {
                    Thread.Sleep(timeLeft);
                }
                totalProcessed = 0;
                stopwatch.Reset();
                stopwatch.Start();
            }
        }
    }
}

解决方案

First of all, I would completely get rid of the controlling thread, because its work can be easily done before calling to work().

Then, I would make the worker thread to be different from the main thread, thus unblocking the main thread for other tasks. Next, I would add a function to cancel the processing, which would perhaps set a flag checked the worker thread.

Edit:
According to the comments, our goal is to limit number of work() calls during each throttlePeriod ticks. We can do it better by noting the time in a stopwatch, comparing it after throttleLimit work operations, and sleeping the remaining time. This way we again don't need a timer thread.

Edit: (removed, was incorrect)
Edit:
We can do even some kind of balancing: being within a throttlePeriod, we calculate how much time did the work() take, so we can estimate hw much time all the remaining work()s are going to take, and wait between each two work()s an equal share of the remaining time. This will make us not execute all the work() very fast at the beginning of the allocated period, possibly blocking the DB.

这篇关于有没有更好的方法阻止高吞吐量的工作吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！