并行执行和以顺序方式的线程N个 [英] Executing N number of threads in parallel and in a sequential manner

问题描述

我有一个应用程序，我有1个大文件的1000+小部分。

I have an application where i have 1000+ small parts of 1 large file.

我必须在同一时间上传最多16个部分。

I have to upload maximum of 16 parts at a time.

我所使用的.NET的线程并行库。

I used Thread parallel library of .Net.

我用的Parallel.For多个部分来划分和分配1方法，应该为每个部分执行，并设置DegreeOfParallelism为16。

I used Parallel.For to divide in multiple parts and assigned 1 method which should be executed for each part and set DegreeOfParallelism to 16.

我需要由不同的部分上传生成的校验和值执行1的方法，所以我必须设置一定的机制，在那里我必须等待所有部件上传说1000来完成。
在TPL库我面临的问题，1是随机从1000执行任何16个线程的

I need to execute 1 method with checksum values which are generated by different part uploads, so i have to set certain mechanism where i have to wait for all parts upload say 1000 to complete. In TPL library i am facing 1 issue is it is randomly executing any of the 16 threads from 1000.

我想用，我可以先运行某种机制16个线程开始，如果第一或第二或任何16线程完成其任务，接下来的17部分应启动。

I want some mechanism using which i can run first 16 threads initially, if the 1st or 2nd or any of the 16 thread completes its task next 17th part should be started.

我怎样才能做到这一点？

How can i achieve this ?

推荐答案

下面是手动这样做的方式。

Here is the manual way of doing this.

您需要一个队列。队列是尚未完成的任务序列。你要出队，并把它们的内部工作任务的列表。当过任务完成从工作任务列表中删除，并采取另一种从队列中。主线程控制此过程。下面是如何做到这一点的样本。

You need a queue. The queue is sequence of pending tasks. You have to dequeue and put them inside list of working task. When ever the task is done remove it from list of working task and take another from queue. Main thread controls this process. Here is the sample of how to do this.

有关我用整数的列表，但它应该是因为它使用泛型对于其他类型的工作。测试

For the test i used List of integer but it should work for other types because its using generics.

private static void Main()
{
    Random r = new Random();
    var items = Enumerable.Range(0, 100).Select(x => r.Next(100, 200)).ToList();

    ParallelQueue(items, DoWork);
}

private static void ParallelQueue<T>(List<T> items, Action<T> action)
{
    Queue pending = new Queue(items);
    List<Task> working = new List<Task>();

    while (pending.Count + working.Count != 0)
    {
        if (pending.Count != 0 && working.Count < 16)  // Maximum tasks
        {
            var item = pending.Dequeue(); // get item from queue
            working.Add(Task.Run(() => action((T)item))); // run task
        }
        else
        {
            Task.WaitAny(working.ToArray());
            working.RemoveAll(x => x.IsCompleted); // remove finished tasks
        }
    }
}

private static void DoWork(int i) // do your work here.
{
    // this is just an example
    Task.Delay(i).Wait(); 
    Console.WriteLine(i);
}

请让我知道如果你遇到的是如何实现的DoWork为你自己的问题。因为如果你改变方法签名，你可能需要做一些改变。

Please let me know if you encounter problem of how to implement DoWork for your self. because if you change method signature you may need to do some changes.

更新

您也可以做到这一点与异步等待而不会阻塞主线程。

You can also do this with async await without blocking the main thread.

private static void Main()
{
    Random r = new Random();
    var items = Enumerable.Range(0, 100).Select(x => r.Next(100, 200)).ToList();

    Task t = ParallelQueue(items, DoWork);

    // able to do other things.

    t.Wait();
}

private static async Task ParallelQueue<T>(List<T> items, Func<T, Task> func)
{
    Queue pending = new Queue(items);
    List<Task> working = new List<Task>();

    while (pending.Count + working.Count != 0)
    {
        if (working.Count < 16 && pending.Count != 0)
        {
            var item = pending.Dequeue();
            working.Add(Task.Run(async () => await func((T)item)));
        }
        else
        {
            await Task.WhenAny(working);
            working.RemoveAll(x => x.IsCompleted);
        }
    }
}

private static async Task DoWork(int i)
{
    await Task.Delay(i);
}

这篇关于并行执行和以顺序方式的线程N个的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！