并行不能与实体框架一起工作 [英] Parallel doesnt work with Entity Framework

问题描述

我有一个ID列表，我需要在每个ID上运行几个存储过程。

当我使用标准foreach循环时，它可以正常工作，但是当我有很多记录时，它的工作速度很慢。

我想将代码转换为使用EF，但是我遇到了一个例外：底层提供程序失败打开。

我在Parallel.ForEach中使用这段代码：

 code> using（XmlEntities osContext = new XmlEntities（））
 {
 //代码
} 
  

但它仍然引发异常。

任何想法如何使用Parallel与EF？我需要为我运行的每个程序创建一个新的上下文吗？我有大约10个程序，所以我觉得创建10个上下文是非常糟糕的，每个都有一个。

解决方案

Entity Framework正在使用的底层数据库连接是不线程安全。您将 需要为您要执行的另一个线程上的每个操作创建一个新的上下文。

您对如何并行化的关注操作是有效的;许多上下文开放和关闭将是非常昂贵的。

相反，您可能需要反转对代码并行化的思考。看来你正在循环一些项目，然后为每个项目串行调用存储过程。

如果可以，创建一个新的 任务< TResult> （或 任务 ，如果您不需要结果）对于每个过程，然后在任务< TResult> 中，打开一个单个上下文，循环遍历所有项目，然后执行存储过程。这样，你只有一些上下文相等于并行运行的存储过程的数量。

我们假设你有一个 MyDbContext 具有两个存储过程， DoSomething1 和 DoSomething2 ，两者都采用一个类， MyItem 。

实现上面的内容会像：

  //你'可能希望将其实现为IList< T>以避免
 //关于IEnumerable< T>的多次迭代的警告。 
 //你肯定*不要*希望这是一个IQueryable< T> 
 //从上下文返回。 
 IEnumerable< MyItem> items = ...; 
 
 //第一个存储过程在这里被调用。 
任务t1 = Task.Run（（）=> {
 //创建上下文
使用（var ctx = new MyDbContext（））
 //循环遍历项目
 foreach（项目中的MyItem项目）
 {
 //调用第一个存储过程
 //您当然必须在这里做项目。 
 ctx.DoSomething1（item）; 
} 
}）; 
 
 //第二个存储过程在这里被调用。 
任务t2 = Task.Run（（）=> {
 //创建上下文
使用（var ctx = new MyDbContext（））
 //循环遍历项目
 foreach（项目中的MyItem项目）
 {
 //调用第一个存储过程
 //您当然必须在这里做项目。 
 ctx.DoSomething2（item）; 
} 
}）; 
 
 //当两个任务都完成时，请执行一些操作。 
  

如果您不能并行执行存储过程（每个依赖于以某种顺序运行），那么您仍然可以并行化您的操作，这只是一个更复杂的一个。

您将看到创建自定义分区（使用静态 创建方法在 分区者 class ）。这将使您有机会获得 IEnumerator< T> 实现（注意，这是不 IEnumerable< T> ，所以你不能 foreach

对于每个 IEnumerator< T> 实例，您将返回，您将创建一个新的任务< TResult> （如果需要结果），并且在任务< TResult> 正文中，您将创建上下文，然后循环通过 IEnumerator< T> 返回的项目，按顺序调用存储过程。

像这样：

  //获取分区器。 
 OrdinalPartitioner< MyItem> partitioner = Partitioner.Create（items）; 
 
 //获取分区。 
 //你必须在这里设置分区数的参数。 
 //查看更多
 //创建策略创建自定义分区的链接。 
 IList< IEnumerator< MyItem>> paritions = partitioner.GetPartitions（
 Environment.ProcessorCount）; 
 
 //为每个分区创建一个任务。 
任务[] tasks = partitions.Select（p => Task.Run（（）=> {
 //创建上下文
使用（var ctx = new MyDbContext ）
 //记住，IEnumerator< T>实现
 //可能会实现IDisposable 
使用（p）
 //虽然p。
中有项目while（p.MoveNext（））
 {
 //获取当前项目
 MyItem current = p.Current; 
 
 //调用存储过程。处理项目
 ctx.DoSomething1（当前）; 
 ctx.DoSomething2（当前）; 
} 
}））。 
 // ToArray需要（或某些要实现列表）到
 //避免延迟执行。 
 ToArray（）; 
  

I have a list of IDs, and I need to run several stored procedures on each ID.

When I am using a standard foreach loop, it works OK, but when I have many records, it works pretty slow.

I wanted to convert the code to work with EF, but I am getting an exception: "The underlying provider failed on Open".

I am using this code, inside the Parallel.ForEach:

using (XmlEntities osContext = new XmlEntities())
{
    //The code
}

But it still throws the exception.

Any idea how can I use Parallel with EF? do I need to create a new context for every procedure I am running? I have around 10 procedures, so I think its very bad to create 10 contexts, one for each.

解决方案

The underlying database connections that the Entity Framework are using are not thread-safe. You will need to create a new context for each operation on another thread that you're going to perform.

Your concern about how to parallelize the operation is a valid one; that many contexts are going to be expensive to open and close.

Instead, you might want to invert how your thinking about parallelizing the code. It seems you're looping over a number of items and then calling the stored procedures in serial for each item.

If you can, create a new Task<TResult> (or Task, if you don't need a result) for each procedure and then in that Task<TResult>, open a single context, loop through all of the items, and then execute the stored procedure. This way, you only have a number of contexts equal to the number of stored procedures that you are running in parallel.

Let's assume you have a MyDbContext with two stored procedures, DoSomething1 and DoSomething2, both of which take an instance of a class, MyItem.

Implementing the above would look something like:

// You'd probably want to materialize this into an IList<T> to avoid
// warnings about multiple iterations of an IEnumerable<T>.
// You definitely *don't* want this to be an IQueryable<T>
// returned from a context.
IEnumerable<MyItem> items = ...;

// The first stored procedure is called here.
Task t1 = Task.Run(() => { 
    // Create the context.
    using (var ctx = new MyDbContext())
    // Cycle through each item.
    foreach (MyItem item in items)
    {
        // Call the first stored procedure.
        // You'd of course, have to do something with item here.
        ctx.DoSomething1(item);
    }
});

// The second stored procedure is called here.
Task t2 = Task.Run(() => { 
    // Create the context.
    using (var ctx = new MyDbContext())
    // Cycle through each item.
    foreach (MyItem item in items)
    {
        // Call the first stored procedure.
        // You'd of course, have to do something with item here.
        ctx.DoSomething2(item);
    }
});

// Do something when both of the tasks are done.

If you can't execute the stored procedures in parallel (each one is dependent on being run in a certain order), then you can still parallelize your operations, it's just a little more complex.

You would look at creating custom partitions across your items (using the static Create method on the Partitioner class). This will give you the means to get IEnumerator<T> implementations (note, this is not IEnumerable<T> so you can't foreach over it).

For each IEnumerator<T> instance you get back, you'd create a new Task<TResult> (if you need a result), and in the Task<TResult> body, you would create the context and then cycle through the items returned by the IEnumerator<T>, calling the stored procedures in order.

That would look like this:

// Get the partitioner.
OrdinalPartitioner<MyItem> partitioner = Partitioner.Create(items);

// Get the partitions.
// You'll have to set the parameter for the number of partitions here.
// See the link for creating custom partitions for more
// creation strategies.
IList<IEnumerator<MyItem>> paritions = partitioner.GetPartitions(
    Environment.ProcessorCount);

// Create a task for each partition.
Task[] tasks = partitions.Select(p => Task.Run(() => { 
        // Create the context.
        using (var ctx = new MyDbContext())
        // Remember, the IEnumerator<T> implementation
        // might implement IDisposable.
        using (p)
        // While there are items in p.
        while (p.MoveNext())
        {
            // Get the current item.
            MyItem current = p.Current;

            // Call the stored procedures.  Process the item
            ctx.DoSomething1(current);
            ctx.DoSomething2(current);
        }
    })).
    // ToArray is needed (or something to materialize the list) to
    // avoid deferred execution.
    ToArray();

这篇关于并行不能与实体框架一起工作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！