PLINQ查询给出溢出异常 [英] PLINQ query giving overflow exception

问题描述

我正在按以下方式运行PLINQ查询:

I'm running a PLINQ query as follows:

ParallelQuery<string> winningCombos = from n in nextComboMaker.GetNextCombo()
                                              .AsParallel().WithCancellation(_cancelSource.Token)
                                              where ComboWasAWinner(n) 
                                              select n;

ConcurrentBag<string> wins = new ConcurrentBag<string>();

foreach (var winningCombo in winningCombos)
{
        wins.Add(winningCombo);
        if (wins.Count == _maxWinsAllowed)
            break;
}

GetNextCombo方法仅返回字母和数字的下一个组合，其可能性高达十亿/万亿.

The GetNextCombo method is just returning the next combination of letters and numbers, with possibilities up to the billions/trillions.

现在，当我选择的组合范围大于Int32的允许大小时，这将引发异常，当组合运行的计数器为2147483584时，它将始终引发.

Now this is throwing the exception when I choose a range of combinations greater than the allowed size of an Int32, it always throws when the counter of combos it's run is 2147483584.

我通过创建每次返回的假组合来确保GetNextCombo中没有任何内容(执行yield return"234gf24fa23 ..."等)

I've made sure it's nothing in the GetNextCombo by creating a fake combo to return every time (doing a yield return "234gf24fa23..." etc.)

LINQ正在抛出异常:

The exception is being thrown by LINQ:

System.AggregateException was unhandled by user code
  Message=One or more errors occurred.
  Source=System.Core
  StackTrace:
       at System.Linq.Parallel.QueryTaskGroupState.QueryEnd(Boolean userInitiatedDispose)
       at System.Linq.Parallel.MergeExecutor`1.Execute[TKey](PartitionedStream`2 partitions, Boolean ignoreOutput, ParallelMergeOptions options, TaskScheduler taskScheduler, Boolean isOrdered, CancellationState cancellationState, Int32 queryId)
       at System.Linq.Parallel.PartitionedStreamMerger`1.Receive[TKey](PartitionedStream`2 partitionedStream)
       at System.Linq.Parallel.ForAllOperator`1.WrapPartitionedStream[TKey](PartitionedStream`2 inputStream, IPartitionedStreamRecipient`1 recipient, Boolean preferStriping, QuerySettings settings)
       at System.Linq.Parallel.UnaryQueryOperator`2.UnaryQueryOperatorResults.ChildResultsRecipient.Receive[TKey](PartitionedStream`2 inputStream)
       at System.Linq.Parallel.WhereQueryOperator`1.WrapPartitionedStream[TKey](PartitionedStream`2 inputStream, IPartitionedStreamRecipient`1 recipient, Boolean preferStriping, QuerySettings settings)
       at System.Linq.Parallel.UnaryQueryOperator`2.UnaryQueryOperatorResults.ChildResultsRecipient.Receive[TKey](PartitionedStream`2 inputStream)
       at System.Linq.Parallel.ScanQueryOperator`1.ScanEnumerableQueryOperatorResults.GivePartitionedStream(IPartitionedStreamRecipient`1 recipient)
       at System.Linq.Parallel.UnaryQueryOperator`2.UnaryQueryOperatorResults.GivePartitionedStream(IPartitionedStreamRecipient`1 recipient)
       at System.Linq.Parallel.UnaryQueryOperator`2.UnaryQueryOperatorResults.GivePartitionedStream(IPartitionedStreamRecipient`1 recipient)
       at System.Linq.Parallel.QueryOperator`1.GetOpenedEnumerator(Nullable`1 mergeOptions, Boolean suppressOrder, Boolean forEffect, QuerySettings querySettings)
       at System.Linq.Parallel.ForAllOperator`1.RunSynchronously()
       at StockWiz.Library.PLINQArrayProcessor.DoProcessing() in C:\Users\dad\Documents\BitBucket\stockwiz_clone\stockwiz\StockWiz.Library\PLINQArrayProcessor.cs:line 50
       at System.Threading.Tasks.Task.Execute()
  InnerException: System.OverflowException
       Message=Arithmetic operation resulted in an overflow.
       Source=System.Core
       StackTrace:
            at System.Linq.Parallel.PartitionedDataSource`1.ContiguousChunkLazyEnumerator.MoveNext(T& currentElement, Int32& currentKey)
            at System.Linq.Parallel.WhereQueryOperator`1.WhereQueryOperatorEnumerator`1.MoveNext(TInputOutput& currentElement, TKey& currentKey)
            at System.Linq.Parallel.ForAllOperator`1.ForAllEnumerator`1.MoveNext(TInput& currentElement, Int32& currentKey)
            at System.Linq.Parallel.ForAllSpoolingTask`2.SpoolingWork()
            at System.Linq.Parallel.SpoolingTaskBase.Work()
            at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
            at System.Threading.Tasks.Task.Execute()
       InnerException: 

我想知道是否可以做些什么来更改此查询以使其不溢出，执行事务的顺序如何等等.也许我没有让linq查询在何处进行选择，尽管我已经尝试过了:

I'm wondering if there's anything I can do to change this query to not overflow, any order in which I do things, etc. Maybe not have the linq query do a where and select, although I've tried this:

 var query = nextComboMaker.GetNextCombo().AsParallel();

 query.ForAll(x => if(ComboWasAWinner(x) wins.Add(x) );

还是一样的溢出.

推荐答案

您应该可以使用自定义分区程序解决此问题.在这种情况下，PLINQ选择的默认分区程序仅支持整数范围的项目.

You should be able to get around this by using a custom partitioner. The default partitioner chosen by PLINQ in this case only supports int-range number of items.

请参见 http://msdn.microsoft.com/en-us/library/dd997416.aspx 了解如何执行此操作.

See http://msdn.microsoft.com/en-us/library/dd997416.aspx for how to do this.

这篇关于PLINQ查询给出溢出异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！