SortedSet的/排序列表具有更好的LINQ的性能？ [英] SortedSet / SortedList with better LINQ performance?

问题描述

让我们说我们有一个排序的集合，如的SortedSet 或< A HREF =htt​​p://msdn.microsoft.com/en-us/library/system.collections.sortedlist.aspx相对=nofollow>排序列表很多（10M +）元素。很多查询的正在发生的事情，所以性能问题。从运行时间的比较，我在IM pression的LINQ到对象不采取排序的优势，因此不考虑潜在的性能提升优势。

Let's say we have a sorted collection such as SortedSet or SortedList with many (10M+) elements. Lots of querying is happening, so performance matters. From runtime comparisons, I'm under the impression that LINQ to Objects doesn't take advantage of the sorting, therefore not taking advantage of potential performance gains.

第一个例子 - 在计算中的元素范围：

First example - counting the elements in a range:

        var mySortedSet1 = new SortedSet<int>();
        // populate ...
        int rangeCount = (from n in mySortedSet1
                          where ((n >= 1000000000) && (n <= 2000000000))
                          select n).Count();

不完全知道LINQ到对象在这里所做的内部，最坏的情况下，它检查这将是O（n）的每一个元素。的可以通过取与在O（log n）的二分搜索的下限和上限分选的优点进行快很多。

Not exactly sure what LINQ to Objects does here internally, worst case it's checking every single element which would be O(n). The can be done a lot faster by taking advantage of the sorting with a binary search for the lower and upper bound in O(log n).

第二个例子 - 在的SelectMany的集列表：

Second example - SelectMany over list of sets:

        var myListOfSortedSets = new List<SortedSet<int>>();
        // populate...

        var q = myListOfSortedSets.SelectMany(s => s).OrderBy(s => s);
        foreach (var n in q)
        {
            Console.WriteLine(n);
        }

如果LINQ到<打击> SQL 对象分别采取排序的优势，它可以有效地拉链合并所有的有序集合到一个大的分类列表中为O（n）。该.OrderBy的结果可能再被忽略，因为名单已经排序。

If LINQ to SQL Objects were to take advantage of the sorting, it could effectively zipper-merge all the sorted sets into one large sorted list in O(n). The .OrderBy on the result could then be ignored as the list is already sorted.

相反，的SelectMany将所有的有序集合为（N log n）的排序，这将需要另一o一个大的（现在的无序）列表。这可以很容易通过除去.OrderBy并观察在该元素被写入到控制台的顺序进行验证。

Instead, SelectMany concatenates all the sorted sets into one large (now unsorted) list which will required another O(n log n) sort. This can easily be verified by removing the .OrderBy and observing the order in which the elements are written to the console.

我的问题是：是已经有一个替代，更有效的实现LINQ到SortedSet的/排序列表那里

i4o 看起来很有趣，但它似乎需要二级索引集合，以提高查询性能原来的集合。我只是想查询我的分类集合更快地采取排序的优势运行。

i4o looks very interesting, but it seems to require secondary index collections to improve query performance on the original collection. I just want queries on my sorted collections to run faster by taking advantage of the sorting.

推荐答案

有LINQ的问题是，它无法知道有序集合是有序的完全相同的方式作为查询的期望。 T＆GT; <因为任何有序集合可以用的IComparer / IComparable的 / 比较＆LT创建/ code>，也没有明知＆GT; 500000 实际上是有意义的。也许你已经得到了第一个由各种奇/偶，然后通过数字上的比较器的自定义方法。在这种情况下，为了将完全搞砸了和O（n）被要求在所有情况下。

The problem for LINQ is that it can't know the sorted set is ordered exactly the same way as the query expects. Since any ordered collection can be created with an IComparer / IComparable / Comparison<T>, there is no knowing that > 500000 actually makes sense. Maybe you've got a custom method on the comparer that first sorts by Odd/Even, then by number. In which case the order would be completely messed up and O(n) is required in all cases.

因此​​为了安全起见，LINQ需要遍历集合中的所有元素，即使是按某种方式。默认。凡的实施不包含优化有序集合。

So to be on the safe side, LINQ will need to iterate through all elements in the Collection, even when it is sorted in some way. The default .Where implementation does not contain an optimization for ordered collections.

这也许可以创建一个保持现有订货记在迭代优化的版本，但是这将是非常困难的事，并使其在所有情况下工作。

It might be possible to create an optimized version which keeps the existing ordering in mind while iterating, but it will be very difficult to do and to make it work in all cases.

您可以创建一个使用的SortedSet GetViewBetween 方法在方法code>返回一个新的pre-有序集合。或者会增加标准。凡因为你通常会为任何非pre-有序set。

You could create a Between method that uses the GetViewBetween method of SortedSet to return a new pre-ordered collection. Or would add the standard .Where as you'd normally would for any non-pre-sorted set.

LINQ到SQL和Entity Framework的充分利用，如果IQueryable的和实际上将翻译你的LINQ查询到SQL，并让服务器处理索引，排序，筛选等。

Linq-to-SQL and Entity Framework make use if the IQueryable and will actually translate your Linq query to SQL and let the server handle the indexing, sorting, filtering etc.

这篇关于SortedSet的/排序列表具有更好的LINQ的性能？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！