与ConcurrentDictionary< Tkey，TValue>一起使用时，C#LINQ OrderBy线程安全吗? [英] Is C# LINQ OrderBy threadsafe when used with ConcurrentDictionary<Tkey, TValue>?
问题描述
我的工作假设是，与 System.Collections.Concurrent 集合(包括 ConcurrentDictionary )一起使用时，LINQ是线程安全的.
My working assumption is that LINQ is thread-safe when used with the System.Collections.Concurrent collections (including ConcurrentDictionary).
(其他Overflow帖子似乎也同意:链接)
(Other Overflow posts seem to agree: link)
但是，对LINQ OrderBy 扩展方法的实现的检查表明，对于实现 ICollection 的并发集合的子集，它似乎不是线程安全的(例如， ConcurrentDictionary ).
However, an inspection of the implementation of the LINQ OrderBy extension method shows that it appears not to be threadsafe with the subset of concurrent collections which implement ICollection (e.g. ConcurrentDictionary).
OrderedEnumerable GetEnumerator (此处的资源)构造了 Buffer 结构(
The OrderedEnumerable GetEnumerator (source here) constructs an instance of a Buffer struct (source here) which tries to cast the collection to an ICollection (which ConcurrentDictionary implements) and then performs a collection.CopyTo with an array initialised to the size of the collection.
因此，如果在OrderBy操作期间 ConcurrentDictionary (在本例中为具体的 ICollection )的大小增加，则在初始化数组与复制到数组之间，此操作会扔.
Therefore, if the ConcurrentDictionary (as the concrete ICollection in this case) grows in size during the OrderBy operation, between initialising the array and copying into it, this operation will throw.
以下测试代码显示了此异常:
The following test code shows this exception:
(注意:我很欣赏在您下面发生变化的线程安全集合上执行 OrderBy 并不那么有意义，但我不认为它应该抛出)
(Note: I appreciate that performing an OrderBy on a thread-safe collection which is changing underneath you is not that meaningful, but I do not believe it should throw)
using System;
using System.Collections.Concurrent;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
namespace Program
{
class Program
{
static void Main(string[] args)
{
try
{
int loop = 0;
while (true) //Run many loops until exception thrown
{
Console.WriteLine($"Loop: {++loop}");
_DoConcurrentDictionaryWork().Wait();
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
private static async Task _DoConcurrentDictionaryWork()
{
var concurrentDictionary = new ConcurrentDictionary<int, object>();
var keyGenerator = new Random();
var tokenSource = new CancellationTokenSource();
var orderByTaskLoop = Task.Run(() =>
{
var token = tokenSource.Token;
while (token.IsCancellationRequested == false)
{
//Keep ordering concurrent dictionary on a loop
var orderedPairs = concurrentDictionary.OrderBy(x => x.Key).ToArray(); //THROWS EXCEPTION HERE
//...do some more work with ordered snapshot...
}
});
var updateDictTaskLoop = Task.Run(() =>
{
var token = tokenSource.Token;
while (token.IsCancellationRequested == false)
{
//keep mutating dictionary on a loop
var key = keyGenerator.Next(0, 1000);
concurrentDictionary[key] = new object();
}
});
//Wait for 1 second
await Task.Delay(TimeSpan.FromSeconds(1));
//Cancel and dispose token
tokenSource.Cancel();
tokenSource.Dispose();
//Wait for orderBy and update loops to finish (now token cancelled)
await Task.WhenAll(orderByTaskLoop, updateDictTaskLoop);
}
}
}
OrderBy 引发异常会导致一些可能的结论之一:
That the OrderBy throws an exception leads to one of a few possible conclusions:
1)我对LINQ对并发集合具有线程安全性的假设是不正确的，并且仅对在LINQ查询期间不会发生变化的集合(无论是否并发)执行LINQ是安全的.
1) My assumption about LINQ being threadsafe with concurrent collections is incorrect, and it is only safe to perform LINQ on collections (be they concurrent or not) which are not mutating during the LINQ query
2)LINQ OrderBy 的实现存在一个错误，并且尝试将源集合转换为ICollection并尝试执行集合副本是不正确的.只需遍历其默认行为(迭代IEnumerable)即可.
2) There is a bug with the implementation of LINQ OrderBy and it is incorrect for the implementation to try and cast the source collection to an ICollection and try and perform the collection copy (and It should just drop through to its default behaviour iterating the IEnumerable).
3)我误解了这里发生的事情...
3) I have misunderstood what is going on here...
非常感谢！
推荐答案
在任何地方都没有声明OrderBy(或其他LINQ方法)应始终使用源IEnumerable的GetEnumerator或在线程安全时应使用并发集合.所承诺的就是这种方法
It's not stated anywhere that OrderBy (or other LINQ methods) should always use GetEnumerator of source IEnumerable or that it should be thread safe on concurrent collections. All that is promised is this method
根据 键.
Sorts the elements of a sequence in ascending order according to a key.
在某些全局意义上，
ConcurrentDictionary也不是线程安全的.关于在其上执行的其他操作，它是线程安全的.文档还说
ConcurrentDictionary is not thread-safe in some global sense either. It's thread-safe with respect to other operations performed on it. Even more, documentation says that
ConcurrentDictionary的所有公共成员和受保护成员 是线程安全的，可以在多个线程中同时使用. 但是，成员通过以下界面之一访问 ConcurrentDictionary工具，包括扩展 方法，不能保证是线程安全的，可能需要 由呼叫者同步.
All public and protected members of ConcurrentDictionary are thread-safe and may be used concurrently from multiple threads. However, members accessed through one of the interfaces the ConcurrentDictionary implements, including extension methods, are not guaranteed to be thread safe and may need to be synchronized by the caller.
因此，您的理解是正确的(OrderBy将看到传递给它的IEnumerable实际上是ICollection，然后将获取该集合的长度，分配该大小的缓冲区，然后调用ICollection.CopyTo，然后当然，这对任何类型的集合都不是线程安全的)，但这不是OrderBy中的错误，因为OrderBy和ConcurrentDictionary都没有承诺您的假设.
So, your understanding is correct (OrderBy will see IEnumerable you pass to it is really ICollection, will then get length of that collection, allocate buffer of that size, then will call ICollection.CopyTo, and this is of course not thread safe on any type of collection), but it's not a bug in OrderBy because neither OrderBy nor ConcurrentDictionary ever promised what you assume.
如果要在ConcurrentDictionary上以线程安全的方式执行OrderBy，则需要依赖被认为是线程安全的方法.例如:
If you want to do OrderBy in a thread safe way on ConcurrentDictionary, you need to rely on methods that are promised to be thread safe. For example:
// note: this is NOT IEnumerable.ToArray()
// but public ToArray() method of ConcurrentDictionary itself
// it is guaranteed to be thread safe with respect to other operations
// on this dictionary
var snapshot = concurrentDictionary.ToArray();
// we are working on snapshot so no one other thread can modify it
// of course at this point real contents of dictionary might not be
// the same as our snapshot
var sorted = snapshot.OrderBy(c => c.Key);
如果您不想分配额外的数组(使用ToArray)，则可以使用Select(c => c)，在这种情况下它可以工作，但随后我们又陷入了困境，并依靠一些安全的方法在未答应的情况下使用(Select也不总是枚举您的集合.如果集合是数组或列表-它将使用快捷方式并使用索引器).因此，您可以创建如下扩展方法:
If you don't want to allocate additional array (with ToArray), you can use Select(c => c) and it will work in this case, but then we are again in moot territory and relying on something to be safe to use in situation it was not promised to (Select will also not always enumerate your collection. If collection is array or list - it will shortcut and use indexers instead). So you can create extension method like this:
public static class Extensions {
public static IEnumerable<T> ForceEnumerate<T>(this ICollection<T> collection) {
foreach (var item in collection)
yield return item;
}
}
如果您想安全并且不想分配数组，请像这样使用它:
And use it like this if you want to be safe and don't want to allocate array:
concurrentDictionary.ForceEnumerate().OrderBy(c => c.Key).ToArray();
在这种情况下，我们将强制执行ConcurrentDictionary的枚举(我们从文档中知道这是安全的)，然后将其传递给OrderBy，因为它知道不会对纯IEnumerable造成任何损害.请注意，正如mjwills的注释中正确指出的那样，这与ToArray并不完全相同，因为ToArray会生成快照(在构建数组时进行锁收集以防止修改)，并且Select \ yield不获取任何锁(因此，在进行枚举时可能会添加/删除项目).尽管我怀疑在执行上述问题时是否很重要-在完成OrderBy的两种情况下-您都不知道您的排序结果是否反映了收集的当前状态.
In this case we are forcing enumeration of ConcurrentDictionary (which we know is safe from documentation) and then pass that to OrderBy knowing that it cannot do any harm with that pure IEnumerable. Note that as correctly pointed out in comments by mjwills, this is not exactly the same as ToArray, because ToArray produces snapshot (locks collection preventing modifications while building array) and Select \ yield does not acquire any locks (so items might be added\removed right when enumeration is in progress). Though I doubt it matters when doing things like described in question - in both cases after OrderBy is completed - you have no idea whether your ordered results reflect current state of collection or not.
这篇关于与ConcurrentDictionary< Tkey，TValue>一起使用时，C#LINQ OrderBy线程安全吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
