为什么 ConcurrentBag<T>.Net (4.0) 这么慢?我做错了吗? [英] Why is ConcurrentBag<T> so slow in .Net (4.0)? Am I doing it wrong?
问题描述
在我开始一个项目之前，我写了一个简单的测试来比较 ConcurrentBag from (System.Collections.Concurrent) 相对于locking & 的性能.列表.我非常惊讶 ConcurrentBag 比使用简单列表锁定慢 10 倍以上.据我所知，当读者和作者是同一个线程时， ConcurrentBag 效果最好.然而，我没想到它的性能会比传统锁差这么多.
我使用两个 Parallel for 循环对列表/包进行写入和读取进行了测试.然而，写入本身显示出巨大的差异:
private static void ConcurrentBagTest(){int collSize = 10000000;秒表 stopWatch = new Stopwatch();ConcurrentBagbag1 = new ConcurrentBag();stopWatch.Start();Parallel.For(0, collSize, delegate(int i){bag1.Add(i);});stopWatch.Stop();Console.WriteLine("已用时间 = {0}",stopWatch.Elapsed.TotalSeconds);} 在我的机器上，这需要 3-4 秒才能运行，而此代码的运行时间为 0.5 - 0.9 秒:
private static void LockCollTest(){int collSize = 10000000;object list1_lock=new object();列表lst1 = new List(collSize);秒表 stopWatch = new Stopwatch();stopWatch.Start();Parallel.For(0, collSize, delegate(int i){锁(list1_lock){lst1.Add(i);}});stopWatch.Stop();Console.WriteLine("Elapsed = {0}",stopWatch.Elapsed.TotalSeconds);} 正如我所提到的，并发读写对并发包测试没有帮助.是我做错了什么还是这个数据结构真的很慢?
- 我删除了任务，因为我在这里不需要它们(完整代码有另一个任务阅读)
非常感谢您的回答.我很难选择正确答案"，因为它似乎是几个答案的混合体.
正如 Michael Goldshteyn 指出的那样，速度确实取决于数据.Darin 指出应该有更多的竞争让 ConcurrentBag 更快，而 Parallel.For 不一定启动相同数量的线程.需要注意的一点是不要在锁内做任何您不必必须做的事情.在上述情况下，除了可能将值分配给临时变量之外，我没有看到自己在锁内做任何事情.
另外，sixlettervariables 指出碰巧运行的线程数也可能会影响结果，虽然我尝试以相反的顺序运行原始测试并且 ConcurrentBag 仍然较慢.
我在启动 15 个任务时运行了一些测试，结果取决于集合大小等.但是，对于多达 100 万次插入，ConcurrentBag 的性能几乎与锁定列表一样好或更好.超过 100 万，锁定有时似乎要快得多，但我的项目可能永远不会有更大的数据结构.这是我运行的代码:
int collSize = 1000000;object list1_lock=new object();列表lst1 = new List();ConcurrentBagconcBag = new ConcurrentBag();int numTasks = 15;int i = 0;秒表 sWatch = new Stopwatch();sWatch.Start();//首先，尝试锁Task.WaitAll(Enumerable.Range(1, numTasks).Select(x => Task.Factory.StartNew(() =>{for (i = 0; i Task.Factory.StartNew(() =>){for (i = 0; i 让我问你这个问题:你的应用程序不断添加到集合中却从不读取它，这有多现实?这样的合集有什么用?(这不是一个纯粹的修辞问题.我可以想象有一些用途，例如，您仅在关闭时(用于日志记录)或在用户请求时从集合中读取.我相信这些场景是不过相当少见.)
这就是您的代码正在模拟的内容.调用 List<T>.Add 将在所有情况下都快如闪电，但偶尔会出现列表必须调整其内部数组大小的情况；但这被所有其他快速发生的添加所抵消.因此，在这种情况下，您不太可能看到大量争用，特别是在具有例如 8 个内核的个人 PC 上进行测试(正如您在某处的评论中所说).也许您可能会在诸如 24 核机器之类的机器上看到更多争用，其中许多内核可能会同时尝试字面上添加到列表中.
争用更有可能出现在您阅读收藏的地方，尤其是.在 foreach 循环(或相当于 foreach 引擎盖下的循环的 LINQ 查询)中，需要锁定整个操作，以便您在迭代时不会修改集合.
如果你能真实地重现这个场景，我相信你会看到 ConcurrentBag<T> 的规模比你当前的测试显示的要好得多.
更新:这里是我写的一个程序，用于在场景中比较这些集合我在上面描述过(多位作者，多位读者).以 10000 个集合大小和 8 个读者线程运行 25 次试验，我得到以下结果:
<前>花了 529.0095 毫秒将 10000 个元素添加到 List很明显，这取决于您对这些集合的处理方式.
Before I started a project, I wrote a simple test to compare the performance of ConcurrentBag from (System.Collections.Concurrent) relative to locking & lists. I am extremely surprised that ConcurrentBag is over 10 times slower than locking with a simple List. From what I understand, the ConcurrentBag works best when the reader and writer is the same thread. However, I hadn't thought it's performance would be so much worse than traditional locks.
I have run a test with two Parallel for loops writing to and reading from a list/bag. However, the write by itself shows a huge difference:
private static void ConcurrentBagTest()
{
int collSize = 10000000;
Stopwatch stopWatch = new Stopwatch();
ConcurrentBag<int> bag1 = new ConcurrentBag<int>();
stopWatch.Start();
Parallel.For(0, collSize, delegate(int i)
{
bag1.Add(i);
});
stopWatch.Stop();
Console.WriteLine("Elapsed Time = {0}",
stopWatch.Elapsed.TotalSeconds);
}
On my box, this takes between 3-4 secs to run, compared to 0.5 - 0.9 secs of this code:
private static void LockCollTest()
{
int collSize = 10000000;
object list1_lock=new object();
List<int> lst1 = new List<int>(collSize);
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
Parallel.For(0, collSize, delegate(int i)
{
lock(list1_lock)
{
lst1.Add(i);
}
});
stopWatch.Stop();
Console.WriteLine("Elapsed = {0}",
stopWatch.Elapsed.TotalSeconds);
}
As I mentioned, doing concurrent reads and writes doesn't help the concurrent bag test. Am I doing something wrong or is this data structure just really slow?
[EDIT] - I removed the Tasks because I don't need them here (Full code had another task reading)
[EDIT] Thanks a lot for the answers. I am having a hard time picking "the right answer" since it seems to be a mix of a few answers.
As Michael Goldshteyn pointed out, the speed really depends on the data. Darin pointed out that there should be more contention for ConcurrentBag to be faster, and Parallel.For doesn't necessarily start the same number of threads. One point to take away is to not do anything you don't have to inside a lock. In the above case, I don't see myself doing anything inside the lock except may be assigning the value to a temp variable.
Additionally, sixlettervariables pointed out that the number of threads that happen to be running may also affect results, although I tried running the original test in reverse order and ConcurrentBag was still slower.
I ran some tests with starting 15 Tasks and the results depended on the collection size among other things. However, ConcurrentBag performed almost as well as or better than locking a list, for up to 1 million insertions. Above 1 million, locking seemed to be much faster sometimes, but I'll probably never have a larger datastructure for my project. Here's the code I ran:
int collSize = 1000000;
object list1_lock=new object();
List<int> lst1 = new List<int>();
ConcurrentBag<int> concBag = new ConcurrentBag<int>();
int numTasks = 15;
int i = 0;
Stopwatch sWatch = new Stopwatch();
sWatch.Start();
//First, try locks
Task.WaitAll(Enumerable.Range(1, numTasks)
.Select(x => Task.Factory.StartNew(() =>
{
for (i = 0; i < collSize / numTasks; i++)
{
lock (list1_lock)
{
lst1.Add(x);
}
}
})).ToArray());
sWatch.Stop();
Console.WriteLine("lock test. Elapsed = {0}",
sWatch.Elapsed.TotalSeconds);
// now try concurrentBag
sWatch.Restart();
Task.WaitAll(Enumerable.Range(1, numTasks).
Select(x => Task.Factory.StartNew(() =>
{
for (i = 0; i < collSize / numTasks; i++)
{
concBag.Add(x);
}
})).ToArray());
sWatch.Stop();
Console.WriteLine("Conc Bag test. Elapsed = {0}",
sWatch.Elapsed.TotalSeconds);
Let me ask you this: how realistic is it that you'd have an application which is constantly adding to a collection and never reading from it? What's the use of such a collection? (This is not a purely rhetorical question. I could imagine there being uses where, e.g., you only read from the collection on shutdown (for logging) or when requested by the user. I believe these scenarios are fairly rare, though.)
This is what your code is simulating. Calling List<T>.Add is going to be lightning-fast in all but the occasional case where the list has to resize its internal array; but this is smoothed out by all the other adds that happen quite quickly. So you're not likely to see a significant amount of contention in this context, especially testing on a personal PC with, e.g., even 8 cores (as you stated you have in a comment somewhere). Maybe you might see more contention on something like a 24-core machine, where many cores can be trying to add to the list literally at the same time.
Contention is much more likely to creep in where you read from your collection, esp. in foreach loops (or LINQ queries which amount to foreach loops under the hood) which require locking the entire operation so that you aren't modifying your collection while iterating over it.
If you can realistically reproduce this scenario, I believe you will see ConcurrentBag<T> scale much better than your current test is showing.
Update: Here is a program I wrote to compare these collections in the scenario I described above (multiple writers, many readers). Running 25 trials with a collection size of 10000 and 8 reader threads, I got the following results:
Took 529.0095 ms to add 10000 elements to a List<double> with 8 reader threads. Took 39.5237 ms to add 10000 elements to a ConcurrentBag<double> with 8 reader threads. Took 309.4475 ms to add 10000 elements to a List<double> with 8 reader threads. Took 81.1967 ms to add 10000 elements to a ConcurrentBag<double> with 8 reader threads. Took 228.7669 ms to add 10000 elements to a List<double> with 8 reader threads. Took 164.8376 ms to add 10000 elements to a ConcurrentBag<double> with 8 reader threads. [ ... ] Average list time: 176.072456 ms. Average bag time: 59.603656 ms.
So clearly it depends on exactly what you're doing with these collections.
这篇关于为什么 ConcurrentBag<T>.Net (4.0) 这么慢?我做错了吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
