多线程：在什么时候你创建的线程太多？ [英] Multi-Threading: At what point have you created too many threads?

问题描述

我工作的一个多线程应用程序。

I am working on a multi-threaded application.

该应用程序开始作为一个单独的线程，并扩展到多线程以实现性能增益。

This application started out as a single thread and was expanded to multiple threads in order to realize a gain in performance.

我有一个划分了工作成小块，并将其卸载到处理块工作线程的主线程。该部分是利用一个信号，以允许工作线程在任何一个时间只有X号控制。工作线程生成的数据块其然后存储在队列或环形缓冲器，然后由一个储蓄线程读取。这个线程负责保存数据的组块到盘（有时通过本地网络）。

I have a main thread which divides up the work into smaller chunks and offloads it to worker threads which process the chunks. This portion is controlled using a semaphore to allow only X number of worker threads at any one time. The worker threads produce chunks of data which are then stored in a queue or ring buffer which is then read by one saving thread. This thread is responsible for saving the chunks of data to the disk (sometimes across the local network).

我的机器是四核8GB的RAM。跑在我的机器上的应用与3个工作线程和1金丹线程结果数据通过网络与该处理器被利用到平均75％的稳定流动。

My development machine is a Quad Core with 8GB of RAM. Running the application on my machine with 3 worker threads and 1 saver thread results in a steady flow of data over the network with the processors being utilized to an average 75%.

攻击这个问题的第二种方法是我另增加一组线程的工作线程和金丹线之间（即取一个任务了当前工作线程，并把它添加到另一个线程）（我还添加了一个队列对于每个线程）的应用程序似乎并没有得到任何速度我的机器上似乎有太多的资源争用RAM总线饱和度和处理器争用。

The second method of attacking this problem is where I add another set of threads between the worker threads and the saver thread (i.e. taking one task out of the current worker thread and add it to another thread) (I also add a queue for each of these threads) the application does not seem to gain any speed on my machine as there seems to be too much contention for resources RAM bus saturation and processor contention.

通过多次实验与线程和优先级的号，我已经找到了理想的设置，我的机器来说，无论是第一和第二处理这个问题的方法。现在生产的机器将有8个内核和64GB的RAM。一个非常不同的环境和应用程序将对它进行配置。

Through much experimentation with the number of threads and their priorities, I have found the ideal settings for my machine, for both the first and second methods of approaching this problem. Now the production machine will have 8 cores and 64GB of RAM. A much different environment and the application will have to be configured for it.

我的问题是，在什么时候你创建的线程太多？是否总是尝试，以确定给定机器的理想设置的问题？有determing或观察，如果锁定是从应用考虑太多了的方法？

My question is, At what point have you created too many threads? Is it always a matter of experimenting to determine the ideal settings for a given machine? Is there a method of determing or observing if locking is taking too much away from the application?

（我不使用一个线程池，因为它不适合我与信号量和其他锁机制进行管理长时间运行的线程需求。）

(I'm not using a thread-pool because it does not fit my needs with long running threads being managed by semaphores and other locking mechanisms.)

推荐答案

您已经创建的线程太多的应用程序降低了整体性能还是在同一机器上运行的其他应用程序的影响都是负面影响到不可接受的水平

You've created too many threads when the overall performance of your application degrades or the impact on other applications running on that same box are negatively affected to an unacceptable level.

的要点是，没有绝对的答案。

The point is that there is no absolute answer.

一个应用程序，我已经工作了1000个线程的线程池，并为我们在做什么，这似乎是正确的数字。在一个配置中，我们并没有限制它，它上升到30,000，基本上把机器戛然而止。

One application I've been working on uses a thread pool of 1000 threads and for what we're doing, that seems to be the right number. In one configuration we didn't limit it and it went up to 30,000+ and basically brought the machine to a grinding halt.

您基本上都要性能测试，并有足够的监测/仪器来确定应用程序的总吞吐量，资源使用，线程利用率和知道如何空闲线程是多长时间的工作在等待队列被拾起。然后，调整是必要的。

You basically have to performance test it and have enough monitoring/instrumentation to determine the overall throughput of your application, resource usage, thread utilization and know how idle threads were and how long work was waiting on queues to be picked up. You then tune as necessary.

一个谨慎注意：非常仔细地添加线程另一层之前。正如我敢肯定你知道，编写多线程code是困难的。尽量保持尽可能简单。添加另一层是一种危险的一步。

One cautionary note: think very carefully before you add another layer of threads. As I'm sure you know, writing multithreaded code is hard. Try to keep it as simple as possible. Adding another layer is a risky step.

这篇关于多线程：在什么时候你创建的线程太多？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！