Keras/TF CPU创建太多线程 [英] Keras/TF CPU creating too many threads
问题描述
即使在Linux机器上使用Tensorflow CPU(运行简单的CNN模型适合)设置tf.config.threading.set_inter_op_parallelism_threads(1)和tf.config.threading.set_intra_op_parallelism_threads(1) Keras之后，也会创建太多线程.无论我尝试什么，在经历合适的时期时似乎都会创建94个线程.尝试使用tf.compat.v1.ConfigProto设置进行播放，但没有任何帮助.如何限制线程数?
Even after setting tf.config.threading.set_inter_op_parallelism_threads(1) and tf.config.threading.set_intra_op_parallelism_threads(1) Keras with Tensorflow CPU (running a simple CNN model fit) on a linux machine is creating too many threads. Whatever I try it seems to be creating 94 threads while going through the fitting epochs. Have tried playing with tf.compat.v1.ConfigProto settings but nothing helps. How do I limit the number of threads?
推荐答案
此是tensorflow创建许多线程的原因.
This is why tensorflow created many threads.
使用上述两种类型的并行性(内部和内部)，您对TensorFlow生成的线程数的控制有限.通过设置这两个变量可以获得的最小线程数是N，其中N是CPU上的内核数(我不知道您是否使用gpu).
Using the mentioned 2 types of parallelism (inter and intra) you have limited control over the number of threads generated by TensorFlow. The minimum number of threads that you can get by setting these two variables is N, where N is the number of cores on your cpu (I don't know if you use gpu).
intra_op_parallelism_threads = 1
inter_op_parallelism_threads = 1
即使设置环境变量OMP_NUM_THREADS和MKL_NUM_THREADS也无助于进一步减少线程数.
Even by setting the environment variables OMP_NUM_THREADS and MKL_NUM_THREADS can't help in further reducing the number of threads.
以下讨论建议在不更改TensorFlow的源代码的情况下，不可能将线程数减少到N以下.
The following discussions suggest that without changing the source code of TensorFlow, it is not possible to reduce the number threads below N.
- 如何禁用Tensorflow的多线程?
- 如何从多线程中阻止TensorFlow
- https://github.com/tensorflow/tensorflow/issues/42510
- https://github.com/tensorflow/tensorflow/issues/33627
- How can I confine TensorFlow C API to use one and only one thread in total
- How to disable Tensorflow's multi-threading?
- How to stop TensorFlow from multi-threading
- https://github.com/tensorflow/tensorflow/issues/42510
- https://github.com/tensorflow/tensorflow/issues/33627
这篇关于Keras/TF CPU创建太多线程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
