同时执行多个线程 [英] Execute multiple threads concurrently
问题描述
当前代码为:
def export_data(file):
<runs the db2 database command to export tables to file>
def export_to_files(yaml):
logger = logging.getLogger("export_to_files")
thread1 = threading.Thread(target=export_data, args=[out_file1])
thread1.start()
thread2 = threading.Thread(target=export_data, args=[out_file2])
thread2.start()
thread1.join()
thread2.join()
def main():
export_to_files()
if __name__ == "__main__":
main()
我的理解是join()
仅阻止调用线程.但是,我没有意识到thread1.join()
甚至会阻止thread2
的执行,实际上使代码只能运行1个线程,即thread1
.
My understanding was that join()
only blocks the calling thread. However, I did not realize that thread1.join()
would even block thread2
from executing, essentially making the code to only run 1 thread i.e. thread1
.
我如何同时执行两个线程,同时让主线程等待两个线程完成?
How can I execute both the threads concurrently, while have the main thread wait for both to complete?
我纠正了,这2个线程确实在运行,但是似乎只有1个线程实际上在某个时间点正在做"事情.
I stand corrected, the 2 threads do run, but it seems like only 1 thread is actually "doing" things at a point in time.
为了进一步详细说明,callable_method
正在从数据库读取数据并写入文件.现在我可以看到有2个文件正在更新(每个线程都写入一个单独的文件),但是其中一个文件已经有一段时间没有更新了,而另一个文件是最新的.
To elaborate further, the callable_method
is reading data from the database and writing to a file. While I can now see 2 files being updated(each thread writes to a separate file), one of the files is not updated for quite some time now, while the other file is up-to-date as to current time.
没有没有连接对象.查询是从db2命令行界面运行的.
There is no connection object being used. The queries are run from the db2 command line interface.
推荐答案
您可以使用很大程度上未记录在案的 ThreadPool
multiprocessing.pool
中的类,以按照以下方式进行操作:
You could use the largely undocumented ThreadPool
class in multiprocessing.pool
to do something along these lines:
from multiprocessing.pool import ThreadPool
import random
import threading
import time
MAX_THREADS = 2
print_lock = threading.Lock()
def export_data(fileName):
# simulate writing to file
runtime = random.randint(1, 10)
while runtime:
with print_lock: # prevent overlapped printing
print('[{:2d}] Writing to {}...'.format(runtime, fileName))
time.sleep(1)
runtime -= 1
def export_to_files(filenames):
pool = ThreadPool(processes=MAX_THREADS)
pool.map_async(export_data, filenames)
pool.close()
pool.join() # block until all threads exit
def main():
export_to_files(['out_file1', 'out_file2', 'out_file3'])
if __name__ == "__main__":
main()
示例输出:
[ 9] Writing to out_file1...
[ 6] Writing to out_file2...
[ 5] Writing to out_file2...
[ 8] Writing to out_file1...
[ 4] Writing to out_file2...
[ 7] Writing to out_file1...
[ 3] Writing to out_file2...
[ 6] Writing to out_file1...
[ 2] Writing to out_file2...
[ 5] Writing to out_file1...
[ 1] Writing to out_file2...
[ 4] Writing to out_file1...
[ 8] Writing to out_file3...
[ 3] Writing to out_file1...
[ 7] Writing to out_file3...
[ 2] Writing to out_file1...
[ 6] Writing to out_file3...
[ 1] Writing to out_file1...
[ 5] Writing to out_file3...
[ 4] Writing to out_file3...
[ 3] Writing to out_file3...
[ 2] Writing to out_file3...
[ 1] Writing to out_file3...
这篇关于同时执行多个线程的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!