Python多线程访问同一文件 [英] Python multiple threads accessing same file

问题描述

我有两个线程，一个线程写入文件，另一个线程定期 将文件移动到其他位置.写操作始终在写消息之前调用open，并在写消息之后调用close.移动器使用shutil.move进行移动.

I have two threads, one which writes to a file, and another which periodically moves the file to a different location. The writes always calls open before writing a message, and calls close after writing the message. The mover uses shutil.move to do the move.

我看到第一步完成后，编写者无法再写入文件，即，第一步之后文件的大小始终为0.我在做错什么吗?

I see that after the first move is done, the writer cannot write to the file anymore, i.e. the size of the file is always 0 after the first move. Am I doing something wrong?

推荐答案

锁定是一种可能的解决方案，但我更喜欢使用单个单独的线程处理每个外部资源(包括文件)的一般体系结构.其他线程将工作请求发送到队列的专用线程.队列实例(并在需要返回结果时提供一个单独的队列作为工作请求参数的一部分)，专用线程将大部分时间都花在该队列上的.get上以及何时得到它请求继续执行并执行(并在需要时返回传入队列中的结果).

Locking is a possible solution, but I prefer the general architecture of having each external resource (including a file) dealt with by a single, separate thread. Other threads send work requests to the dedicated thread on a Queue.Queue instance (and provide a separate queue of their own as part of the work request's parameters if they need result back), the dedicated thread spends most of its time waiting on a .get on that queue and whenever it gets a requests goes on and executes it (and returns results on the passed-in queue if needed).

我提供了这种方法的详细示例，例如在坚果中的Python"中. Python的Queue本质上是线程安全的，极大地简化了您的生活.

I've provided detailed examples of this approach e.g. in "Python in a Nutshell". Python's Queue is intrinsically thread-safe and simplifies your life enormously.

此体系结构的优点之一是可以平滑地转换为多处理如果并且当您决定将某些工作切换到单独的进程而不是单独的线程(例如，利用多个内核)时，-multiprocessing提供其自己的类似工作的Queue类型来进行这种过渡像丝绸一样光滑；-).

Among the advantages of this architecture is that it translates smoothly to multiprocessing if and when you decide to switch some work to a separate process instead of a separate thread (e.g. to take advantage of multiple cores) -- multiprocessing provides its own workalike Queue type to make such a transition smooth as silk;-).

这篇关于Python多线程访问同一文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！