PyInstaller内置的Windows EXE失败并进行多处理 [英] PyInstaller-built Windows EXE fails with multiprocessing

问题描述

在我的项目中，我正在使用Python的multiprocessing库在__main__中创建多个进程.使用PyInstaller 2.1.1将项目打包到单个Windows EXE中.

In my project I'm using Python's multiprocessing library to create multiple processes in __main__. The project is being packaged into a single Windows EXE using PyInstaller 2.1.1.

我这样创建新流程:

from multiprocessing import Process
from Queue import Empty

def _start():
    while True:
        try:
            command = queue.get_nowait()
        # ... and some more code to actually interpret commands
        except Empty:
            time.sleep(0.015)

def start():
    process = Process(target=_start, args=args)
    process.start()
    return process

在__main__中:

And in __main__:

if __name__ == '__main__':
    freeze_support()

    start()

不幸的是，当将应用程序打包到EXE中并启动它时，我在这行得到WindowsError 5或6(似乎是随机的):

Unfortunately, when packaging the application into an EXE and launching it, I get WindowsError 5 or 6 (seems random) at this line:

command = queue.get_nowait()

PyInstaller主页上的食谱声称，将应用程序打包为单个文件时，我必须修改代码以在Windows中启用多处理.

A recipe at PyInstaller's homepage claims that I have to modify my code to enable multiprocessing in Windows when packaging the application as a single file.

我在这里重现代码:

import multiprocessing.forking
import os
import sys


class _Popen(multiprocessing.forking.Popen):
    def __init__(self, *args, **kw):
        if hasattr(sys, 'frozen'):
            # We have to set original _MEIPASS2 value from sys._MEIPASS
            # to get --onefile mode working.
            # Last character is stripped in C-loader. We have to add
            # '/' or '\\' at the end.
            os.putenv('_MEIPASS2', sys._MEIPASS + os.sep)
        try:
            super(_Popen, self).__init__(*args, **kw)
        finally:
            if hasattr(sys, 'frozen'):
                # On some platforms (e.g. AIX) 'os.unsetenv()' is not
                # available. In those cases we cannot delete the variable
                # but only set it to the empty string. The bootloader
                # can handle this case.
                if hasattr(os, 'unsetenv'):
                    os.unsetenv('_MEIPASS2')
                else:
                    os.putenv('_MEIPASS2', '')


class Process(multiprocessing.Process):
    _Popen = _Popen


class SendeventProcess(Process):
    def __init__(self, resultQueue):
        self.resultQueue = resultQueue

        multiprocessing.Process.__init__(self)
        self.start()

    def run(self):
        print 'SendeventProcess'
        self.resultQueue.put((1, 2))
        print 'SendeventProcess'


if __name__ == '__main__':
    # On Windows calling this function is necessary.
    if sys.platform.startswith('win'):
        multiprocessing.freeze_support()
    print 'main'
    resultQueue = multiprocessing.Queue()
    SendeventProcess(resultQueue)
    print 'main'

我对这个解决方案"感到沮丧的是，一个，它完全不知道要修补的是什么，其二，它以令人费解的方式写成，以致无法推断出哪个部分是解决方案，以及哪个是解决方案，这让我感到沮丧.只是一个例子.

My frustration with this "solution" is that, one, it's absolutely unclear what exactly it is patching, and, two, that it's written in such a convoluted way that it becomes impossible to infer which parts are the solution, and which are just an illustration.

任何人都可以在这个问题上分享一些看法，并提供洞察力，使在PyInstaller构建的单文件Windows可执行文件中实现多处理的项目中到底需要更改什么?

Can anyone share some light on this issue, and provide insight what exactly needs to be changed in a project that enables multiprocessing in PyInstaller-built single-file Windows executables?

推荐答案

找到此PyInstaller后回答我自己的问题票证:

显然，我们要做的只是提供一个Process(和_Popen)类，如下所示，并使用它代替multiprocessing.Process.我已经纠正并简化了该类，使其仅可在Windows上使用，* ix系统可能需要不同的代码.

Apparently all we have to do is provide a Process (and _Popen) class as shown below, and use it instead of multiprocessing.Process. I've corrected and simplified the class to work on Windows only, *ix systems might need different code.

出于完整性考虑，以下是上述问题的改编示例:

For the sake of completeness, here's the adapted sample from the above question:

import multiprocessing
from Queue import Empty

class _Popen(multiprocessing.forking.Popen):
    def __init__(self, *args, **kw):
        if hasattr(sys, 'frozen'):
            os.putenv('_MEIPASS2', sys._MEIPASS)
        try:
            super(_Popen, self).__init__(*args, **kw)
        finally:
            if hasattr(sys, 'frozen'):
                os.unsetenv('_MEIPASS2')


class Process(multiprocessing.Process):
    _Popen = _Popen


def _start():
    while True:
        try:
            command = queue.get_nowait()
        # ... and some more code to actually interpret commands
        except Empty:
            time.sleep(0.015)

def start():
    process = Process(target=_start, args=args)
    process.start()
    return process

这篇关于PyInstaller内置的Windows EXE失败并进行多处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！