用pyximport重新加载模块？ [英] reload module with pyximport?

问题描述

我有一个python程序，在运行之前会加载大量数据。因此，我希望能够在不重新加载数据的情况下重新加载代码。使用常规python， importlib.reload 可以正常工作。例如：

I have a python program that loads quite a bit of data before running. As such, I'd like to be able to reload code without reloading data. With regular python, importlib.reload has been working fine. Here's an example:

setup.py：

from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize

extensions = [
    Extension("foo.bar", ["foo/bar.pyx"],
              language="c++",
              extra_compile_args=["-std=c++11"],
              extra_link_args=["-std=c++11"])
]
setup(
    name="system2",
    ext_modules=cythonize(extensions, compiler_directives={'language_level' : "3"}),
)

foo / bar.py

foo/bar.py

cpdef say_hello():
    print('Hello!')

runner.py：

runner.py:

import pyximport
pyximport.install(reload_support=True)

import foo.bar
import subprocess
from importlib import reload

if __name__ == '__main__':

    def reload_bar():
        p = subprocess.Popen('python setup.py build_ext --inplace',
                             shell=True,
                             cwd='<your directory>')
        p.wait()

        reload(foo.bar)
        foo.bar.say_hello()

但这似乎不起作用。如果我编辑bar.pyx并运行 reload_bar ，则看不到我的更改。我也尝试了 pyximport.build_module（）没有运气–重建了模块但没有重新加载。我在普通 python外壳中运行，如果有区别，则不是IPython。

But this doesn't seem to work. If I edit bar.pyx and run reload_bar I don't see my changes. I also tried pyximport.build_module() with no luck -- the module rebuilt but didn't reload. I'm running in a "normal" python shell, not IPython if it makes a difference.

推荐答案

我能够适用于Python 2.x的解决方案比Python 3.x容易得多。无论出于何种原因，Cython似乎都在缓存它从中导入模块的共享对象文件（ .so ），即使在运行时重建并删除了旧文件之后，它仍然从旧的共享对象文件导入。但是，无论如何这都不是必须的（当您 import foo.bar 时，它不会创建一个），因此我们还是可以跳过这一步。

I was able to get a solution working for Python 2.x a lot easier than Python 3.x. For whatever reason, Cython seems to be caching the shareable object (.so) file it imports your module from, and even after rebuilding and deleting the old file while running, it still imports from the old shareable object file. However, this isn't necessary anyways (when you import foo.bar, it doesn't create one), so we can just skip this anyways.

最大的问题是，即使在 reload ing之后，python仍然保留了对旧模块的引用。普通的python模块似乎可以找到，但是与cython无关。为了解决这个问题，我运行了两个语句来代替 reload（foo.bar）

The largest problem was that python kept a reference to the old module, even after reloading. Normal python modules seem to work find, but not anything cython related. To fix this, I run execute two statements in place of reload(foo.bar)

del sys.modules['foo.bar']
import foo.bar

此操作成功（尽管效率可能较低）重新加载了cython模块。运行该子流程的Python 3.x中唯一存在的问题是创建有问题的可共享对象。取而代之的是，全部跳过，让 import foo.bar 与 pyximporter 模块一起发挥作用，然后重新编译为您。我还向 pyxinstall 命令添加了一个选项，以指定语言级别以匹配您在 setup.py

This successfully (though probably less efficiently) reloads the cython module. The only issue that remains in in Python 3.x running that subprocess creates a problematic shareable objects. Instead, skip that all together and let the import foo.bar work its magic with the pyximporter module, and recompile for you. I also added an option to the pyxinstall command to specify the language level to match what you've specified in the setup.py

pyximport.install(reload_support=True, language_level=3)

所以加在一起：

runner.py

import sys
import pyximport
pyximport.install(reload_support=True, language_level=3)

import foo.bar

if __name__ == '__main__':
    def reload_bar():
        del sys.modules['foo.bar']
        import foo.bar

    foo.bar.say_hello()
    input("  press enter to proceed  ")
    reload_bar()
    foo.bar.say_hello()

其他两个文件保持不变

运行：

Hello!
  press enter to proceed

-替换您好！ / code>在 foo / bar.pyx 中与 Hello world！ ，然后按 Enter 。

-replace "Hello!" in foo/bar.pyx with "Hello world!", and press Enter.

Hello world!

这篇关于用pyximport重新加载模块？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！