Python 3.6 项目结构导致 RuntimeWarning [英] Python 3.6 project structure leads to RuntimeWarning

问题描述

我正在尝试打包我的项目以进行分发，但是我在运行模块时遇到了RuntimeWarning.

I'm trying to package up my project for distribution, but I'm hitting a RuntimeWarning when I run the module.

我在 Python 邮件列表 上发现了一个错误报告，表明 RuntimeWarning 是 Python 3.5.2 中引入的新行为.

I've found a bug report on the Python mailing list which indicates that the RuntimeWarning is new behaviour that was introduced in Python 3.5.2.

阅读错误报告，似乎发生了双重导入，并且此 RuntimeWarning 在提醒用户方面是正确的.但是，我看不出需要对自己的项目结构进行哪些更改才能避免此问题.

Reading through the bug report, it appears that there is a double-import which happens, and this RuntimeWarning is correct in alerting the user. However, I don't see what changes that I need to make to my own project structure to avoid this issue.

这是我尝试正确"构建的第一个项目.想要代码推送时布局整洁，项目结构可以方便别人克隆和运行.

This is the first project that I have attempted to structure "correctly". I would like to have a tidy layout for when I push the code, and a project structure which can be cloned and run easily by others.

我的结构主要基于 http://docs.python-guide.org/en/latest/writing/structure/.

我在下面添加了最小工作示例的详细信息.

I have added details of a minimum working example below.

为了重现这个问题，我使用 python -m 运行主文件:

To replicate the issue, I run the main file with python -m:

(py36) X:\test_proj>python -m proj.proj
C:\Users\Matthew\Anaconda\envs\py36\lib\runpy.py:125: RuntimeWarning: 
'proj.proj' found in sys.modules after import of package 'proj', but prior 
to execution of 'proj.proj'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
This is a test project.`

运行我的测试没问题:

(py36) X:\test_proj>python -m unittest tests.test_proj
This is a test project.
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

复制该问题的项目结构如下:

A project structure to replicate the issue is as follows:

myproject/
    proj/
        __init__.py
        proj.py
    tests/
        __init__.py
        context.py
        test_proj.py

在文件proj/proj.py中:

def main():
    print('This is a test project.')
    raise ValueError

if __name__ == '__main__':
    main()

在 proj/__init__.py 中:

from .proj import main

在 tests/context.py 中:

import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
import proj

最后，在tests/test_proj.py:

import unittest

from .context import proj


class SampleTestCase(unittest.TestCase):
    """Test case for this sample project"""
    def test_raise_error(self):
        """Test that we correctly raise an error."""
        with self.assertRaises(ValueError):
            proj.main()


if __name__ == '__main__':
    unittest.main()

谁能帮我纠正我的项目结构以避免这种双重导入的情况?对此的任何帮助将不胜感激.

Can anyone help me correct my project structure to avoid this double-import scenario? Any help with this would be greatly appreciated.

推荐答案

对于这种特殊情况，双重导入警告是由于 proj/__init__.py 中的这一行:

For this particular case, the double import warning is due to this line in proj/__init__.py:

from .proj import main

该行的意思是当 -m 开关实现完成 import proj 步骤时，proj.proj 已经 已经作为导入父包的副作用而被导入.

What that line means is that by the time the -m switch implementation finishes the import proj step, proj.proj has already been imported as a side effect of importing the parent package.

避免警告

为避免警告，您需要找到一种方法来确保导入父包不会隐式导入使用 -m 开关执行的包.

To avoid the warning, you need to find a way to ensure that importing the parent package doesn't implicitly import the package being executed with the -m switch.

解决这些问题的两个主要选项是:

The two main options for resolving that are:

1. 删除 from .proj import main 行(如@John Moutafis 建议的那样)，假设可以在不破坏 API 兼容性保证的情况下完成；或

2. 从 proj 子模块中删除 if __name__ == "__main__": 块并将其替换为单独的 proj/__main__.py 文件:

1. Drop the from .proj import main line (as @John Moutafis suggested), assuming that can be done without breaking API compatibility guarantees; or

2. Delete the if __name__ == "__main__": block from the proj submodule and replace it with a separate proj/__main__.py file that just does:

from .proj import main
main()

如果您选择选项 2，那么命令行调用也将更改为 python -m proj，而不是引用子模块.

If you go with option 2, then the command line invocation would also change to just be python -m proj, rather than referencing a submodule.

选项 2 的一个向后兼容的变体是添加 __main__.py 而不从当前子模块中删除 CLI 块，当与 DeprecationWarning<结合使用时，这可能是一种特别好的方法/代码>:

A more backwards compatible variant of option 2 is to add __main__.py without deleting the CLI block from the current submodule, and that can be an especially good approach when combined with DeprecationWarning:

if __name__ == "__main__":
    import warnings
    warnings.warn("use 'python -m proj', not 'python -m proj.proj'", DeprecationWarning)
    main()

如果 proj/__main__.py 已经被用于其他目的，那么你也可以用 替换 python -m proj.projpython -m proj.proj_cli，其中 proj/proj_cli.py 看起来像:

If proj/__main__.py is already being used for some other purpose, then you can also do things like replacing python -m proj.proj with python -m proj.proj_cli, where proj/proj_cli.py looks like:

if __name__ != "__main__":
    raise RuntimeError("Only for use with the -m switch, not as a Python API")
from .proj import main
main()

为什么会出现警告?

当 -m 开关实现即将运行并在 __main__ 模块中再次 运行已导入模块的代码时，会发出此警告，这意味着您将拥有它定义的所有内容的两个不同副本 - 类、函数、容器等.

This warning gets emitted when the -m switch implementation is about to go and run an already imported module's code again in the __main__ module, which means you will have two distinct copies of everything it defines - classes, functions, containers, etc.

根据应用程序的具体情况，这可能工作正常(这就是为什么它是警告而不是错误的原因)，或者它可能导致奇怪的行为，例如模块级状态修改未按预期共享，甚至异常未共享被捕获是因为异常处理程序试图从模块的一个实例中捕获异常类型，而引发的异常使用了另一个实例中的类型.

Depending on the specifics of the application, this may work fine (which is why it's a warning rather than an error), or it may lead to bizarre behaviour like module level state modifications not being shared as expected, or even exceptions not being caught because the exception handler was trying to catch the exception type from one instance of the module, while the exception raised used the type from the other instance.

因此，含糊不清的这可能会导致不可预测的行为警告——如果由于两次运行模块的顶级代码而导致出现问题，则症状可能几乎是任何情况.

Hence the vague this may cause unpredictable behaviour warning - if things do go wrong as a result of running the module's top level code twice, the symptoms may be pretty much anything.

如何调试更复杂的案例?

虽然在这个特定的例子中，副作用导入直接在 proj/__init__.py 中，但有一个更微妙且难以调试的变体，父包代替:

While in this particular example, the side-effect import is directly in proj/__init__.py, there's a far more subtle and hard to debug variant where the parent package instead does:

import some_other_module

然后是 some_other_module(或它导入的模块):

and then it is some_other_module (or a module that it imports) that does:

import proj.proj # or "from proj import proj"

假设错误行为是可重现的，调试这类问题的主要方法是在详细模式下运行 python 并检查导入顺序:

Assuming the misbehaviour is reproducible, the main way to debug these kinds of problems is to run python in verbose mode and check the import sequence:

$ python -v -c "print('Hello')" 2>&1 | grep '^import'
import zipimport # builtin
import site # precompiled from /usr/lib64/python2.7/site.pyc
import os # precompiled from /usr/lib64/python2.7/os.pyc
import errno # builtin
import posix # builtin
import posixpath # precompiled from /usr/lib64/python2.7/posixpath.pyc
import stat # precompiled from /usr/lib64/python2.7/stat.pyc
import genericpath # precompiled from /usr/lib64/python2.7/genericpath.pyc
import warnings # precompiled from /usr/lib64/python2.7/warnings.pyc
import linecache # precompiled from /usr/lib64/python2.7/linecache.pyc
import types # precompiled from /usr/lib64/python2.7/types.pyc
import UserDict # precompiled from /usr/lib64/python2.7/UserDict.pyc
import _abcoll # precompiled from /usr/lib64/python2.7/_abcoll.pyc
import abc # precompiled from /usr/lib64/python2.7/abc.pyc
import _weakrefset # precompiled from /usr/lib64/python2.7/_weakrefset.pyc
import _weakref # builtin
import copy_reg # precompiled from /usr/lib64/python2.7/copy_reg.pyc
import traceback # precompiled from /usr/lib64/python2.7/traceback.pyc
import sysconfig # precompiled from /usr/lib64/python2.7/sysconfig.pyc
import re # precompiled from /usr/lib64/python2.7/re.pyc
import sre_compile # precompiled from /usr/lib64/python2.7/sre_compile.pyc
import _sre # builtin
import sre_parse # precompiled from /usr/lib64/python2.7/sre_parse.pyc
import sre_constants # precompiled from /usr/lib64/python2.7/sre_constants.pyc
import _locale # dynamically loaded from /usr/lib64/python2.7/lib-dynload/_localemodule.so
import _sysconfigdata # precompiled from /usr/lib64/python2.7/_sysconfigdata.pyc
import abrt_exception_handler # precompiled from /usr/lib64/python2.7/site-packages/abrt_exception_handler.pyc
import encodings # directory /usr/lib64/python2.7/encodings
import encodings # precompiled from /usr/lib64/python2.7/encodings/__init__.pyc
import codecs # precompiled from /usr/lib64/python2.7/codecs.pyc
import _codecs # builtin
import encodings.aliases # precompiled from /usr/lib64/python2.7/encodings/aliases.pyc
import encodings.utf_8 # precompiled from /usr/lib64/python2.7/encodings/utf_8.pyc

此特定示例仅显示 Fedora 上的 Python 2.7 在启动时执行的基本导入集.在调试像这个问题中的双重导入 RuntimeWarning 时，您将在详细输出中搜索import proj"和import proj.proj"行，然后仔细查看在import proj.proj"行之前的导入处.

This particular example just shows the base set of imports that Python 2.7 on Fedora does at startup. When debugging a double-import RuntimeWarning like the one in this question, you'd be searching for the "import proj" and then "import proj.proj" lines in the verbose output, and then looking closely at the imports immediately preceding the "import proj.proj" line.

这篇关于Python 3.6 项目结构导致 RuntimeWarning的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！