在Python中，如何判断模块是否来自C扩展? [英] In Python how can one tell if a module comes from a C extension?

问题描述

从Python判断导入的模块是否来自 C扩展，而不是纯Python模块?例如，如果Python包中的模块同时具有纯Python实现和C实现，并且您希望能够在运行时知道正在使用哪个模块，则此功能非常有用.

What is the correct or most robust way to tell from Python if an imported module comes from a C extension as opposed to a pure Python module? This is useful, for example, if a Python package has a module with both a pure Python implementation and a C implementation, and you want to be able to tell at runtime which one is being used.

一个想法是检查module.__file__的文件扩展名，但是我不确定一个人应该检查的所有文件扩展名以及这种方法是否一定是最可靠的.

One idea is to examine the file extension of module.__file__, but I'm not sure all the file extensions one should check for and if this approach is necessarily the most reliable.

推荐答案

首先，我认为这根本没有用.对于模块来说，通常是围绕C扩展模块的纯Python包装，或者在某些情况下(如果有)围绕C扩展模块的纯Python包装，或者如果不存在，则是纯Python的实现.

First, I don't think this is at all useful. It's very common for modules to be pure-Python wrappers around a C extension module—or, in some cases, pure-Python wrappers around a C extension module if it's available, or a pure Python implementation if not.

对于一些流行的第三方示例:numpy是纯Python，即使所有重要的事情都用C实现了； bintrees是纯Python，即使其类可以全部用C或Python实现，具体取决于您如何构建它.等

For some popular third-party examples: numpy is pure Python, even though everything important is implemented in C; bintrees is pure Python, even though its classes may all be implemented either in C or in Python depending on how you build it; etc.

从3.2版本开始的大多数stdlib中都是如此.例如，如果您只是import pickle，则实现类将在CPython中以C语言构建(您从2.7中的cpickle中获取的东西)，而在PyPy中它们将为纯Python版本，但是无论哪种方式pickle本身就是纯Python.

And this is true in most of the stdlib from 3.2 on. For example, if you just import pickle, the implementation classes will be built in C (what you used to get from cpickle in 2.7) in CPython, while they'll be pure-Python versions in PyPy, but either way pickle itself is pure Python.

但是，如果您要做要这样做，则实际上需要区分三件事:

But if you do want to do this, you actually need to distinguish three things:

• 内置模块，例如sys.
• C扩展模块，例如2.x的cpickle.
• 纯Python模块，例如2.x的pickle.

• Built-in modules, like sys.
• C extension modules, like 2.x's cpickle.
• Pure Python modules, like 2.x's pickle.

那是假设您只关心CPython；如果您的代码在Jython或IronPython中运行，则实现可能是JVM或.NET，而不是本机代码.

And that's assuming you only care about CPython; if your code runs in, say, Jython, or IronPython, the implementation could be JVM or .NET rather than native code.

由于多种原因，您无法完全根据__file__进行区分:

You can't distinguish perfectly based on __file__, for a number of reasons:

• 内置模块根本没有__file__. (这记录在几个地方，例如类型和成员表在inspect文档中.)请注意，如果您使用的是py2app或cx_freeze之类的东西，则内置"项可能与独立安装有所不同.
• 纯Python模块可能具有.pyc/.pyo文件，而分布式应用程序中没有.py文件.
• 作为单文件egg安装的软件包中的模块(在easy_install中很常见，在pip中较少)，其空白或无用__file__.
• 如果您构建二进制发行版，则很有可能将整个库打包到一个zip文件中，从而导致与单文件鸡蛋相同的问题.

• Built-in modules have no __file__ at all. (This is documented in a few places—e.g., the Types and members table in the inspect docs.) Note that if you're using something like py2app or cx_freeze, what counts as "built-in" may be different from a standalone installation.
• A pure-Python module may have a .pyc/.pyo file without having a .py file in a distributed app.
• A module in a a package installed as a single-file egg (which is common with easy_install, less so with pip) will have either a blank or useless __file__.
• If you build a binary distribution, there's a good chance your whole library will be packed in a zip file, causing the same problem as single-file eggs.

在3.1及更高版本中，导入过程已被大量清理，大部分用Python重写，并且大部分暴露于Python层.

In 3.1+, the import process has been massively cleaned up, mostly rewritten in Python, and mostly exposed to the Python layer.

因此，您可以使用 importlib 模块来查看用于加载模块的加载程序，最终您将获得BuiltinImporter(内置文件)，ExtensionFileLoader(.so/.pyd/etc.)，SourceFileLoader(.py)或SourcelessFileLoader(.pyc)/.pyo).

So, you can use the importlib module to see the chain of loaders used to load a module, and ultimately you'll get to BuiltinImporter (builtins), ExtensionFileLoader (.so/.pyd/etc.), SourceFileLoader (.py), or SourcelessFileLoader (.pyc/.pyo).

在importlib.machinery中，您还可以在当前目标平台上看到为这四个变量分配的后缀.因此，您可以检查any(pathname.endswith(suffix) for suffix in importlib.machinery.EXTENSION_SUFFIXES))，但这实际上对例如鸡蛋/拉链盒没有帮助，除非您已经沿链条向上走过.

You can also see the suffixes assigned to each of the four, on the current target platform, as constants in importlib.machinery. So, you could check that the any(pathname.endswith(suffix) for suffix in importlib.machinery.EXTENSION_SUFFIXES)), but that won't actually help in, e.g., the egg/zip case unless you've already traveled up the chain anyway.

为此，任何人都想到的最好的启发式方法是在inspect模块中实现的启发式方法，所以最好的办法就是使用它.

The best heuristics anyone has come up with for this are the ones implemented in the inspect module, so the best thing to do is to use that.

最佳选择是getsource，getsourcefile和getfile中的一个或多个；最好取决于您想要的启发式方法.

The best choice will be one or more of getsource, getsourcefile, and getfile; which is best depends on which heuristics you want.

内置模块将为其中任何一个引发TypeError.

A built-in module will raise a TypeError for any of them.

扩展模块应该为getsourcefile返回一个空字符串.这似乎适用于我拥有的所有2.5-3.4版本，但我没有2.4左右.对于getsource，至少在某些版本中，它返回.so文件的实际字节，即使它应该返回空字符串或引发IOError. (在3.x中，几乎可以肯定会得到UnicodeError或SyntaxError，但您可能不想依赖它……)

An extension module ought to return an empty string for getsourcefile. This seems to work in all the 2.5-3.4 versions I have, but I don't have 2.4 around. For getsource, at least in some versions, it returns the actual bytes of the .so file, even though it should be returning an empty string or raising an IOError. (In 3.x, you will almost certainly get a UnicodeError or SyntaxError, but you probably don't want to rely on that…)

如果在egg/zip/etc中，纯Python模块可能会为getsourcefile返回一个空字符串.如果源可用，即使在egg/zip/etc之内，它们也应始终为getsource返回一个非空字符串，但是，如果它们是无源字节码(.pyc/etc.)，则它们将返回一个空字符串或引发IOError.

Pure Python modules may return an empty string for getsourcefile if in an egg/zip/etc. They should always return a non-empty string for getsource if source is available, even inside an egg/zip/etc., but if they're sourceless bytecode (.pyc/etc.) they will return an empty string or raise an IOError.

最好的选择是在您关心的发行版/设置中，在您关心的平台上试验您关心的版本.

The best bet is to experiment with the version you care about on the platform(s) you care about in the distribution/setup(s) you care about.

这篇关于在Python中，如何判断模块是否来自C扩展?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！