我们可以看出python包装和进口的工作原理吗？ [英] Can we shed some definitive light on how python packaging and import works?

问题描述

我有机会通过python管理模块，每一次都是一个挑战：包装不是人们每天都在做的事情，它成为一个学习的负担，也是一个需要记住的负担，即使你实际上是这样做的，因为这通常会发生一次。

我想在这里收集关于如何导入，包管理和分发在python中的最终概述，以便这个问题成为在引擎盖下发生的所有魔法的最终解释。虽然我了解这个问题的广泛性，但这些事情是如此交织在一起，任何一个重点的答案都不能解决主要的问题：了解所有的工作，什么是过时的，什么是现在，什么是同一个任务的替代品，什么是这个怪癖。

要引用的关键字列表如下，但这只是一个例子。还有更多的东西，欢迎你添加更多的细节。

• PyPI


• setuptools / Distribute


• distutils


• egg


• egg-link


• pip


• zipimport


• site.py


• site-packages


• .pth文件


• virtualenv


• 在鸡蛋中处理已编译的模块（有或没有通过easy_install安装）


• 使用get_data（）


• pypm


• bento


• PEP 376


• 奶酪店


• 可执行

链接到其他答案可能是一个好主意。正如我所说，这个问题是高层次的概述。

解决方案

在大多数情况下，这是试图看在包装/分发方面，不是 import 的机制。不幸的是，包装是Python提供多种方法的地方。我只是试图让球滚动，希望其他人会帮助填补我错过或指出错误。

首先，这里有一些凌乱的术语。包含 __ init __。py 文件的目录是一个包。然而，我们在这里谈论的大部分内容是在PyPI上发布的特定版本的软件，其中一个是镜像，或者是像Debian的Apt，Redhat的Yum，Fink，Macports，Homebrew或ActiveState的pypm之类的供应商特定的软件包管理系统。

这些发布的软件包是人们正在尝试使用分发，试图使用软件包作为Python语言构造。您可以在 PEP-376 PEP-376 。

现在，您的关键字列表涉及Python生态系统的几个不同方面：

查找和发布python分发：

• PyPI（也称奶酪店）


• PyPI镜像


• 各种软件包管理工具/系统：apt，yum，fink，macports，homebrew


• pypm（ActiveState替代PyPI）

以上都是提供以各种格式发布Python发行版的地方的所有服务。有些像PyPI镜像和apt / yum存储库可以在本地机器上或公司网络中运行，但人们通常使用官方的。大多数（如果不是全部）提供了一个工具（或者是PyPI的多个工具）来帮助查找和下载发行版。

用于创建和安装发行版的库： h2>

• setuptools /发行


• distutils

Distutils是编译和内置到发行版中的Python包的标准基础架构。 distutils 中有大量功能，但大多数人只知道：

  from distutils.core import setup 
 
 setup（name ='Distutils'，
 version ='1.0'，
 description ='Python Distribution Utilities'，
 author = 'Greg Ward'，
 author_email='[e-mail]'，
 url ='http：//www.python.org/sigs/distutils-sig/'，
 packages = ['distutils'，'distutils.command']，
）
  

在一定程度上这是你最需要的。使用以前的9行代码，您可以获得足够的信息来安装纯Python包以及发布包在PyPI上的发行版所需的最小元数据。

Setuptools提供需要支持Egg格式的钩子以及所有这些功能和优点。分发是一种替代Setuptools，它在尝试大部分向后兼容时添加了一些功能。我相信Distribute将被包含在Python 3中，作为来自distutils.core import setup 的Distutil的的继承者。

Setuptools和Distribute都提供了一个自定义版本的 distutils 安装程序命令
，它有助于支持Egg格式。

Python发布格式：

• 源


• eggs

分配通常是提供源文件（tarball或zipfile）。安装源分发的标准方法是通过下载和解压缩存档，然后在其中运行 setup.py 文件。

例如，以下内容将下载，构建和安装Pygments语法高亮库：

 卷曲-O -G http://pypi.python.org/packages/source/P/Pygments/Pygments-1.4.tar.gz 
 tar -zxvf Pygments-1.4.tar.gz 
 cd Pygments-1.4 
 python setup.py build 
 sudo python setup.py install 
  

或者，您可以下载蛋档并安装。通常这是通过使用easy_install或pip来实现的：

  sudo easy_install pygments 
或
 sudo pip install pygments 
  

鸡蛋受到Java的Jarfiles的启发，他们有很多功能，你应该阅读关于这里

Python包格式：

• 未压缩的目录
• 
  
• zipimport（zip压缩目录）

一个普通的python包只是一个包含 __ init __。py 文件和任意数量的附加模块或子包。 Python还支持在* .zip文件中查找和加载源代码，只要它们包含在 PYTHONPATH （ sys.path



安装Python软件包：

• easy_install ：原始的egg安装工具取决于 setuptools


• pip ：目前最流行的方式来安装python包。类似于 easy_install ，但更灵活，并具有一些不错的功能，如需求文件，以帮助文档依赖和复制部署。


• pypm ， apt ， yum ，fink等等

环境管理/自动部署：

• bento


• buildout


• virtualenv （和 virtualenvwrapper ）

以上工具用于帮助自动化和管理Python项目的依赖关系。基本上，它们为您提供工具来描述应用程序所需的发行版本，并自动安装这些特定版本的依赖项。

软件包/分发版的位置：

• 网站包


• PYTHONPATH


• 当前工作目录（取决于您的操作系统和环境设置）

默认情况下，安装python发行版将其放入site-packages目录中。该目录通常类似于 /usr/lib/pythonX.Y/site-packages 。

一个简单的程序化找到你的site-packages目录的方法：

 从distuils导入sysconfig 
打印sysconfig.get_python_lib（）
  

修改PYTHONPATH的方法：

Python的导入语句只会找到位于您的 PYTHONPATH 中包含的目录之一的包。

您可以检查和更改您可以通过访问Python中的路径：

  import sys 
打印sys.path 
 sys.path。 append（/ home / myname / lib）
  

此外，您可以设置 PYTHONPATH 环境变量就像您的操作系统上的任何其他环境变量一样，您可以使用：

• .pth文件：* .pth文件位于已经在您的 PYTHONPATH 上的目录中已被读取， * .pth文件的ch行被添加到您的 PYTHONPATH 中。基本上任何时候您都可以将包复制到您的 PYTHONPATH 中的目录中，您可以创建一个 mypackages.pth 。详细了解* .pth文件：站点模块


• egg-link文件：蟒蛇蛋内部结构
  
• site.py 修改

要将上述 / home / myname / lib 添加到* .pth文件的站点包中，创建一个* .pth文件。文件的名称并不重要，但您仍然应该选择合适的方式。

我们创建 myname.pth ：

 ＃myname.pth 
 / home / myname / lib 
  

就是这样。在您的系统或您的 PYTHONPATH 和 sysconfig.get_python_lib（） > / home / myname / lib 将被添加到路径。

I had my fair chance of getting through the python management of modules, and every time is a challenge: packaging is not what people do every day, and it becomes a burden to learn, and a burden to remember, even when you actually do it, since this happens normally once.

I would like to collect here the definitive overview of how import, package management and distribution works in python, so that this question becomes the definitive explanation for all the magic that happens under the hood. Although I understand the broad level of the question, these things are so intertwined that any focused answer will not solve the main problem: understand how all works, what is outdated, what is current, what are just alternatives for the same task, what are the quirks.

The list of keywords to refer to is the following, but this is just a sample out of the bunch. There's a lot more and you are welcome to add additional details.

• PyPI
• setuptools / Distribute
• distutils
• eggs
• egg-link
• pip
• zipimport
• site.py
• site-packages
• .pth files
• virtualenv
• handling of compiled modules in eggs (with and without installation via easy_install)
• use of get_data()
• pypm
• bento
• PEP 376
• the cheese shop
• eggsecutable

Linking to other answers is probably a good idea. As I said, this question is for the high-level overview.

解决方案

For the most part, this is an attempt to look at the packaging/distribution side, not the mechanics of import. Unfortunately, packaging is the place where Python provides way more than one way to do it. I'm just trying to get the ball rolling, hopefully others will help fill what I miss or point out mistakes.

First of all there's some messy terminology here. A directory containing an __init__.py file is a package. However, most of what we're talking about here are specific versions of packages published on PyPI, one of it's mirrors, or in a vendor specific package management system like Debian's Apt, Redhat's Yum, Fink, Macports, Homebrew, or ActiveState's pypm.

These published packages are what folks are trying to call "Distributions" going forward in an attempt to use "Package" only as the Python language construct. You can see some of that usage in PEP-376 PEP-376.

Now, your list of keywords relate to several different aspects of the Python Ecosystem:

Finding and publishing python distributions:

• PyPI (aka the cheese shop)
• PyPI Mirrors
• Various package management tools / systems: apt, yum, fink, macports, homebrew
• pypm (ActiveState's alternative to PyPI)

The above are all services that provide a place to publish Python distributions in various formats. Some, like PyPI mirrors and apt / yum repositories can be run on your local machine or within your companies network but folks typically use the official ones. Most, if not all provide a tool (or multiple tools in the case of PyPI) to help find and download distributions.

Libraries used to create and install distributions:

• setuptools / Distribute
• distutils

Distutils is the standard infrastructure on which Python packages are compiled and built into distributions. There's a ton of functionality in distutils but most folks just know:

from distutils.core import setup

setup(name='Distutils',
      version='1.0',
      description='Python Distribution Utilities',
      author='Greg Ward',
      author_email='[e-mail]',
      url='http://www.python.org/sigs/distutils-sig/',
      packages=['distutils', 'distutils.command'],
 )

And to some extent that's a most of what you need. With the prior 9 lines of code you have enough information to install a pure Python package and also the minimal metadata required to publish that package a distribution on PyPI.

Setuptools provides the hooks necessary to support the Egg format and all of it's features and foibles. Distribute is an alternative to Setuptools that adds some features while trying to be mostly backwards compatible. I believe Distribute is going to be included in Python 3 as the successor to Distutil's from distutils.core import setup.

Both Setuptools and Distribute provide a custom version of the distutils setup command that does useful things like support the Egg format.

Python Distribution Formats:

• source
• eggs

Distributions are typically provided either as source archives (tarball or zipfile). The standard way to install a source distribution is by downloading and uncompressing the archive and then running the setup.py file inside.

For example, the following will download, build, and install the Pygments syntax highlighting library:

curl -O -G http://pypi.python.org/packages/source/P/Pygments/Pygments-1.4.tar.gz
tar -zxvf Pygments-1.4.tar.gz
cd Pygments-1.4
python setup.py build
sudo python setup.py install

Alternatively you can download the Egg file and install it. Typically this is accomplished by using easy_install or pip:

sudo easy_install pygments
or 
sudo pip install pygments

Eggs were inspired by Java's Jarfiles and they have quite a few features you should read about here

Python Package Formats:

• uncompressed directories
• zipimport (zip compressed directories)

A normal python package is just a directory containing an __init__.py file and an arbitrary number of additional modules or sub-packages. Python also has support for finding and loading source code within *.zip files as long as they are included on the PYTHONPATH (sys.path).

Installing Python Packages:

• easy_install: the original egg installation tool, depends on setuptools
• pip: currently the most popular way to install python packages. Similar to easy_install but more flexible and has some nice features like requirements files to help document dependencies and reproduce deployments.
• pypm, apt, yum, fink, etc

Environment Management / Automated Deployment:

• bento
• buildout
• virtualenv (and virtualenvwrapper)

The above tools are used to help automate and manage dependencies for a Python project. Basically they give you tools to describe what distributions your application requires and automate the installation of those specific versions of your dependencies.

Locations of Packages / Distributions:

• site-packages
• PYTHONPATH
• the current working directory (depends on your OS and environment settings)

By default, installing a python distribution is going to drop it into the site-packages directory. That directory is usually something like /usr/lib/pythonX.Y/site-packages.

A simple programmatic way to find your site-packages directory:

from distuils import sysconfig
print sysconfig.get_python_lib()

Ways to modify your PYTHONPATH:

Python's import statement will only find packages that are located in one of the directories included in your PYTHONPATH.

You can inspect and change your path from within Python by accessing:

import sys
print sys.path
sys.path.append("/home/myname/lib")

Besides that, you can set the PYTHONPATH environment variable like you would any other environment variable on your OS or you could use:

• .pth files: *.pth files located in directories that are already on your PYTHONPATH are read and each line of the *.pth file is added to your PYTHONPATH. Basically any time you would copy a package into a directory on your PYTHONPATH you could instead create a mypackages.pth. Read more about *.pth files: site module
• egg-link files: Internal structure of python eggs they are a cross platform alternative to symbolic links. Creating an egg link file is similar to creating a pth file.
• site.py modifications

To add the above /home/myname/lib to site-packages with a *.pth file you'd create a *.pth file. The name of the file doesn't matter but you should still probably choose something sensible.

Let's create myname.pth:

# myname.pth
/home/myname/lib

That's it. Drop that into sysconfig.get_python_lib() on your system or any other directory in your PYTHONPATH and /home/myname/lib will be added to the path.

这篇关于我们可以看出python包装和进口的工作原理吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！