Python setuptools/pip 将数据文件打包到你的包中 [英] Python setuptools/pip packing data files into your package

问题描述

我有这个 git repo 结构:

I have this git repo structure:

.gitignore
JSONs/subdirA/some.json
JSONs/subdirB/other.json
MyPackage/__init__.py
MyPackage/myModule.py

如何将 JSONs 文件夹正确打包到 MyPackage/JSONs 中，而不将其永久移动到那里(主要是因为客户直接将此 git 存储库用于非 Python 用途，并且 repo 顶部的文件夹很简单/直观......但现在我也想将这个相同的目录发布到我的 PyPi 包中)?

How do I properly pack the JSONs folder into MyPackage/JSONs, without moving it there permanently (mostly because customers use this git repo directly for non-python usage, and the folder at the top of the repo is easy/intuitive... But now I also want to release this same dir into my PyPi package)?

我尝试将它添加到 MANIFEST.in 然后在 setup.py 和 中使用 data_filespackage_data... 但无济于事.也许一些 .pyc 或缓存的构建文件对我来说是最好的......但我还没有从所有其他(不完全)重复的问题中弄清楚它，因为它们没有特别指出它们的目录结构和所需的最终位置.

I've tried adding it to the MANIFEST.in and then playing with data_files in setup.py as well as package_data... But to no avail. Maybe some .pyc or cached build files got the best of me... But I haven't figured it out from all the other (not quite) duplicate questions as they don't specifically call out their directory structure and desired final location.

我尝试在调用 setup 之前诉诸 os.walk 和 shutil.copy，然后在 之后删除该目录>setup... 虽然它似乎在本地工作，但当推送到我们本地的 devpi 包服务器时，出现了问题.我的目标是否完全脱离了 setuptools/pip 理想?或者我只是缺少一些关键的理解?请赐教！

I've tried resorting to os.walk and shutil.copy before the call to setup and then deleting that directory after setup... While it seems to work locally, when pushing to our local devpi package server, something goes wrong. Is my goal totally off the radar for setuptools/pip ideals??? Or am I just missing some key understanding? Please enlighten me!

推荐答案

以下内容可能会有所帮助:

Something like the following could help:

首先我们需要确保 json 文件被添加到源代码分发.

First we need to make sure that the json files are added to the source distribution.

MANIFEST.in:

recursive-include JSONs *.json

然后在实际的设置脚本中，必须动态修改包列表以考虑目标包结构.

Then in the actual setup script, the list of packages has to be modified on the fly to take into account the target package structure.

setup.py:

#!/usr/bin/env python3

import setuptools

PACKAGES = (
    setuptools.find_packages(exclude=['JSONs*'])
    +
    [
        f'MyPackage.{package}'
        for package
        in setuptools.find_namespace_packages(include=['JSONs*'])
    ]
)

setuptools.setup(
    packages=PACKAGES,
    package_dir={
        'MyPackage.JSONs': 'JSONs',
    },
    include_package_data=True,
    #
    name='Something',
    version='1.2.3',
)

JSONs/subdirA/some.json :

{"Marco": "Polo"}

这样的包数据可以这样读取:

MyPackage/myModule.py:

import pkgutil

print(pkgutil.get_data('MyPackage', 'JSONs/subdirA/some.json').decode())

然后像下面这样使用它:

And use it like in the following:

$ python -m pip install .
$ # Move to another directory to prevent that the current working directory
$ # ... overshadows the installed project
$ cd ..
$ python -m MyPackage.myModule
{"Marco": "Polo"}

这篇关于Python setuptools/pip 将数据文件打包到你的包中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！