防止私有和基于 pypi 的 Python 包之间的命名空间冲突 [英] Preventing namespace collisions between private and pypi-based Python packages

问题描述

我们有 100 多个私有包，到目前为止我们一直在使用 s3pypi 在 s3 存储桶中设置私有 pypi.我们的私有包相互依赖(以及公共包)，并且(当然)我们的 GitLab 管道找到它所依赖的包的最新功能版本是很重要的.IE.我们对最新签入的代码不感兴趣.我们仅在测试后创建新轮子，并且 qa 已经运行到主控(这是解释 -e <vcs> 要求将不起作用的冗长方式).

We have 100+ private packages and so far we've been using s3pypi to set up a private pypi in an s3 bucket. Our private packages have dependencies on each other (and on public packages), and it is (of course) important that our GitLab pipelines find the latest functional version of packages it relies on. I.e. we're not interested in the latest checked in code. We create new wheels only after tests and qa has run against a push to master (which is a long-winded way of explaining that -e <vcs> requirements will not work).

我们的设置非常有效，直到有人在官方 pypi 上创建了一个新的公共包，该包隐藏了我们的包名称之一.我们可以通过增加版本号来强制选择我们的私有包，使其高于 pypi.org 上的新包 - 或者通过将我们的包重命名为尚未在 pypi.org 上使用的内容.

Our setup works really well until someone creates a new public package on the official pypi that shadows one of our package names. We can force our private package to be chosen by increasing the version number so it is higher than the new package on pypi.org - or by renaming our package to something that haven't yet been taken on pypi.org.

这显然是一个笨拙和脆弱的解决方案，但显然功能是这样的按设计.

This is obviously a hacky and fragile solution, but apparently the functionality is this way by-design.

在初始存储桶设置后 s3pypi 不需要维护或管理.上面的票建议使用 devpi 但这似乎就像一个非常繁重的解决方案，需要管理/监控/等.

After the initial bucket setup s3pypi has required no maintenance or administration. The above ticket suggests using devpi but that seems like a very heavy solution that requires administration/monitoring/etc.

GitLab 的 pypi 解决方案似乎是在单个包级别(这意味着我们必须列出多达 100 个以上的网址 - 每个包一个).这似乎不太实用，但也许我不明白某些东西(我也可以看到我们组下的包注册表菜单，但文档指向package-pypi"文档).

GitLab's pypi solution seems to be at individual package level (meaning we'd have to list up to 100+ urls - one for each package). This doesn't seem practical, but maybe I'm not understanding something (I can see the package registry menu under our group as well, but the docs point to the "package-pypi" docs).

我们不可能是第一个遇到这个问题的小公司..?有没有比在 pypi.org 上注册我们所有包的虚拟版本更好的方法(版本 = 0.0.1，因此 s3pypi 版本将是首选)?

We can't be the first small company that has faced this issue..? Is there a better way than to register dummy versions of all our packages on pypi.org (with version=0.0.1, so the s3pypi version will be preferred)?

推荐答案

这可能不是您的解决方案，但我告诉我们我们在做什么.

It might not be the solution for you, but I tell what we do.

1. 为包名称加上前缀，并使用命名空间(例如 company.product.tool).
2. 当我们安装我们的包(包括它们的内部依赖项)时，我们使用一个 requirements.txt 文件，其中包括我们的 PyPI URL.我们在容器中运行所有内容，并在构建映像时在其中安装所有公共依赖项.

1. Prefix the package names, and using namespaces (eg. company.product.tool).
2. When we install our packages (including their in-house dependencies), we use a requirements.txt file including our PyPI URL. We run everything in container(s) and we install all public dependencies in them when we are building the images.

这篇关于防止私有和基于 pypi 的 Python 包之间的命名空间冲突的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！