如何使用Docker和Monorepo组织共享库 [英] How to organize shared libraries with docker and monorepo

问题描述

我有2个python应用程序，它们共享一些代码，足以让我试图将共享的部分隔离到模块/软件包/库中(我不确定这个术语是故意的，因为我不确定解决方案是).我所有的代码都存储在monorepo中，因为我希望克服一些管理比我们团队成员更多的存储库的烦恼.

I have 2 python apps that share a few bits of code, enough that I am trying to isolate the shared parts into modules/packages/libraries (I'm keeping the term vague on purpose, as I am not sure what the solution is). All my code is in a monorepo, because I am hoping to overcome some of the annoyances of managing more repos than we have team members.

当前我的文件布局如下:

Currently my file layout looks like:

+ myproject
  + appA
  | + python backend A
  | + js frontend
  + appB
  | + B stuff
  + libs
    + lib1
    + lib2

appA 和 appB 都使用 lib1 和 lib2 (它们本质上是数据模型，用于提取共享的数据库). appA 是一个包含多个组件的Webapp，但并非所有组件都是python.它被部署为涉及一堆容器的Docker堆栈.我使用诗歌来管理我的依赖项，以确保生成可重现的构建等.每个python组件( appA ， appB ...)具有自己的 pyproject.toml 文件，虚拟环境等...

Both appA and appB use lib1 and lib2 (they are essentially data models to abstract away the shared database). appA is a webapp with several components, not all of which are python. It is deployed as a docker stack that involve a bunch of containers. I manage my dependencies with poetry to ensure reproducible builds, etc... Each python component (appA, appB...) have their own pyproject.toml file, virtual env, etc...

appB 单独部署.

所有开发都在linux上进行，如果有任何不同的话.

All development is on linux, if it makes any difference.

我正在寻找一种处理libs的干净方法:

I am looking for a clean way to deal with the libs:

• appA 的开发是在本地 docker-compose 设置中完成的.后端会在文件更改时自动重新加载(使用docker卷)，我也希望库中的更改也能重新加载.
• 针对 appB 的开发较为简单，但是正迁移至docker，因此问题将相同.

• development for appA is done in a local docker-compose setup. The backend auto-reloads on file changes (using a docker volume), and I would like it to happen for changes in the libs too.
• development for appB is simpler, but is moving to docker so the problem will be the same.

• 我最初的解决方案"是将libs文件夹复制到一个临时位置，以便在 appA 中进行开发.它适用于导入，但是一旦我想更改libs代码(仍然很频繁)，就很麻烦，因为我需要更改原始文件，复制过来，然后重建容器.
• 我尝试将库符号链接到后端的docker环境中，但符号链接似乎不适用于docker(它似乎不遵循链接，因此文件不会最终出现在docker映像中，除非我本质上是在docker build上下文中复制文件，这违反了链接的目的.)
• 我尝试将每个库打包到一个python包中，并通过 poetry add ../../libs/lib1 安装它们，因为路径无法在docker内部运行匹配，然后回到符号链接问题.

• My initial "solution" was to copy the libs folder over to a temporary location for development in appA. It works for imports, but it's messy as soon as I want to change the libs code (which is still quite often), as I need to change the original file, copy over, rebuild the container.
• I tried symlinking the libs into the backend's docker environment, but symlinks don't seem to work well with docker (it did not seem to follow the link, so the files don't end up in the docker image, unless I essentially copy the files inside the docker build context, which defeats the purpose of the link.)
• I have tried packaging each lib into a python package, and install them via poetry add ../../libs/lib1 which doesn't really work inside docker because the paths don't match, and then I'm back to the symlink issue.

我敢肯定有一种干净的方法可以做到这一点，但我无法弄清楚.我知道我可以将存储库分解成较小的存储库并安装依赖项，但是开发仍然会在docker内部引起问题，因为每次更改lib文件时仍需要重新构建容器，因此我宁愿保留monorepo./p>

I am sure there is a clean way to do this, but I can't figure it out. I know I could break up the repo into smaller ones and install dependencies, but development would still cause problems inside docker, as I would still need to rebuild the container each time I change the lib files, so I would rather keep the monorepo.

推荐答案

如果您仍在使用docker-compose，则可以使用卷将本地库挂载到容器中，并能够在主机系统和容器中对其进行编辑.不是超级幻想，但这应该可行，对吗?

If you are using docker-compose anyway you could use volumes to mount the local libs in your container and be able to edit them in your host system and the container. Not super fancy, but that should work, right?

@ckaserer，您的建议确实可行.简而言之，在docker文件中，我执行COPY ../libs/lib1/app/lib1，然后进行本地开发，将../libs/lib1挂载到/app/lib1上.这给了我我一直在寻找的行为.我为此使用了一个分割的docker-compose文件.该安装程序会导致各种工具需要一些额外的配置，从而导致一些问题，因此他们知道库是代码库的一部分，但是没有什么不可能.谢谢你的主意！

@ckaserer your suggestion seems to work, indeed. In short, in the docker files I do COPY ../libs/lib1 /app/lib1 and then for local development, I mount ../libs/lib1 onto /app/lib1. That gives me the behavior I was looking for. I use a split docker-compose file for this. The setup causes a few issues with various tools needing some extra config so they know that the libs are part of the code base, but nothing impossible. Thanks for the idea!

因此，即使这不是理想的解决方案，也可以在Linux系统上在本地安装app和lib目录.

So even though it's not an ideal solution locally mounting over the app and lib directories works on Linux systems.

仅供参考:在Windows主机上，如果您要监视文件更改(可能不会从Windows主机传播到Linux容器)，可能会遇到麻烦.

FYI: On Windows hosts you might run into trouble if you want to watch for file changes as that is not propagated from a windows host to a Linux container.

这篇关于如何使用Docker和Monorepo组织共享库的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！