如何在 Airflow (docker-compose) 中安装软件包? [英] How to install packages in Airflow (docker-compose)?

问题描述

这个问题与已经可用的问题非常相似.唯一的区别是我在 docker 中运行 Airflow

The question is very similar to the one already available. The only difference is that I ran Airflow in docker

循序渐进:

1. 将 docker-compose.yaml 放到PyCharm 项目
2. 将 requirements.txt 放入 PyCharm 项目
3. 运行docker-compose up
4. 运行 DAG 并收到 ModuleNotFoundError

1. Put docker-compose.yaml to PyCharm project
2. Put requirements.txt to PyCharm project
3. Run docker-compose up
4. Run DAG and receive a ModuleNotFoundError

我想启动 Airflow 使用 docker-compose 带有来自 requirements.txt 的依赖项.PyCharm 解释器和 DAG 执行期间应该可以使用这些依赖项

I want to start Airflow using docker-compose with the dependencies from requirements.txt. These dependencies should be available by PyCharm interpreter and during DAGs execution

有不需要重建镜像的解决方案吗?

Is there a solution that doesn't require rebuilding the image?

推荐答案

有没有不需要重建镜像的解决方案?

Is there a solution that doesn't require rebuilding the image?

是的，现在有:目前(oct-2021 v2.2.0)它可以作为环境变量使用:

Yes there is now: currently (oct-2021 v2.2.0) it's available as an env variable:

_PIP_ADDITIONAL_REQUIREMENTS

在 docker-compose.yml 文件中使用.正如其他一些答案所解释的那样，这应该可以解决问题而无需构建完整的图像(实际上非​​常好:-)

It is used in the docker-compose.yml file. That should do the trick without building a complete image as some of the other answers explain (very well actually :-)

参见:https://airflow.apache.org/docs/apache-airflow/stable/docker-compose.yaml

官方文档https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html#environment-variables-supported-by-docker-compose

这篇关于如何在 Airflow (docker-compose) 中安装软件包?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！