在 Docker 容器中运行的 Airflow 可以访问本地文件吗? [英] Can Airflow running in a Docker container access a local file?
问题描述
就 Airflow 和 Docker 而言，我是新手；为了让事情变得更复杂，我使用了 Astronomer，更糟糕的是，我在 Windows 上运行 Airflow.(不在 Unix 子系统上 - 无法在 Ubuntu 20.4 上安装 Docker).天文开发开始"因错误而中断，但在 Docker 桌面中，我看到并可以启动 3 个与 Airflow 相关的容器.他们可以很好地看到我的 DAG，但我的 DAG 看不到本地文件系统.因此，Airflow + Docker 组合是不可避免的吗?(似乎是个大障碍；一个人只能使用云中的文件).
I am a newbie as far as both Airflow and Docker are concerned; to make things more complicated, I use Astronomer, and to make things worse, I run Airflow on Windows. (Not on a Unix subsystem - could not install Docker on Ubuntu 20.4). "astro dev start" breaks with an error, but in Docker Desktop I see, and can start, 3 Airflow-related containers. They see my DAGs just fine, but my DAGs don't see the local file system. Is thus unavoidable with the Airflow + Docker combo? (Seems like a big handicap; one can only use a file in the cloud).
推荐答案
通常，您可以在 Docker 中使用 -v 开关和 docker run<在图像运行时声明卷/code> 命令将主机上的本地文件夹挂载到容器中的挂载点，您可以从容器内部访问该点.如果你继续使用 docker-compose up 来编排你的容器，你可以 在 docker-compose.yml 文件中为您的容器指定卷，为运行的容器配置卷.
In general, you can declare a volume at image runtime in Docker using the -v switch with your docker run command to mount a local folder on your host to a mount point in your container, and you can access that point from inside the container.
If you go on to use docker-compose up to orchestrate your containers, you can specify volumes in the docker-compose.yml file for your containers which configures the volumes for the containers that run.
就您而言，天文学家文档这里建议可以在 Astronomer docker-compose.override.yml 文件中创建一个自定义指令，以在作为 astro<一部分创建的 Airflow 容器中安装卷/code> 堆栈的命令，然后应该可以从 DAG 中看到这些命令.
In your case, the Astronomer docs here suggest it is possible to create a custom directive in the Astronomer docker-compose.override.yml file to mount the volumes in the Airflow containers created as part of your astro commands for your stack which should then be visible from your DAGs.
这篇关于在 Docker 容器中运行的 Airflow 可以访问本地文件吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
