Google Cloud Composer和Google Cloud SQL代理 [英] Google Cloud Composer and Google Cloud SQL Proxy

问题描述

我有一个使用Cloud Composer和Cloud SQL的项目. 我能够连接到Cloud SQL，因为我编辑了airflow-sqlproxy-service的Yaml，并将我的Cloud SQL实例添加到用于airflow-db的云代理上，映射到端口3307.

I have a project with Cloud Composer and Cloud SQL. I am able to connect to Cloud SQL because i edited the yaml of airflow-sqlproxy-service and added my Cloud SQL instance on cloud proxy used for the airflow-db, mapping to port 3307.

工作人员可以在端口3307上连接到airflow-sqlproxy-service，但我认为Web服务器无法连接到此端口.

The workers can connect to airflow-sqlproxy-service on port 3307 but i think the webserver can't connect to this.

我是否需要添加一些防火墙规则以映射3307端口，以便Web服务器或UI可以连接到airflow-sqlproxy-service?

Do i need to add some firewall rule to map the 3307 port so the webserver or the UI can connect to airflow-sqlproxy-service?

https://i.stack.imgur.com/LwKQK.png

https://i.stack.imgur.com/CJf7Q.png

https://i.stack.imgur.com/oC2dJ.png

最诚挚的问候.

推荐答案

Composer当前不支持从Web服务器配置其他sql代理.此类情况的一种解决方法是拥有一个单独的DAG，该DAG将从另一个数据库(通过有权有权访问的工作人员)向所需的信息中加载气流变量，然后根据该变量生成DAG， Web服务器可以访问.

Composer does not currently support configuring additional sql proxies from the webserver. One workaround for cases like this is to have a separate DAG which loads Airflow Variables with the information needed from the other database (via the workers which do have access), then generate a DAG based on the Variable which the webserver can access.

https://github.com/apache/incubator-airflow/pull/4170 最近被合并(在Composer中当前不可用)，它定义了CloudSQL连接类型.将来这可能会适用于这些用例.

https://github.com/apache/incubator-airflow/pull/4170 recently got merged (not currently available in Composer), which defines a CloudSQL connection type. This might work for these use cases in the future.

这篇关于Google Cloud Composer和Google Cloud SQL代理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！