从气流KubernetesPodOperator访问Kubernetes秘密 [英] Accessing Kubernetes Secret from Airflow KubernetesPodOperator

问题描述

我正在Google Cloud Composer上设置一个Airflow环境以进行测试.我已经在我的命名空间中添加了一些秘密，它们显示得很好:

I'm setting up an Airflow environment on Google Cloud Composer for testing. I've added some secrets to my namespace, and they show up fine:

$ kubectl describe secrets/eric-env-vars
Name:         eric-env-vars
Namespace:    eric-dev
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
VERSION_NUMBER:  6 bytes

我已经在我的DAG定义文件中引用了这个秘密(为简洁起见，省略了一些代码):

I've referenced this secret in my DAG definition file (leaving out some code for brevity):

env_var_secret = Secret(
    deploy_type='env',
    deploy_target='VERSION_NUMBER',
    secret='eric-env-vars',
    key='VERSION_NUMBER',
)

dag = DAG('env_test', schedule_interval=None, start_date=start_date)

operator = KubernetesPodOperator(
    name='k8s-env-var-test',
    task_id='k8s-env-var-test',
    dag=dag,
    image='ubuntu:16.04',
    cmds=['bash', '-cx'],
    arguments=['env'],
    config_file=os.environ['KUBECONFIG'],
    namespace='eric-dev',
    secrets=[env_var_secret],
)

但是当我运行此DAG时，没有打印出VERSION_NUMBER env var.它似乎也没有正确地链接到Pod(对不精确的语言表示歉意，我对Kubernetes和Airflow都是陌生的).这是来自广告连播创建响应的Airflow任务日志(也为简洁/可读性而设置):

But when I run this DAG, the VERSION_NUMBER env var isn't printed out. It doesn't look like it's being properly linked to the pod either (apologies for imprecise language, I am new to both Kubernetes and Airflow). This is from the Airflow task log of the pod creation response (also formatted for brevity/readability):

'env': [
{
    'name': 'VERSION_NUMBER',
    'value': None,
    'value_from': {
        'config_map_key_ref': None,
        'field_ref': None,
        'resource_field_ref': None,
        'secret_key_ref': {
            'key': 'VERSION_NUMBER',
            'name': 'eric-env-vars',
            'optional': None}
        }
    }
]

我假设我们以某种方式将Secret的构造函数称为错误，但是我不确定.指导表示赞赏！

I'm assuming that we're somehow calling the constructor for the Secret wrong, but I am not entirely sure. Guidance appreciated!

推荐答案

原来是对日志的误解！

当通过Secret向Kubernetes容器提供环境变量时，API响应中的value键为None，因为该值来自secret_key_ref.

When providing an environment variable to a Kubernetes pod via a Secret, that value key in the API response is None because the value comes from the secret_key_ref.

这篇关于从气流KubernetesPodOperator访问Kubernetes秘密的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！