在Terraform中引用gitlab秘密 [英] Referencing gitlab secrets in Terraform

问题描述

我对Terraforms和gitlab CI还是很陌生，我在这里尝试做一些事情.

I am quite new to Terraforms and gitlab CI and there is something that I am trying to do here with it.

我想使用Terraform创建一个IAM用户和一个S3存储桶.使用策略允许该IAM用户对该S3存储桶进行某些操作.将IAM用户的凭据保存在工件中. 现在，以上内容将成为我的核心模块.

I want to use Terraform to create an IAM user and a S3 bucket. Using policies allow certain operations on this S3 bucket to this IAM user. Have the IAM user's credentials saved in the artifactory. Now the above is going to be my core module.

核心模块如下所示:

aws-s3-iam-combo.git

(用于运行所有Terraform的IAM用户的凭据，例如 admin-user ，将存储在gitlab机密中.)

(The credentials for the IAM user using which all the Terraform would be run, say admin-user, would be stored in gitlab secrets.)

main.tf

resource "aws_s3_bucket" "bucket" {
  bucket = "${var.name}"
  acl = "private"
  force_destroy = "true"

  tags {
    environment = "${var.tag_environment}"
    team        = "${var.tag_team}"
  }

  policy =<<EOF
{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "${aws_iam_user.s3.arn}"
      },
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::${var.name}",
        "arn:aws:s3:::${var.name}/*"
      ]
    }
  ]
}
EOF
}

resource "aws_iam_user" "s3" {
  name = "${var.name}-s3"
  force_destroy = "true"
}

resource "aws_iam_access_key" "s3" {
  user = "${aws_iam_user.s3.name}"
}

resource "aws_iam_user_policy" "s3_policy" {
  name = "${var.name}-policy-s3"
  user = "${aws_iam_user.s3.name}"
  policy =<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::${var.name}",
        "arn:aws:s3:::${var.name}/*"
      ]
    }
  ]
}
EOF
}

outputs.tf

output "bucket" {
  value = "${aws_s3_bucket.bucket.bucket}"
}

output "bucket_id" {
  value = "${aws_s3_bucket.bucket.id}"
}

output "iam_access_key_id" {
  value = "${aws_iam_access_key.s3.id}"
}

output "iam_access_key_secret" {
  value = "${aws_iam_access_key.s3.secret}"
}

variables.tf

variable "name" {
  type = "string"
}

variable "tag_team" {
  type = "string"
  default = ""
}

variable "tag_environment" {
  type = "string"
  default = ""
}

variable "versioning" {
  type = "string"
  default = false
}

variable "profile" {
  type = "string"
  default = ""
}

该组织中现在需要创建S3存储桶的任何人都需要创建一个新的存储库，格式如下:

Anyone in the organization who now needs to create S3 buckets, would need to create a new repo, something of the form:

main.tf

module "aws-s3-john-doe" {
  source = "git::https://git@gitlab-address/terraform/aws-s3-iam-combo.git?ref=v0.0.1"
  name = "john-doe"
  tag_team = "my_team"
  tag_environment = "staging"
}

gitlab-ci.yml

stages:
  - plan
  - apply

plan:
  image: hashicorp/terraform
  stage: plan
  script:
    - terraform init
    - terraform plan

apply:
  image: hashicorp/terraform
  stage: apply
  script:
    - terraform init
    - terraform apply
  when: manual
  only:
    - master

然后，管道将触发，并且当此存储库合并到主库时，将创建资源(S3和IAM用户)，并且用户将具有该IAM用户的凭据.

And then the pipeline would trigger and when this repo gets merged to master, the resources (S3 and IAM user) would be created and the user would have this IAM user's credentials.

现在的问题是，我们有多个AWS账户.可以这么说，如果开发人员想要在某个帐户中创建S3，则上述设置将无法实现，因为 admin-user (其信誉属于gitlab机密)仅适用于一个帐户独自的.

Now the problem is that we have multiple AWS accounts. So say if a dev wants to create an S3 in a certain account, it would not be possible with the above set up as the admin-user, whose creds are in gitlab secrets, is only for one account alone.

现在我不明白如何达到我的上述要求.我有以下想法:(请提出是否有更好的方法来做到这一点)

Now I don't understand how do I achieve the above requirement of mine. I have the below idea: (Please suggest if there's a better way to do this)

1. 为每个有问题的AWS账户在gitlab机密中设置了多个不同的凭据
2. 接受用户输入，将他们想要在其中创建资源的AWS账户指定为变量.所以像这样:

main.tf

module "aws-s3-john-doe" {
  source = "git::https://git@gitlab-address/terraform/aws-s3-iam-combo.git?ref=v0.0.1"
  name = "john-doe"
  tag_team = "my_team"
  tag_environment = "staging"
  aws_account = "account1"
}

1. 然后在 aws-s3-iam-combo.git main.tf中以某种方式从gitlab机密中读取 account1 的信条.

1. And then in the aws-s3-iam-combo.git main.tf somehow read the creds for account1 from the gitlab secrets.

现在我不知道如何实现上述目标，就像我如何从gitlab中读取所需的秘密变量等.

Now I do not know how achieve the above, like how do i read from gitlab the required secret variable etc.

有人可以在这里帮忙吗?

Can someone please help here?

推荐答案

您之前问过这个问题，但是也许我的想法仍然可以帮助一个或另一个...

you asked this some time ago, but maybe my idea still helps the one or the other...

您可以使用envsubst执行此操作(要求将pkg gettext安装在运行程序或用于运行管道的Docker映像中).

You can do this with envsubst (requires the pkg gettext to be installed on your runner or in the Docker image used to run the pipeline).

这里是一个例子:

首先，在项目设置中，将不同的用户帐户设置为环境变量(项目机密:

First, in the project settings you set your different user accounts as environment variables (project secrets:

SECRET_1: my-secret-1
SECRET_2: my-secret-2
SECRET_3: my-secret-3

然后，创建一个包含Terraform变量的文件，将其命名为vars_template.tf:

Then, create a file that holds a Terraform variable, let's name it vars_template.tf:

variable "gitlab_secrets" {
    description = "Variables from GitLab"
    type = "map"
    default = {
        secret_1 = "$SECRET_1"
        secret_2 = "$SECRET_2"
        secret_3 = "$SECRET_3"
    }
}

在CI管道中，您现在可以配置以下内容:

In your CI pipeline, you can now configure the following:

plan:dev:
  stage: plan dev
  script:
    - envsubst < vars_template.tf > ./vars_envsubst.tf
    - rm vars_template.tf
    - terraform init
    - terraform plan -out "planfile_dev"
  artifacts:
    paths:
      - environments/dev/planfile_dev
      - environments/dev/vars_envsubst.tf

apply:dev:
  stage: apply dev
  script:
    - cd environments/dev
    - rm vars_template.tf
    - terraform init
    - terraform apply -input=false "planfile_dev"
  dependencies:
    - plan:dev

重要的是要注意必须删除原始的vars_template.tf，否则Terraform将抛出错误，该变量被多次定义.您可以通过将模板文件存储在Terraform工作目录之外的目录中来规避此问题. 但是从Terraform状态中，您可以看到正确替换了变量值:

It's important to note that the original vars_template.tf has to be deleted, otherwise Terraform will throw an error that the variable is defined multiple times. You could circumvent this by storing the template file in a directory which is outside the Terraform working directory though. But from the Terraform state you can see that the variable values where correctly substituted:

"outputs": {
    "gitlab_secrets": {
        "sensitive": false,
        "type": "map",
        "value": {
            "secret_1": "my-secret-1",
            "secret_2": "my-secret-2",
            "secret_3": "my-secret-3"
        }
    }
}

然后您可以在Terraform资源等中使用"${vars.gitlab_secrets["secret_1"]}"访问值.

You can then access the values with "${vars.gitlab_secrets["secret_1"]}" in your Terraform resources etc.

更新:请注意，此方法会将机密信息存储在Terraform状态文件中，如果该文件存储在S3存储桶中以与Terraform协同工作，则可能存在安全隐患.该存储桶至少应被加密.另外，建议使用ACL限制对文件的访问，例如，只有用户 terraform 可以访问它.而且，当然，用户可以通过Terraoform outputs ...

UPDATE: Note that this method will store the secrets in the Terraform state file, which can be a potential security issue if the file is stored in an S3 bucket for collaborative work with Terraform. The bucket should at least be encrypted. In addition, it's recommended to limit the access to the files with ACLs so that, e.g., only a user terraform has access to it. And, of course, a user could reveil the secrets via Terraoform outputs...

这篇关于在Terraform中引用gitlab秘密的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！