Docker从文件组成构建时间参数 [英] Docker compose build time args from file
问题描述
我知道可用的变量替换，可以在项目的根目录使用.env来完成，但是在这种情况下，我要适应现有的项目，其中现有的.env文件位置是预期的，我想避免必须在多个文件上包含var项！
有关更多信息，请参见文档 ，并且所有代码都可以在仓库的docker-support分支上以WIP的形式获得，但是我将在下面简要地描述该项目和问题:
项目结构
|- root
| |- .env # mongo and mongo-express vars (not on git!)
| |- docker-compose.yaml # build and ups a staging env
| |- docker-compose.prod.yaml # future wip
| |- api # the saas-api service
| |- Dockerfile # if 'docked' directly should build production
| |- .env # api relative vars (not on git!)
| |- app # the saas-app service
| |- Dockerfile # if 'docked' directly should build production
| |- .env # api relative vars (not on git!)
或者在此处看到整个内容，目前，还是这样，但是saas-app在构建到目前为止我可以识别的用于登台/生产的图像时存在一个问题.
问题
在构建时，Next.js使用webpack来构建页面的静态版本，以完成process.env替换的工作，因此它需要在docker构建阶段包括最终实际运行的var，因此next.js不需要在运行时再次重建，也可以在流量需要时安全地生成多个实例！
我知道，如果在运行时未发送相同的var，则必须重新构建，这违背了本练习的要点，但这正是我要在此处避免的，如果发送了错误的值这是我们的事，而不是项目的事！
我还需要考虑Next.js BUILD ID管理，但这是另一个时间/问题.
尝试
我一直在测试是否在应用程序的已粘贴完整的docker-compose config输出，并带有存储库中的现有版本:
理想情况下，我想要:
saas-app:
build:
args:
LOG_LEVEL: notice
NODE_ENV: development
PORT: '3000'
context: /home/pedro/src/opensource/saas-boilerplate/app
command: yarn start
container_name: saas-app
depends_on:
- saas-api
environment:
...
成为:
saas-app:
build:
args:
LOG_LEVEL: notice
NODE_ENV: development
PORT: '3000'
BUCKET_FOR_POSTS: xxxxxx
BUCKET_FOR_TEAM_AVATARS: xxxxxx
GA_TRACKING_ID: ''
LAMBDA_API_ENDPOINT: xxxxxxapi
NODE_ENV: development
STRIPEPUBLISHABLEKEY: pk_test_xxxxxxxxxxxxxxx
URL_API: http://api.saas.localhost:8000
URL_APP: http://app.saas.localhost:3000
context: /home/pedro/src/opensource/saas-boilerplate/app
command: yarn start
container_name: saas-app
depends_on:
- saas-api
environment:
...
问题
如果可能的话，我将如何实现这一目标，但是:
- 无需将现有的
.env文件合并到单个根目录中，也不必在多个文件上复制var. - 无需手动在撰写文件中声明值，也不必在命令中推断出它们的值，例如
docker-compose build --build-arg GA_TRACKING_ID=UA-xXxXXXX-X? - 在
.env文件nofollow noreferrer>构建阶段，因为它感觉不正确和/或不安全? - 在我看来，对于撰写团队来说，撰写
build选项功能请求中的args_file似乎是有效的，您也可以这样说吗? - 或者在撰写文件上具有根选项，您可以在其中为多个变量替换设置多个
.env文件? - 还是我看不到的另一种解决方案?有什么想法吗?
- 我不介意将每个
.env文件作为 config 或秘密，它是比拆分组成文件更干净的解决方案，有人在运行这样的示例进行生产吗?
我设法达成了一个妥协，既不影响任何现有的开发工作流程，也不允许 app 用于没有env变量的构建(对生产构建来说，这一要求将更为关键).
我基本上已经决定重用docker的内部功能来读取.env文件，并将其用于组成文件中的变量替换，这是一个示例:
# compose
COMPOSE_TAG_NAME=stage
# common to api and app (build and run)
LOG_LEVEL=notice
NODE_ENV=development
URL_APP=http://app.saas.localhost:3000
URL_API=http://api.saas.localhost:8000
API_PORT=8000
APP_PORT=3000
# api (run)
MONGO_URL=mongodb://saas:secret@saas-mongo:27017/saas
SESSION_NAME=saas.localhost.sid
SESSION_SECRET=3NvS3Cr3t!
COOKIE_DOMAIN=.saas.localhost
GOOGLE_CLIENTID=
GOOGLE_CLIENTSECRET=
AMAZON_ACCESSKEYID=
AMAZON_SECRETACCESSKEY=
EMAIL_SUPPORT_FROM_ADDRESS=
MAILCHIMP_API_KEY=
MAILCHIMP_REGION=
MAILCHIMP_SAAS_ALL_LIST_ID=
STRIPE_TEST_SECRETKEY=
STRIPE_LIVE_SECRETKEY=
STRIPE_TEST_PUBLISHABLEKEY=
STRIPE_LIVE_PUBLISHABLEKEY=
STRIPE_TEST_PLANID=
STRIPE_LIVE_PLANID=
STRIPE_LIVE_ENDPOINTSECRET=
# app (build and run)
STRIPEPUBLISHABLEKEY=
BUCKET_FOR_POSTS=
BUCKET_FOR_TEAM_AVATARS=
LAMBDA_API_ENDPOINT=
GA_TRACKING_ID=
请参阅更新的 docker-compose.yml 我还使用了扩展字段确保在构建和运行过程中仅发送正确有效的var.
它从问题上打破了规则1.，但我觉得这是一个足够好的折衷，因为它不再依赖于其他.env文件，无论如何大多数时候它都可能成为开发的关键！
不幸的是，如果将来vars发生变化，我们将需要维护compose文件，并且必须将相同的.env文件用于生产版本，但是由于这可能是在某些CI/CD上外部完成的，不用担心.
我正在发布此信息，但尚未完全解决问题，如果其他人可以提出更好的主意，我将不胜感激.
I'm aware of the variable substitutions available, where I could use a .env at the root of the project and that would be done, but in this case I'm adapting an existing project, where existing .env file locations are expected and I would like to prevent having to have var entries on multiple files!
See documentation for more info, and all the code is available as WIP on the docker-support branch of the repo, but I'll succinctly describe the project and issue below:
Project structure
|- root
| |- .env # mongo and mongo-express vars (not on git!)
| |- docker-compose.yaml # build and ups a staging env
| |- docker-compose.prod.yaml # future wip
| |- api # the saas-api service
| |- Dockerfile # if 'docked' directly should build production
| |- .env # api relative vars (not on git!)
| |- app # the saas-app service
| |- Dockerfile # if 'docked' directly should build production
| |- .env # api relative vars (not on git!)
Or see the whole thing here, it works great by the way for the moment, but there's one problem with saas-app when building an image for staging/production that I could identify so far.
Issue
At build time Next.js builds a static version of the pages using webpack to do it's thing about process.env substitution, so it requires the actual eventual running vars to be included at docker build stage so next.js doesnt need to rebuild again at runtime and also so that I can safely spawn multiple instances when traffic requires!
I'm aware that if at runtime the same vars are not sent it will have to rebuild again defying the point of this exercise, but that's precisely what I'm trying to prevent here, to that if the wrong values are sent it's on us an not the project!
And I also need to consider Next.js BUILD ID managemement, but that's for another time/question.
Attempts
I've been testing with including the ARG and ENV declarations for each of the variables expected by the app on it's Dockerfile, e.g.:
ARG GA_TRACKING_ID=
ENV GA_TRACKING_ID ${GA_TRACKING_ID}
This works as expected, however it forces me to manually declare them on the docker-compose.yml file, which is not ideal:
saas-app:
build:
context: app
args:
GA_TRACKING_ID: UA-xXxXXXX-X
I cannot use variable substitution here because my root .env does not include this var, it's on ./app/.env, and I also tested leaving the value empty but it is not picking it up from the env_file or enviroment definitions, which I believe is as expected.
I've pastbinned a full output of docker-compose config with the existing version on the repository:
Ideally, I'd like:
saas-app:
build:
args:
LOG_LEVEL: notice
NODE_ENV: development
PORT: '3000'
context: /home/pedro/src/opensource/saas-boilerplate/app
command: yarn start
container_name: saas-app
depends_on:
- saas-api
environment:
...
To become:
saas-app:
build:
args:
LOG_LEVEL: notice
NODE_ENV: development
PORT: '3000'
BUCKET_FOR_POSTS: xxxxxx
BUCKET_FOR_TEAM_AVATARS: xxxxxx
GA_TRACKING_ID: ''
LAMBDA_API_ENDPOINT: xxxxxxapi
NODE_ENV: development
STRIPEPUBLISHABLEKEY: pk_test_xxxxxxxxxxxxxxx
URL_API: http://api.saas.localhost:8000
URL_APP: http://app.saas.localhost:3000
context: /home/pedro/src/opensource/saas-boilerplate/app
command: yarn start
container_name: saas-app
depends_on:
- saas-api
environment:
...
Questions
How would I be able to achieve this, if possible, but:
- Without merging the existing
.envfiles into a single root, or having to duplicate vars on multiple files. - Without manually declaring the values on the compose file, or having to infer them on the command e.g.
docker-compose build --build-arg GA_TRACKING_ID=UA-xXxXXXX-X? - Without having to
COPYeach.envfile during the build stage, because it doesn't feel right and/or secure? - Maybe a
args_fileon the composebuildoptions feature request for the compose team seems to me to be a valid, would you also say so? - Or perhaps have a root option on the compose file where you could set more than one
.envfile for variable substituion? - Or perhaps another solution i'm not seeing? Any ideas?
- I wouldn't mind sending each
.envfile as a config or secret, it's a cleaner solution than splitting the compose files, is anyone running such an example for production?
I've managed to achieve a compromise that does not affect any of the existing development workflows, nor does it allow for app to build without env variables (a requirement that will be more crucial for production builds).
I've basically decided to reuse the internal ability of docker to read the .env file and use those in variable substitution on the compose file, here's an example:
# compose
COMPOSE_TAG_NAME=stage
# common to api and app (build and run)
LOG_LEVEL=notice
NODE_ENV=development
URL_APP=http://app.saas.localhost:3000
URL_API=http://api.saas.localhost:8000
API_PORT=8000
APP_PORT=3000
# api (run)
MONGO_URL=mongodb://saas:secret@saas-mongo:27017/saas
SESSION_NAME=saas.localhost.sid
SESSION_SECRET=3NvS3Cr3t!
COOKIE_DOMAIN=.saas.localhost
GOOGLE_CLIENTID=
GOOGLE_CLIENTSECRET=
AMAZON_ACCESSKEYID=
AMAZON_SECRETACCESSKEY=
EMAIL_SUPPORT_FROM_ADDRESS=
MAILCHIMP_API_KEY=
MAILCHIMP_REGION=
MAILCHIMP_SAAS_ALL_LIST_ID=
STRIPE_TEST_SECRETKEY=
STRIPE_LIVE_SECRETKEY=
STRIPE_TEST_PUBLISHABLEKEY=
STRIPE_LIVE_PUBLISHABLEKEY=
STRIPE_TEST_PLANID=
STRIPE_LIVE_PLANID=
STRIPE_LIVE_ENDPOINTSECRET=
# app (build and run)
STRIPEPUBLISHABLEKEY=
BUCKET_FOR_POSTS=
BUCKET_FOR_TEAM_AVATARS=
LAMBDA_API_ENDPOINT=
GA_TRACKING_ID=
See the updated docker-compose.yml I've also made use of Extension fields to make sure only the correct and valid vars are sent across on build and run.
It breaks rule 1. from the question, but I feel it's a good enough compromise, because it no longer relies on the other .env files, that would potentically be development keys most of the time anyway!
Unfortunately we will need to mantain the compose file if the vars change in the future, and the same .env file has to be used for a production build, but since that will probably be done externally on some CI/CD, that does not worry much.
I'm posting this but not fully closing the question, if anyone else could chip in with a better idea, I'd be greatly appreciated.
这篇关于Docker从文件组成构建时间参数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
