.dockercfg文件应如何托管在AWS上的Mesosphere设置中，以便只有Mesosphere可以使用它? [英] How should a .dockercfg file be hosted in a Mesosphere-on-AWS setup so that only Mesosphere can use it?
问题描述
我们已经在私有VPC中与AWS上的Mesosphere建立了测试集群.我们有一些公开的Docker映像，这些映像很容易部署.但是，我们的大多数服务都是私有映像，托管在Docker Hub私有计划上，并且需要进行身份验证才能访问.
We have set up a test cluster with Mesosphere on AWS, in a private VPC. We have some Docker images which are public, which are easy enough to deploy. However most of our services are private images, hosted on the Docker Hub private plan, and require authentication to access.
Mesosphere能够进行私有注册表身份验证，但是它以不完全理想的方式实现了此目的:需要在所有Mesos/Marathon任务定义中指定指向.dockercfg文件的HTTPS URI.
Mesosphere is capable of private registry authentication, but it achieves this in a not-exactly-ideal way: a HTTPS URI to a .dockercfg file needs to be specified in all Mesos/Marathon task definitions.
顾名思义，问题基本上是:应如何将.dockercfg文件托管在AWS内，以便尽可能严格地将访问限制为仅对Mesos主+从服务器进行限制?
As the title suggests, the question is basically: how should the .dockercfg file be hosted within AWS so that access may be restricted to only the Mesos master+slaves as tightly as possible?
推荐答案
由于Mesos文档在此方面很差，因此，我将回答这种Wiki样式并随时更新此答案.
Since the Mesos docs are pretty poor on this, I'm going to answer this wiki-style and update this answer as I go.
在S3上托管.dockercfg文件.为了提高安全性，您应该考虑将其放在自己的存储桶中，或者放在专用于存储机密信息的存储桶中.这给创建安全策略带来了一些有趣的挑战，该策略实际上将锁定S3存储桶，以便只有Mesos可以看到它，但是可以做到.
Host the .dockercfg file on S3. For better security, you should consider putting it in its own bucket, or otherwise a bucket dedicated to storing secrets. This presents some interesting challenges in creating a security policy that will actually work to lock the S3 bucket down such that only Mesos can see it, but it can be done.
Mesos任务配置:
Mesos task configuration:
{
...
"uris": ["https://s3-eu-west-1.amazonaws.com/my-s3-bucket-name/.dockercfg"]
...
}
S3存储桶策略(使用VPC端点):
S3 bucket policy (using a VPC Endpoint):
注意:该策略允许所允许的主体执行任何对生产来说太草率的事情，但在测试集群中进行调试时应该有所帮助.
Note: this policy lets the allowed principal do anything, which is too sloppy for production, but should help when debugging in a test cluster.
{
"Id": "Policy123456",
"Version": "2012-10-17",
"Statement": [{
"Sid": "Stmt123456",
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::my-s3-bucket",
"arn:aws:s3:::my-s3-bucket/*"
],
"Condition": {
"StringEquals": {
"aws:sourceVpce": "vpce-my-mesos-cluster-vpce-id"
}
},
"Principal": "*"
}]
}
您还需要VPCE配置，以便为您提供VPCE ID以便插入上述S3存储桶条件. (我想如果您不使用VPC端点，您可以只在VPC ID上进行匹配?)
You'll also need a VPCE configuration, to give you a VPCE ID to plug into the S3 bucket condition above. (I guess if you don't use VPC endpoints you could just match on a VPC id instead?)
您可以通过转到Mesos UI(如果使用的是DCOS，这不是漂亮的DCOS UI)并观察带有应用名称的任务是否出现在活动任务"或已完成任务"中，来检查此操作是否正常列表.
You can check whether this is working by going to the Mesos UI (if you are using DCOS, this is not the pretty DCOS UI) and observing whether tasks with the name of your app appear in either the Active Tasks or Completed Tasks lists.
在此S3变体中，我们不是使用基于网络的访问限制，而是使用指向.dockercfg文件的签名URL.
In this S3 variant, rather than use networking-based access restrictions, we use a signed URL to the .dockercfg file instead.
Mesos任务配置应如下所示:
The Mesos task config should look like:
{
...
"uris": ["https://my-s3-bucket/.dockercfg?AWSAccessKeyId=foo&Expires=bar&Signature=baz"]
...
}
不幸的是，由于不起作用 > Mesos-1686 观察到任何下载的文件都精确保留了远程文件名，包括查询字符串，导致文件名类似".dockercfg?AWSAccessKeyId = foo&Expires = bar& Signature = baz".由于Docker客户端无法识别该文件，除非该文件被精确命名为".dockercfg"，否则它将无法看到身份验证凭据.
Unfortunately the above S3 signed URL strategy does not work due to Mesos-1686 which observes that any downloaded file retains the remote filename exactly, including the query string, leading to a filename like ".dockercfg?AWSAccessKeyId=foo&Expires=bar&Signature=baz". Since the Docker client does not recognise the file unless it is exactly named ".dockercfg" it fails to see the auth credentials.
可以将.dockercfg SCP锁定到每个Mesos从站.虽然这是一个快速修复，但它:
One could SCP the .dockercfg to each Mesos slave. While this is a quick fix, it:
- 需要事先了解所有奴隶
- 在将新的从属服务器添加到集群时不会扩展
- 需要对从属服务器进行SSH访问，这些从属服务器是在其自己的VPC中设置的(因此，它们的IP地址通常在10.0.[blah]范围内).
如果使用诸如Chef之类的配置管理工具自动化该方法，它将变成一种更可行的生产方法，该工具将在从属服务器上运行，并将.dockercfg文件拖到正确的位置.
This could be turned into a more viable production approach if automated with a Configuration Management tool like Chef, which would run on the slaves, and pull the .dockercfg file in to the right place.
这将导致如下配置:
{
...
"uris": ["file:///home/core/.dockercfg"]
...
}
由于'core'是基于CoreOS的Mesos从站上的默认用户，因此按惯例，.dockercfg应该位于要使用Docker的当前用户的主目录中.
Since 'core' is the default user on the CoreOS based Mesos slaves, and the .dockercfg is expected by convention to be in the home directory of the current user that wants to use Docker.
更新:这应该是最可靠的方法，但是我还没有找到一种方法.就马拉松而言，该应用程序始终处于部署"阶段.
Update: this should have been the most reliable approach, but I have not found a way to do it yet. the app is still eternally stuck in the 'Deploying' phase as far as Marathon is concerned.
由于我们正在处理用户名和密码，因此，AWS Key Management Service(或什至是CloudHSM)似乎是个好主意-但是AFAIK Mesos没有对此的内置支持，不是处理单个变量而是文件.
As we are dealing with usernames and passwords, the AWS Key Management Service (or even CloudHSM at the extreme) thing seems like it should be a good idea - but AFAIK Mesos has no built-in support for this, and we are not handling individual variables but a file.
设置完您选择的解决方案后，您可能会发现.dockercfg文件已被下拉正常，但您的应用仍停留在部署"阶段.检查这些东西...
After you have set up your solution of choice, you may find that the .dockercfg file is being pulled down OK but your app is still stuck in the 'Deploying' phase. Check these things...
在某些时候，"auth"字段的格式已更改.如果您提供的.dockercfg与该格式不匹配，则docker pull将静默失败.集群从属服务器上的Mesos Docker版本期望的格式为:
At some point, the format for the 'auth' field was changed. If the .dockercfg you supply doesn't match this format then the docker pull will silently fail. The format that the Mesos Docker version on the cluster slaves expects is:
{
"https://index.docker.io/v1/": {
"auth": [base64 of the username:password],
"email": "your_docker_registry_user@yourdomain.com"
}
}
请勿将端口80用于您的应用
如果您尝试部署Web应用程序，请确保未使用主机端口80-它没有写在文档中的任何位置，但是Mesos Web服务本身需要端口80，如果尝试将80用于您自己的应用程序将永远挂起.精明的读者会注意到，除其他原因外，这就是为什么Mesosphere"Oinker" Web应用绑定到端口0的稍微不同寻常的选择的原因.
Do not use port 80 for your app
If you are trying to deploy a Web app, make sure you did not use the host port 80 - it's not written anywhere in the docs, but Mesos Web services require port 80 for themselves, and if you try and take 80 for your own app it will just hang forever. The astute reader will notice that, among other reasons, this is why the Mesosphere "Oinker" Web app binds to the slightly unusual choice of port 0 instead.
这篇关于.dockercfg文件应如何托管在AWS上的Mesosphere设置中，以便只有Mesosphere可以使用它?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
