使用 Workload Identity 对 Kubernetes 上的 Cloud ML Engine 容器中的独立 gsutil 进行身份验证 [英] Authenticating standalone gsutil in containers in Cloud ML Engine on Kubernetes with Workload Identity

问题描述

我将在 Google Cloud AI Training (Cloud ML Engine) 上启动容器映像

在这些容器中，我需要使用 gsutil.一些容器有 gsutil.在这种情况下，我可以立即使用它而无需任何身份验证步骤.

有些容器没有 gsutil，所以我必须安装它.问题是安装的 gsutil 不起作用.

当我使用官方 cloud-sdk 映像时，gsutil 无需任何身份验证步骤即可工作.

当我使用 python:3.7 映像并从 PyPI 安装 gsutil 时，它不起作用:

python -m pip install gsutil --quietgsutil cp a gs://b/c

<块引用>

ServiceException: 401 匿名调用者没有 storage.objects.get 访问权限...

如何才能让独立的 gsutil 获得所需的凭据?

大多数指南侧重于手动调用 gcloud auth、复制 URL 和复制回令牌.这不是我寻求的解决方案(应该是自动化的).我知道自动化解决方案是可能的，因为在某些图像中 gsutil 开箱即用.

解决方案

这是因为 pip install gsutil 单独没有配置凭据，这就是错误所说的匿名用户的原因.您需要配置凭据以访问受保护的数据.

将以下行放入您的 docker 文件中，它应该可以工作:

RUN echo '[GoogleCompute] service_account = default' >/etc/boto.cfg

配置 gsutil 以使用默认服务帐户.

I'm launching container images on Google Cloud AI Training (Cloud ML Engine)

Inside those containers I need to use gsutil. Some containers have gsutil. In that case I can use it right away without any authentication steps.

Some containers do not have gsutil, so I have to install it. The problem is that the installed gsutil does not work.

When I'm using the official cloud-sdk image, gsutil works without any auth steps.

When I use the python:3.7 image and install gsutil from PyPI it does not work:

python -m pip install gsutil --quiet
gsutil cp a gs://b/c

ServiceException: 401 Anonymous caller does not have storage.objects.get access to ...

How can I make it so that the standalone gsutil obtains the needed credentials?

Most guides focus on manually calling gcloud auth, copying URL and copying back the token. This is not the solution that I seek (which should be automated). I know that the automated solution is possible since in some images gsutil works out of the box.

解决方案

This is because that pip install gsutil alone does not configure the credentials, which is why it's anonymous user as the error says. You'll want to configure credentials to access protected data.

Put following line in your docker file and it should work:

RUN echo '[GoogleCompute] service_account = default' > /etc/boto.cfg

It's to configure gsutil to use the default service account.

这篇关于使用 Workload Identity 对 Kubernetes 上的 Cloud ML Engine 容器中的独立 gsutil 进行身份验证的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！