带有Java服务帐户的Google云端存储 - 403 Caller没有对存储桶的storage.objects.list访问权限 [英] Google Cloud Storage with a service account in Java - 403 Caller does not have storage.objects.list access to bucket

问题描述

我们希望从我们的应用服务器中的Google Storage下载文件。重要的是对单个存储桶进行只读限制访问，而不是其他任何操作。

We want to download files from Google Storage in our application server. It is important to have read-only restricted access to a single bucket and nothing else.

首先，我使用了具有权限的常规用户帐户（不是服务帐户）访问我们的Google Cloud项目中的所有存储桶，一切正常 - 我的Java代码打开存储桶并下载文件没有问题。

At first I used a regular user account (not a service account) which have permissions to access all buckets in our Google Cloud project, and everything worked fine - my Java code opened buckets and downloaded files without problems.

Storage storage = StorageOptions.getDefaultInstance().getService();
Bucket b = storage.get( "mybucketname" );

然后我想切换到使用专门创建的服务帐户，该帐户只能访问一个存储桶。所以我创建了一个服务帐户，授予读取单个存储桶的权限，并下载了其密钥文件。 Google Cloud Console中的权限命名为：

Then I wanted to switch to use a specially created service account which has access to a single bucket only. So I created a service account, gave permissions to read a single bucket, and downloaded its key file. The permissions in Google Cloud Console are named as:

存储对象查看器（3个成员）读取对GCS对象的访问权限。

Storage Object Viewer (3 members) Read access to GCS objects.

gsutil命令行实用程序可以正常使用此帐户 - 从命令行可以访问此存储桶但不能访问其他存储桶。

gsutil command line utility works fine with this account - from the command line it allows accessing this bucket but not the others.

使用以下命令从命令行初始化：

The initialization from the command line is done using the following command:

gcloud --project myprojectname auth activate-service-account [e-mail] --key-file=/.../keyfilename.json

我甚至尝试了两个可以访问不同存储桶的不同服务帐户，从命令行我可以在它们之间切换，gsutil只能访问相关的存储桶，对于任何其他它返回错误：

I even tried two different service accounts which have access to different buckets, and from the command line I can switch between them and gsutil gives access to a relevant bucket only, and for any other it returns the error:

AccessDeniedException：403调用者没有对存储桶xxxxxxxxxx的storage.objects.list访问权限。

"AccessDeniedException: 403 Caller does not have storage.objects.list access to bucket xxxxxxxxxx."

因此，从命令行开始，一切正常。
但在Java中，身份验证存在一些问题。

So, from the command line everything worked fine. But in Java there is some problem with the authentication.

我之前使用普通用户帐户的默认身份验证停止工作 - 它报告错误：

The default authentication I previously used with a regular user account stopped working - it reports the error:

com.google.cloud.storage.StorageException：匿名用户没有对bucket xxxxxxxxxx的storage.buckets.get访问权限。

com.google.cloud.storage.StorageException: Anonymous users does not have storage.buckets.get access to bucket xxxxxxxxxx.

然后我尝试了以下代码（这是最简单的变体，因为它依赖于密钥json文件，但我已经尝试了一个数字在各种论坛中发现的其他变种，没有成功）：

Then I've tried the following code (this is the simplest variant because it relies on the key json file, but I've already tried a number of other variants found in various forums, with no success):

FileInputStream fis = new FileInputStream( "/path/to/the/key-file.json" );
ServiceAccountCredentials credentials = ServiceAccountCredentials.fromStream( fis );
Storage storage = StorageOptions.newBuilder().setCredentials( credentials )
    .setProjectId( "myprojectid" ).build().getService();
Bucket b = storage.get( "mybucketname" );

我收到的是这个错误：

com.google.cloud.storage.StorageException：调用者没有对bucket mybucketname的storage.buckets.get访问权限。
原因：com.google.api.client.googleapis.json.GoogleJsonResponseException：403 Forbidden

com.google.cloud.storage.StorageException: Caller does not have storage.buckets.get access to bucket mybucketname. Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden

同样的错误是无论我想要访问什么桶（甚至不存在）都返回。
令我困惑的是，使用相同的JSON密钥文件初始化的相同服务帐户在命令行中运行良好。
所以我认为Java代码中缺少某些东西来确保正确的身份验证。

The same error is returned no matter to what buckets I'm trying to access (even non-existing). What confuses me is that the same service account, initialized with the same JSON key file, works fine from the command line. So I think something is missing in Java code that ensures correct authentication.

推荐答案

TL; DR - 如果您使用的是 应用程序默认凭据 （当你执行 StorageOptions.getDefaultInstance（）。getService（）; ）时你就是BTW，如果你需要使用来自服务帐户的凭据，您可以而不更改代码。您只需将 GOOGLE_APPLICATION_CREDENTIALS 环境变量设置为服务帐户json文件的完整路径，即可完成设置。

TL;DR - If you're using Application Default Credentials (which BTW you are when you do StorageOptions.getDefaultInstance().getService();), and if you need to use the credentials from a service account, you can do so without changing your code. All you need to do is set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the full path of your service account json file and you are all set.

• 按原样使用原始代码

• Use your original code as-is

Storage storage = StorageOptions.getDefaultInstance().getService();
Bucket b = storage.get( "mybucketname" );

设置环境变量 GOOGLE_APPLICATION_CREDENTIALS 到包含服务帐户凭据的json文件的完整路径。

Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the full path of your json file containing the service account credentials.

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service_account_credentials.json

再次运行java应用程序以验证它是否按预期工作。

Run your java application once again to verify that it is working as expected.

代码示例您发布初始化 ServiceAccountCredentials 对我来说很快就会对我有效。我尝试了以下代码片段，它按预期正在为我工​​作。

The code example you posted for initializing ServiceAccountCredentials looks valid to me on a quick glance. I tried the following code snippet and it is working for me as expected.

String SERVICE_ACCOUNT_JSON_PATH = "/path/to/service_account_credentials.json";

Storage storage =
    StorageOptions.newBuilder()
        .setCredentials(
            ServiceAccountCredentials.fromStream(
                new FileInputStream(SERVICE_ACCOUNT_JSON_PATH)))
        .build()
        .getService();
Bucket b = storage.get("mybucketname");

指定服务帐户凭证时，项目ID会自动从json文件中的信息中获取。所以你不必再次指定它。我不完全确定这是否与您正在观察的问题有关。

When specifying a service account credential, the project ID is automatically picked up from the information present in the json file. So you do not have to specify it once again. I'm not entirely sure though if this is related to the issue you're observing.

这是关于 https://developers.google.com/identity/protocols/application-default-credentialsrel =noreferrer> 应用程序默认凭据 ，说明哪些凭据是根据您的环境选择。

Here is the full documentation regarding Application Default Credentials explaining which credentials are picked up based on your environment.

应用程序默认凭据如何工作

您可以通过创建单个客户端$ b来获取应用程序默认凭据$ b库调用。返回的凭据由代码运行的
环境确定。条件在
下面的订单中检查：

How the Application Default Credentials work

You can get Application Default Credentials by making a single client library call. The credentials returned are determined by the environment the code is running in. Conditions are checked in the following order:

1. 选中环境变量 GOOGLE_APPLICATION_CREDENTIALS 。如果指定了此变量，则应指向
   定义凭据的文件。为此
   目的获取凭据的最简单方法是在Google API控制台中创建服务帐户密钥：

1. The environment variable GOOGLE_APPLICATION_CREDENTIALS is checked. If this variable is specified it should point to a file that defines the credentials. The simplest way to get a credential for this purpose is to create a Service account key in the Google API Console:

a。转到 API控制台凭据页。

b。从项目下拉列表中选择您的项目。

b. From the project drop-down, select your project.

c。在凭据页面上，选择创建凭据下拉菜单，然后选择服务帐户密钥。

c. On the Credentials page, select the Create credentials drop-down, then select Service account key.

d。从服务帐户下拉菜单中选择一个现有服务
帐户或创建一个新帐户。

d.From the Service account drop-down, select an existing service account or create a new one.

e。对于Key type，选择JSON键选项，然后选择Create。
文件会自动下载到您的计算机。

e. For Key type, select the JSON key option, then select Create. The file automatically downloads to your computer.

f。将刚下载的* .json文件放在
选择的目录中。此目录必须是私有的（您不能让任何人获得
访问权限），但可以访问您的Web服务器代码。

f. Put the *.json file you just downloaded in a directory of your choosing. This directory must be private (you can't let anyone get access to this), but accessible to your web server code.

g。将环境变量 GOOGLE_APPLICATION_CREDENTIALS 设置为
下载的JSON文件的路径。

g. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the JSON file downloaded.

如果您已在计算机上安装了Google Cloud SDK并运行了命令 gcloud auth application-default login ，您的
身份可以用作代理来测试从
机器调用API的代码。

If you have installed the Google Cloud SDK on your machine and have run the command gcloud auth application-default login, your identity can be used as a proxy to test code calling APIs from that machine.

如果您正在使用Google App Engine生产，则将使用与该应用程序关联的内置服务帐户。

If you are running in Google App Engine production, the built-in service account associated with the application will be used.

如果您在Google Compute Engine生产中运行，将使用与虚拟机实例
关联的内置服务帐户。

If you are running in Google Compute Engine production, the built-in service account associated with the virtual machine instance will be used.

如果这些条件都不成立，则会发生错误。

If none of these conditions is true, an error will occur.

IAM角色

我会推荐goi了解 IAM权限 和 IAM角色 可用于云存储。这些提供项目和桶级别的控制。此外，您可以使用 ACL来控制存储桶中对象级别的权限。

• 如果您的用例只涉及调用 storage.get（bucketName）。此操作只需要 storage.buckets.get 权限，此权限的最佳IAM角色是 roles / storage.legacyObjectReader 。

• If your use case involves just invoking storage.get(bucketName). This operation will require just storage.buckets.get permission and the best IAM role for just this permission is roles/storage.legacyObjectReader.

如果您还想授予服务帐户获取权限（ storage.objects.get ）并列出（ storage.objects.list ）单个对象，然后还添加角色 roles / storage.objectViewer 到服务帐户。

If you also want to grant the service account permissions to get (storage.objects.get) and list (storage.objects.list) individual objects, then also add the role roles/storage.objectViewer to the service account.

这篇关于带有Java服务帐户的Google云端存储 - 403 Caller没有对存储桶的storage.objects.list访问权限的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！