gsutil不能用于Google Cloud Storage的存储桶名称使用通配符吗? [英] Can't use wildcards for bucket names with gsutil for Google Cloud Storage?

问题描述

问题:通配符可以与gsutil一起用于GCS存储桶名称吗?

Question: can wildcards be used in GCS bucketnames with gsutil?

我想使用在存储桶之间分割的通配符在GCS中抓取多个文件.但是，当在带有gsutil的存储桶名称中使用通配符时，我总是遇到错误.我正在使用这样的通配符:

I want to grab multiple files in GCS using wildcards that are split across buckets. But, I'm consistently running into errors when using wildcards in bucket names with gsutil. I'm using wildcards like this:

gsutil ls gs://myBucket-abcd-*/log/data_*

我要匹配所有这些文件名(存储桶名称和对象名称中的变量):

I want to match all these file names (variations in bucket name AND in object name):

gs://myBucket-abcd-1234/log/data_foo.csv
gs://myBucket-abcd-1234/log/data_bar.csv
gs://myBucket-abcd-5678/log/data_foo.csv
gs://myBucket-abcd-5678/log/data_bar.csv

有关存储桶通配符的文档告诉我应该能够在存储区名称和对象名称中都使用通配符，但是上面的代码示例始终获得"BadRequestException:400 Invalid parameter".

Documentation on Bucket Wildcards tells me I should be able to use wildcards both in the bucketname and object name, but the code sample above always gets "BadRequestException: 400 Invalid argument."

gsutil可以正常工作.但是，在存储桶名称中添加通配符会导致错误.是否有变通方法可以使通配符在存储桶名称中起作用，还是我误解了链接的文档?

gsutil is otherwise working when I use no wildcards or use wildcards in the object name only. But adding a wildcard to the bucket name results in the error. Are there workarounds to make the wildcard work in bucket names, or am I misinterpreting the linked documentation?

推荐答案

发现在这种情况下，无法使用存储桶通配符正在按预期方式工作，并且是由于权限设置不同所致.可以在存储桶级和项目级设置Google Cloud Storage权限.

Found that not being able to use bucket wildcards in this case is working as intended, and is due to differences in permission settings. Google Cloud Storage permissions can be set at both bucket and project levels.

尽管在这种情况下使用的访问令牌可以访问每个单独的存储桶，但它没有对顶层项目(系统的许多用户共享)的读取器/编辑器/所有者访问权限.如果无法访问该项目，则不能在存储桶上使用通配符.

Though the access token used in this case can access every individual bucket, it doesn't have reader/editor/owner access to the top-level project (shared across many users of the system). Without access to the project, wildcards cannot be used on buckets.

这可以通过让项目所有者将用户添加为项目的读取者/编辑者/所有者来解决.

This can be fixed by having a project owner add the user as a reader/editor/owner to the project.

在这种情况下，出于安全原因，我们无法授予单个令牌访问项目中所有存储桶的权限，但这有助于理解通配符为何不起作用的原因.感谢所有人的投入，尤其是Travis的联系.

In this case, for security reasons we can't give an individual token access to all buckets in the project, but its helpful to understand why the wildcard didn't work. Thanks all for the input, and especially Travis for the contact.

这篇关于gsutil不能用于Google Cloud Storage的存储桶名称使用通配符吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！