如何使用python下载谷歌云平台上文件夹内的文件? [英] How can i download the files inside a folder on google cloud platform using python?

问题描述

from google.cloud import storage
client = storage.Client()
bucket = client.get_bucket([bucket_name])
blob = bucket.get_blob([path to the .txt file])
blob.download_to_filename([local path to the downloaded .txt file])

我如何调整我的 python 代码以添加类似 for filename in os.listdir(path): 的内容，以便将特定文件夹中的所有文件复制到本地

How can i adjust my python code to add something like for filename in os.listdir(path): to just copy all the files in a certain folder on there locally

推荐答案

首先，我觉得有必要强调一下 Google Cloud Storage 使用的是平面命名空间，而实际上目录"的概念并不存在，因为 GCS 中没有存储分层文件架构.有关目录如何工作的更多信息可以在文档中找到，因此如果您对这个主题感兴趣，这是一个很好的阅读.

First of all, I think it is interesting to highlight that Google Cloud Storage uses a flat name space, and in fact the concept of "directories" does not exist, as there is no hierarchical file architecture being stored in GCS. More information about how directories work can be found in the documentation, so it is a good read if you are interested in this topic.

话虽如此，您可以使用我在下面分享的脚本，以便将 GCS 中文件夹"中的所有文件下载到本地环境中的同一文件夹中.基本上，您自己的代码中唯一重要的添加部分是 bucket.list_blobs() 方法 被调用，prefix 字段指向文件夹名称, 以便查找仅与名称中的文件夹模式匹配的 blob.然后，您遍历它们，丢弃目录 blob 本身(在 GCS 中只是名称以 "/" 结尾的 blob)，然后下载文件.

That being said, you can use a script such as the one I share below, in order to download all files in a "folder" in GCS to the same folder in your local environment. Basically, the only important addition a part from your own code is that the bucket.list_blobs() method is being called, with the prefix field pointing to the folder name, in order to look for blobs which only match the folder-pattern in their name. Then, you iterate over them, discard the directory blob itself (which in GCS is just a blob with a name ending in "/"), and download the files.

from google.cloud import storage
import os

# Instantiate a CGS client
client=storage.Client()
bucket_name= "<YOUR_BUCKET_NAME>"

# The "folder" where the files you want to download are
folder="<YOUR_FOLDER_NAME>/"

# Create this folder locally
if not os.path.exists(folder):
    os.makedirs(folder)

# Retrieve all blobs with a prefix matching the folder
bucket=client.get_bucket(bucket_name)
blobs=list(bucket.list_blobs(prefix=folder))
for blob in blobs:
    if(not blob.name.endswith("/")):
        blob.download_to_filename(blob.name)

这篇关于如何使用python下载谷歌云平台上文件夹内的文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！