我如何使用python下载谷歌云平台上的文件夹内的文件？ [英] How can i download the files inside a folder on google cloud platform using python?

问题描述

from google.cloud import storage
client = storage.Client()
bucket = client.get_bucket([bucket_name])
blob = bucket.get_blob([path to the .txt file])
blob.download_to_filename([local path to the downloaded .txt file])

如何在os.listdir（path）中调整我的python代码以添加类似于的文件名：只需复制所有文件在本地的某个文件夹中

How can i adjust my python code to add something like for filename in os.listdir(path): to just copy all the files in a certain folder on there locally

推荐答案

首先，我认为有必要强调的是，Google云端存储使用平面名称空间，实际上目录的概念不存在，因为在GCS中没有存储分层文件体系结构。有关目录的工作方式的更多信息可以在文档中找到，所以这是一个很好的阅读，如果你有兴趣在这个主题。

First of all, I think it is interesting to highlight that Google Cloud Storage uses a flat name space, and in fact the concept of "directories" does not exist, as there is no hierarchical file architecture being stored in GCS. More information about how directories work can be found in the documentation, so it is a good read if you are interested in this topic.

这就是说，你可以使用一个脚本，如下面我分享的脚本，以便下载所有将GCS中文件夹中的文件保存到本地环境中的相同文件夹中。基本上，您自己的代码中唯一重要的部分是前缀方法 / code>字段指向文件夹名称，以便查找只与名称中的文件夹模式匹配的blob。然后，你迭代它们，放弃目录blob本身（在GCS中它只是一个以结尾的blob> /），然后下载这些文件。 p>

That being said, you can use a script such as the one I share below, in order to download all files in a "folder" in GCS to the same folder in your local environment. Basically, the only important addition a part from your own code is that the bucket.list_blobs() method is being called, with the prefix field pointing to the folder name, in order to look for blobs which only match the folder-pattern in their name. Then, you iterate over them, discard the directory blob itself (which in GCS is just a blob with a name ending in "/"), and download the files.

from google.cloud import storage
import os

# Instantiate a CGS client
client=storage.Client()
bucket_name= "<YOUR_BUCKET_NAME>"

# The "folder" where the files you want to download are
folder="<YOUR_FOLDER_NAME>/"

# Create this folder locally
if not os.path.exists(folder):
    os.makedirs(folder)

# Retrieve all blobs with a prefix matching the folder
bucket=client.get_bucket(bucket_name)
blobs=list(bucket.list_blobs(prefix=folder))
for blob in blobs:
    if(not blob.name.endswith("/")):
        blob.download_to_filename(blob.name)

这篇关于我如何使用python下载谷歌云平台上的文件夹内的文件？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！