为什么Google云存储功能upload_from_file会引发超时错误? [英] Why does upload_from_file Google Cloud Storage Function Throws timeout error?

问题描述

我创建了一个对我有用的功能，它将文件上传到Google Cloud Storage.

I created a function that works for me, It uploads a file to Google Cloud Storage.

问题是，当我的朋友尝试使用本地计算机上的相同代码将相同文件上传到相同存储桶时，他会收到超时错误.他的互联网非常好，他应该可以在他的连接范围内毫无问题地上传文件.

The problem is when my friend tries to upload the same file to the same bucket using the same code from his local machine, he gets timeout error. His internet is very good and he should be able to upload the file with no problems within his connections.

知道为什么会这样吗?

def upload_to_cloud(file_path):
    """
        saves a file in the google storage. As Google requires audio files greater than 60 seconds to be saved on cloud before processing
        It always saves in 'audio-files-bucket' (folder)
        Input:
            Path of file to be saved
        Output:
            URI of the saved file
    """
    print("Uploading to cloud...")
    client = storage.Client().from_service_account_json(KEY_PATH)
    bucket = client.get_bucket('audio-files-bucket')
    file_name = str(file_path).split('\\')[-1]
    print(file_name)
    blob = bucket.blob(file_name)
    f = open(file_path, 'rb')
    blob.upload_from_file(f)
    f.close()
    print("uploaded at: ", "gs://audio-files-bucket/{}".format(file_name))
    return "gs://audio-files-bucket/{}".format(file_name)

它通过 upload_from_file(f)行中的超时异常.

It throughs the timeout exception in upload_from_file(f) line.

我们尝试使用upload_from_filename函数，但仍然会发生相同的错误.

We tried to use upload_from_filename function, but the same error still occurs.

推荐答案

问题通过减小Blob的块大小来解决.代码更改为:

The problem is solved by reducing the chunk size of the blob. The code changed to be:

def upload_to_cloud(file_path):
    """
        saves a file in the google storage. As Google requires audio files greater than 60 seconds to be saved on cloud before processing
        It always saves in 'audio-files-bucket' (folder)
        Input:
            Path of file to be saved
        Output:
            URI of the saved file
    """
    print("Uploading to cloud...")
    client = storage.Client().from_service_account_json(KEY_PATH)
    bucket = client.get_bucket('audio-files-bucket')
    file_name = str(file_path).split('\\')[-1]
    print(file_name)
    blob = bucket.blob(file_name)

    ## For slow upload speed
    storage.blob._DEFAULT_CHUNKSIZE = 2097152 # 1024 * 1024 B * 2 = 2 MB
    storage.blob._MAX_MULTIPART_SIZE = 2097152 # 2 MB

    with open(file_path, 'rb') as f:
        blob.upload_from_file(f)
    print("uploaded at: ", "gs://audio-files-bucket/{}".format(file_name))
    return "gs://audio-files-bucket/{}".format(file_name)

这篇关于为什么Google云存储功能upload_from_file会引发超时错误?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！