GCloud上传httplib2.RedirectMissingLocation:已重定向，但响应缺少Location:标头 [英] GCloud Upload httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header

问题描述

我正在尝试使用简单的python程序将小文件上传到gcloud

I am attempting to upload a small file to gcloud using a simple python program

client = storage.Client(project=GCLOUD_PROJECT)
bucket = client.get_bucket(GCLOUD_BUCKET)
blob = bucket.blob(GCLOUD_FILE_ON_CLOUD)
blob.upload_from_filename(GCLOUD_FILE_LOCAL)

直到最近它一直在工作，并且发生了一些变化.现在，每当我上传大于5MB的文件时，都会出现以下错误.小于或等于5MB的文件将通过.大小还不足以实现可恢复的上传，是吗?

It had been working until recently and something changed. Now, whenever I upload a file greater than 5MB I get the below error. Files less than or equal to 5MB goes through. The size isn't large enough to implement resumable upload, is it?

Traceback (most recent call last):
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/gcloud_upload.py", line 40, in <module>
    blob.upload_from_filename(GCLOUD_FILE_LOCAL)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/storage/blob.py", line 597, in upload_from_filename
    encryption_key=encryption_key, client=client)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/storage/blob.py", line 543, in upload_from_file
    http_response = upload.stream_file(use_chunks=True)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/streaming/transfer.py", line 1086, in stream_file
    response = send_func(self.stream.tell())
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/streaming/transfer.py", line 1215, in _send_chunk
    return self._send_media_request(request, end)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/streaming/transfer.py", line 1125, in _send_media_request
    self.bytes_http, request, retries=self.num_retries)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/streaming/http_wrapper.py", line 423, in make_api_request
    check_response_func=check_response_func)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/streaming/http_wrapper.py", line 371, in _make_api_request_no_retry
    redirections=redirections, connection_type=connection_type)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/oauth2client/transport.py", line 175, in new_request
    redirections, connection_type)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/oauth2client/transport.py", line 282, in request
    connection_type=connection_type)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/httplib2/__init__.py", line 1986, in request
    cachekey,
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/httplib2/__init__.py", line 1685, in _request
    content,
httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header.

当我调试时，我看到以下内容.

When I debug I see the below.

{
 'content-type': 'text/plain; charset=utf-8', 
 'range': 'bytes=0-1048575', 
 'content-length': '0', 
 'date': 'Sun, 19 Jan 2020 23:52:13 GMT', 
 'server': 'UploadServer', 
 'alt-svc': 'quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000', 
 'status': '308'}

我的点子列表以供参考.操作系统是MAC OSx.我也在Linux上进行了测试.同样的问题.

My pip list for reference. OS is MAC OSx. I tested on Linux as well. Same issue.

adal                     1.2.2    
bson                     0.5.8    
cachetools               3.1.1    
certifi                  2019.9.11
cffi                     1.13.1   
chardet                  3.0.4    
cryptography             2.8      
dnspython                1.16.0   
gcloud                   0.18.3   
gitdb2                   2.0.6    
GitPython                3.0.5    
google-auth              1.6.3    
googleapis-common-protos 1.51.0   
httplib2                 0.16.0   
idna                     2.7      
itsdangerous             1.1.0    
MarkupSafe               1.1.1    
oauth2client             4.1.3    
oauthlib                 3.1.0    
pip                      19.0.3   
protobuf                 3.11.2   
pyasn1                   0.4.7    
pyasn1-modules           0.2.7    
pycparser                2.19     
PyJWT                    1.7.1    
python-dateutil          2.8.0    
requests                 2.19.1   
requests-oauthlib        1.2.0    
rsa                      4.0      
setuptools               40.8.0   
six                      1.12.0   
smmap2                   2.0.5    
urllib3                  1.23     
websocket-client         0.56.0   
Werkzeug                 0.16.0   

这是上周的工作.最近有什么变化吗?

This was working last week. Has something changed recently?

推荐答案

解决方案

gcloud软件包已弃用两次，并且与httplib2>=0.16不兼容.正确的解决方案是使用google-cloud-*软件包家族.

Solutions

gcloud package is deprecated twice and is not compatible with httplib2>=0.16. Proper solution is to use google-cloud-* packages family.

google-api-python-client>=1.7.12使用的是redirect_codes API，请升级，它可以正常工作.

google-api-python-client>=1.7.12 is using redirect_codes API, please upgrade, it just works.

httplib2 v0.17.0刚刚发布，具有修改被视为重定向的响应代码集的功能.如果可以修改创建Http对象的代码，这是最好的选择:

httplib2 v0.17.0 is just released with ability to modify set of response codes treated as redirects. This is the best option if you can modify code that creates Http object:

http = httplib2.Http()
http.redirect_codes = http.redirect_codes - {308}

不可能，请编辑您的requirements.txt以固定httplib2<0.16.0

Iff that's not possible, edit your requirements.txt to pin down httplib2<0.16.0

Google云存储服务器使用HTTP 308 用于特殊的可恢复上传功能，该功能有点类似于重试相同方法到相同位置" ，但不完全是这样.

google cloud storage server uses HTTP 308 for special resumable uploads feature which somewhat resembles "retry same method to same location", but not quite.

以上(可能)是PyPI软件包 google-resumable-media 的理由.与通用的HTTP客户端不同，该版本由gcloud相关软件包的最新版本使用，并以类似的方式处理200和308.

Above is (probably) rationale for PyPI package google-resumable-media which is used by more recent incarnations of gcloud related packages and handles 200 and 308 in similar manner, unlike generic HTTP client should.

历史背景:

• 2016 软件包gcloud 已被弃用，转而使用软件包google-cloud
• 2017 google-cloud-python 从httplib2切换HTTP传输请求
• 2018 软件包google-cloud 被再次弃用，转而支持google-cloud- *包
• 2020 httplib2 v0.16获得了对每个 RFC7538
的308个重定向的支持.

• 2016 package gcloud was deprecated in favor of package google-cloud
• 2017 google-cloud-python switches HTTP transport from httplib2 to requests
• 2018 package google-cloud was deprecated again in favor of google-cloud-* packages
• 2020 httplib2 v0.16 gained support for 308 redirects per RFC7538

很抱歉有坏消息.作为HTTP爱好者，我偏向308支持.如果您有更好的主意，如何更好地处理这种情况，请与我们联系.

Sorry for bad news. As HTTP enthusiast, I'm biased towards 308 support. Please reach if you have better idea how to handle this situation more gracefully.

这篇关于GCloud上传httplib2.RedirectMissingLocation:已重定向，但响应缺少Location:标头的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！