如何在 urllib.urlretrieve 中捕获 404 错误 [英] How to catch 404 error in urllib.urlretrieve
问题描述
背景:我正在使用 urllib.urlretrieve
,而不是 urllib*
模块中的任何其他函数,因为钩子函数支持(见下面的 reporthook
).. 用于显示一个文本进度条.这是 Python >=2.6.
然而,urlretrieve
非常愚蠢,以至于无法检测 HTTP 请求的状态(例如:是 404 还是 200?).
下载具有类似钩子的支持(以显示进度条)和适当的 HTTP 错误处理的远程 HTTP 文件的最广为人知的方法是什么?
查看urllib.urlretrieve
的完整代码:
def urlretrieve(url, filename=None, reporthook=None, data=None):全局_urlopener如果不是 _urlopener:_urlopener = FancyURLopener()return _urlopener.retrieve(url, filename, reporthook, data)
换句话说,您可以使用 urllib.FancyURLopener(它是公共 urllib API 的一部分).您可以覆盖 http_error_default
以检测 404:
class MyURLopener(urllib.FancyURLopener):def http_error_default(self, url, fp, errcode, errmsg, headers):# 以你喜欢的方式处理错误fn, h = MyURLopener().retrieve(url, reporthook=my_report_hook)
Background: I am using urllib.urlretrieve
, as opposed to any other function in the urllib*
modules, because of the hook function support (see reporthook
below) .. which is used to display a textual progress bar. This is Python >=2.6.
>>> urllib.urlretrieve(url[, filename[, reporthook[, data]]])
However, urlretrieve
is so dumb that it leaves no way to detect the status of the HTTP request (eg: was it 404 or 200?).
>>> fn, h = urllib.urlretrieve('http://google.com/foo/bar')
>>> h.items()
[('date', 'Thu, 20 Aug 2009 20:07:40 GMT'),
('expires', '-1'),
('content-type', 'text/html; charset=ISO-8859-1'),
('server', 'gws'),
('cache-control', 'private, max-age=0')]
>>> h.status
''
>>>
What is the best known way to download a remote HTTP file with hook-like support (to show progress bar) and a decent HTTP error handling?
Check out urllib.urlretrieve
's complete code:
def urlretrieve(url, filename=None, reporthook=None, data=None):
global _urlopener
if not _urlopener:
_urlopener = FancyURLopener()
return _urlopener.retrieve(url, filename, reporthook, data)
In other words, you can use urllib.FancyURLopener (it's part of the public urllib API). You can override http_error_default
to detect 404s:
class MyURLopener(urllib.FancyURLopener):
def http_error_default(self, url, fp, errcode, errmsg, headers):
# handle errors the way you'd like to
fn, h = MyURLopener().retrieve(url, reporthook=my_report_hook)
这篇关于如何在 urllib.urlretrieve 中捕获 404 错误的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!