使用python下载时如何优雅地超时 [英] how to time-out gracefully while downloading with python

问题描述

  try：
 urllib 
我正在下载一大堆文件，包含以下代码： .urlretrieve（url2download，destination_on_local_filesystem）
除了KeyboardInterrupt：
 break 
除了：
 print超时或有其他异常：+ url2download 
  

如果连接正在启动时，URL url2download上的服务器超时，最后一个异常被正确处理。但有时服务器回应，下载是开始的，但是服务器是如此之慢，即使是一个文件也需要几个小时，最终会返回以下内容：

 仅在albrightandomalley.com输入客户端的用户名：
仅在albrightandomalley.com中输入客户端的密码：
  

，只是挂在那里（虽然没有用户名/密码是通过浏览器下载相同的链接）。

我在这种情况下的意图是 - 跳过这个文件，然后去下一个。问题是 - 怎么做？在python中有没有办法指定下载一个文件可以工作多久？如果有更多的时间已经花费了，中断和转发？

解决方案

尝试：

import socket

socket.setdefaulttimeout（30）

I'm downloading a huge set of files with following code in a loop:

try:
    urllib.urlretrieve(url2download, destination_on_local_filesystem)
except KeyboardInterrupt:
    break
except:
    print "Timed-out or got some other exception: "+url2download

If the server times-out on URL url2download when connection is just initiating, the last exception is handled properly. But sometimes server responded, and downloading is started, but the server is so slow, that it'll takes hours for even one file, and eventually it returns something like:

Enter username for Clients Only at albrightandomalley.com:
Enter password for  in Clients Only at albrightandomalley.com:

and just hangs there (although no username/passworde is aksed if the same link is downloaded through the browser).

My intention in this situation would be -- skip this file and go to the next one. The question is -- how to do that? Is there a way in python to specify how long is OK to work on downloading one file, and if more time is already spent, interrupt, and go forward?

解决方案

Try:

import socket

socket.setdefaulttimeout(30)

这篇关于使用python下载时如何优雅地超时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！