在Python中打开URL并获得X字节的最佳方法是什么？ [英] What is the best way to open a URL and get up to X bytes in Python?

问题描述

我想让机器人每小时获取一个URL，但如果该网站的运营商是恶意的，他可以让他的服务器向我发送一个1 GB的文件。是否有一种很好的方法可以将下载限制为100 KB并在该限制之后停止？

I want to have a robot fetch a URL every hour, but if the site's operator is malicious he could have his server send me a 1 GB file. Is there a good way to limit downloading to, say, 100 KB and stop after that limit?

我可以想象从头开始编写自己的连接处理程序，但我会喜欢尽可能使用urllib2，只是以某种方式指定限制。

I can imagine writing my own connection handler from scratch, but I'd like to use urllib2 if at all possible, just specifying the limit somehow.

谢谢！

推荐答案

这可能就是你要找的东西：

This is probably what you're looking for:

import urllib

def download(url, bytes = 1024):
    """Copy the contents of a file from a given URL
    to a local file.
    """
    webFile = urllib.urlopen(url)
    localFile = open(url.split('/')[-1], 'w')
    localFile.write(webFile.read(bytes))
    webFile.close()
    localFile.close()

这篇关于在Python中打开URL并获得X字节的最佳方法是什么？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！