使用 urllib2 将大型二进制文件流式传输到文件 [英] Stream large binary files with urllib2 to file

问题描述

我使用以下代码将大文件从 Internet 流式传输到本地文件:

I use the following code to stream large files from the Internet into a local file:

fp = open(file, 'wb')
req = urllib2.urlopen(url)
for line in req:
    fp.write(line)
fp.close()

这有效，但下载速度很慢.有没有更快的方法?(文件很大，所以我不想将它们保存在内存中.)

This works but it downloads quite slowly. Is there a faster way? (The files are large so I don't want to keep them in memory.)

推荐答案

没有理由逐行工作(小块并且需要 Python 为您找到行尾！-)，只需将其分成更大的块，例如:

No reason to work line by line (small chunks AND requires Python to find the line ends for you!-), just chunk it up in bigger chunks, e.g.:

# from urllib2 import urlopen # Python 2
from urllib.request import urlopen # Python 3

response = urlopen(url)
CHUNK = 16 * 1024
with open(file, 'wb') as f:
    while True:
        chunk = response.read(CHUNK)
        if not chunk:
            break
        f.write(chunk)

尝试使用各种 CHUNK 大小来找到满足您要求的最佳位置".

Experiment a bit with various CHUNK sizes to find the "sweet spot" for your requirements.

这篇关于使用 urllib2 将大型二进制文件流式传输到文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！