下载大文件 - 反馈？ [英] Downloading Large Files -- Feedback?
问题描述
此代码可以很好地从网上下载文件并将其写入
本地驱动器：
import urllib
f = urllib.urlopen（" http：//www.python.org/blah/blah.zip"）
g = f.read（）
file = open （< blah.zip"，" wb"）
file.write（g）
file.close（）
$ b但是，这个过程非常不透明。这下载并写入
文件，没有任何反馈。你还没有看到你已经下载了多少字节，等等。特别是g = f.read（）。下载一个大文件，然后出现一个怀孕的，闪烁的
光标，只需坐下
。
所以我的问题是，什么是编码这种基本反馈的好方法？此外，由于我的测试只使用*
代码*，我很好奇是否会在下载错误的情况下抛出一个可见错误
错误。
感谢您的任何指示。我正忙着用谷歌搜索。
This code works fine to download files from the web and write them to
the local drive:
import urllib
f = urllib.urlopen("http://www.python.org/blah/blah.zip")
g = f.read()
file = open("blah.zip", "wb")
file.write(g)
file.close()
The process is pretty opaque, however. This downloads and writes the
file with no feedback whatsoever. You don''t see how many bytes you''ve
downloaded already, etc. Especially the "g = f.read()" step just sits
there while downloading a large file, presenting a pregnant, blinking
cursor.
So my question is, what is a good way to go about coding this kind of
basic feedback? Also, since my testing has only *worked* with this
code, I''m curious if it will throw a visibile error if something goes
wrong with the download.
Thanks for any pointers. I''m busily Googling away.
推荐答案
" mwt" < MI ********* @ gmail.com>写道：
"mwt" <mi*********@gmail.com> writes:
f = urllib.urlopen（" http：//www.python.org/blah/blah.zip"）
g = f.read（）＃...
所以我的问题是，有什么好的方法来编写这种基本的反馈？此外，由于我的测试只使用*代码*，我很好奇是否会在下载错误的情况下抛出一个可见错误。
f = urllib.urlopen("http://www.python.org/blah/blah.zip")
g = f.read() # ... So my question is, what is a good way to go about coding this kind of
basic feedback? Also, since my testing has only *worked* with this
code, I''m curious if it will throw a visibile error if something goes
wrong with the download.
如果文件太大，那么一种明显的失败就是内存不足。到时候发生这种情况时，Python可以相当软（VM抖动等）。通常你不应该像这样读一个可能很大的文件
未知大小。你会说些什么
喜欢
而True：
block = f.read（4096）#read a文件中的4k块
如果len（块）== 0：
中断＃文件结束
＃用块做一些事情
你的做某事......可能涉及更新状态显示
或其他东西，说到目前为止读了多少。
One obvious type of failure is running out of memory if the file is
too large. Python can be fairly hosed (VM thrashing etc.) by the time
that happens. Normally you shouldn''t read a potentially big file of
unknown size all in one gulp like that. You''d instead say something
like
while True:
block = f.read(4096) # read a 4k block from the file
if len(block) == 0:
break # end of file
# do something with the block
Your "do something with..." could involve updating a status display
or something, saying how much has been read so far.
请原谅我的无知，但你能不能给
构成无理或危险大的文件的一个例子？我是一个在一个ubuntu盒子上运行python的
大约有一个ram。
另外，你知道任何在线的例子吗？健壮，
您描述的真实世界代码？
谢谢。
Pardon my ignorance here, but could you give me an example of what
would constitute file that is unreasonably or dangerously large? I''m
running python on a ubuntu box with about a gig of ram.
Also, do you know of any online examples of the kind of robust,
real-world code you''re describing?
Thanks.
mwt< mi ********* @ gmail.com>写道：
...
mwt <mi*********@gmail.com> wrote:
...
然而，这个过程非常不透明。这下载并写入
文件，没有任何反馈。你还没有看到你已经下载了多少字节，等等。特别是g = f.read（）。下载一个大文件，然后出现一个怀孕的，闪烁的光标，这就是坐下来。
所以我的问题是，有什么好方法去编码这种
基本反馈？此外，由于我的测试只有*工作*与此
The process is pretty opaque, however. This downloads and writes the
file with no feedback whatsoever. You don''t see how many bytes you''ve
downloaded already, etc. Especially the "g = f.read()" step just sits
there while downloading a large file, presenting a pregnant, blinking
cursor.
So my question is, what is a good way to go about coding this kind of
basic feedback? Also, since my testing has only *worked* with this
您可以使用urlretrieve而不是urlopen：urlretrieve接受名为
的可选参数reporthook，偶尔调用它（零
或更多次...... ;-)有三个参数block_count（块数目
目前已下载），block_size（每个块的大小，以字节为单位），file_size
（文件的总大小，以字节为单位，如果已知，否则为-1）。
reporthook函数（或其他可调用函数）可能会显示进度条或
无论你最喜欢什么。
urlretrieve保存了什么'下载到磁盘文件（你可以指定一个
文件名，或让它选择一个合适的临时文件名）并返回
两件事，文件名在哪里'下载了数据和一个
mimetools.Message实例，其标题包含元数据（例如内容
类型信息）。
如果这不能很好地满足您的需求，您可以在Python的库源目录中学习
urllib.py的来源，以确切了解
它正在做和编写你自己的修改版本。
Alex
Alex
You may use urlretrieve instead of urlopen: urlretrieve accepts an
optional argument named reporthook, and calls it once in a while ("zero
or more times"...;-) with three arguments block_count (number of blocks
downloaded so far), block_size (size of each block in bytes), file_size
(total size of the file in bytes if known, otherwise -1). The
reporthook function (or other callable) may display a progress bar or
whatever you like best.
urlretrieve saves what''s downloading to a disk file (you may specify a
filename, or let it pick an appropriate temporary filename) and returns
two things, the filename where it''s downloaded the data and a
mimetools.Message instance whose headers have metadata (such as content
type information).
If that doesn''t fit your needs well, you may study the sources of
urllib.py in your Python''s library source directory, to see exactly what
it''s doing and code your own modified version.
Alex
Alex
这篇关于下载大文件 - 反馈？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
