阻止ftplib下载正在进行中的文件？ [英] Prevent ftplib from Downloading a File in Progress?
问题描述
我们有一个ftp系统设置，用于监控远程ftp服务器并从我们不受控制的ftp服务器上下载。脚本连接到远程ftp，并抓取服务器上文件的文件名，然后检查是否已经下载了它的文件。如果尚未下载，那么我们下载该文件并将其添加到列表中。
我们最近遇到了一个问题，远程ftp端的某个人复制一个巨大的单个文件（> 1GB），然后脚本会醒来看到一个新文件并开始下载正在复制的文件。
什么是最好的方法来检查这个？我正在考虑抓取文件大小等待几秒钟，再次检查文件大小，看看它是否增加了，如果没有，我们再下载它。但是由于时间问题，我们不能等待每个文件集几秒钟，看看文件大小是否增加。
最好的方法是什么，目前所有的事情都是通过pythons ftplib来完成的，除了使用前面提到的方法，我们怎么能做到这一点。
然而，我再次重申一下，我们对远程ftp站点有0个控制权。
谢谢。
UPDATE1：
我想如果我尝试重命名它，因为我们对ftp拥有完全权限，如果文件上传正在进行中，重命名命令会失败吗？
我们在这里没有任何实际选项...我们是吗？
UPDATE2：
这里有一些有趣的内容，我们测试过的一些ftps似乎会在传输开始时自动分配空间。
例如如果我将200MB文件传输到FTP服务器。如果我连接到ftp服务器并在上传过程中进行大小调整，则传输处于活动状态。它显示大小为200mb。即使该文件只有10％完成。
权限似乎也随机设置，IIS附带的FTP服务器会在文件完成复制后设置权限。虽然其他一些较老的FTP服务器会在您发送文件时立即设置它。
：'（
）该死的鱼雷！全速前进！
只要下载文件即可是一个大文件，然后在下载完成后等待，只要您的方案合理，并从停止点继续下载。重复，直到没有更多内容可供下载为止。
We have a ftp system setup to monitor/download from remote ftp servers that are not under our control. The script connects to the remote ftp, and grabs the file names of files on the server, we then check to see if its something that has already been downloaded. If it hasn't been downloaded then we download the file and add it to the list.
We recently ran into an issue, where someone on the remote ftp side, will copy in a massive single file(>1GB) then the script will wake up see a new file and begin downloading the file that is being copied in.
What is the best way to check this? I was thinking of grabbing the file size waiting a few seconds checking the file size again and see if it has increased, if it hasn't then we download it. But since time is of the concern, we can't wait a few seconds for every single file set and see if it's file size has increased.
What would be the best way to go about this, currently everything is done via pythons ftplib, how can we do this aside from using the aforementioned method.
Yet again let me reiterate this, we have 0 control over the remote ftp sites.
Thanks.
UPDATE1:
I was thinking what if i tried to rename it... since we have full permissions on the ftp, if the file upload is in progress would the rename command fail?
We don't have any real options here... do we?
UPDATE2: Well here's something interesting some of the ftps we tested on appear to automatically allocate the space once the transfer starts.
E.g. If i transfer a 200mb file to the ftp server. While the transfer is active if i connect to the ftp server and do a size while the upload is happening. It shows 200mb for the size. Even though the file is only like 10% complete.
Permissions also seem to be randomly set the FTP Server that comes with IIS sets the permissions AFTER the file is finished copying. While some of the other older ftp servers set it as soon as you send the file.
:'(
"Damn the torpedoes! Full speed ahead!"
Just download the file. If it is a large file then after the download completes wait as long as is reasonable for your scenario and continue the download from the point it stopped. Repeat until there is no more stuff to download.
这篇关于阻止ftplib下载正在进行中的文件？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
