从URL打开大文件时防止超时 [英] Prevent timeout when opening large files from URL

问题描述

我正在编写一个Ruby 1.8.7脚本，该脚本必须从服务器请求非常大的XML文件(1-5MB)，这非常慢(1MB需要1min30sec).所请求的文件已写入磁盘.

I am writing a Ruby 1.8.7 script which has to request really large XML files(1 - 5MB) from server which is quite slow(1min30sec for 1MB). The requested file is written to disk.

我将脚本中的超时设置为一些荒谬的秒数，因为我真的想获取文件，而不是花费太长时间继续前进.仍然有很高的秒数，我一直在超时.

I set the timeout in my script to some ridiculous amount of seconds, since I really want to get the file, not just move on if it takes too long. Still with the high amount of seconds I keep getting timeouts.

这是否有最佳实践?

现在我使用

  open(DIR + "" + number + "" + ".xml", 'wb') do |file|
  begin
    status = Timeout::timeout(6000000) do
      file << open(url).read
      end
    rescue Timeout::Error => e
      Rails.logger.info "Timeout for:" + number.to_s
    end
  end

现在，tought超时设置为几秒钟，这会使6000000的时间超过1分30秒，但是不知何故，它没有使用我的几秒钟的超时.再次注意，我仅限于使用Ruby 1.8.7

now tought timeout was set in seconds which would make 6000000 way more then 1min30sec, but somehow it isn't using my timeout in seconds. Note again that i'm restricted to using Ruby 1.8.7

推荐答案

不幸的是，这是有问题的.在Ruby 1.9.x中，open-uri扩展的open可以采用read_timeout参数，并将其传递给http库.但是在您使用的Ruby 1.8.x中，此参数不可用.

Unfortunately, this is problematic. In Ruby 1.9.x, open-uri-extended open can take read_timeout parameter, which it passes over to http library. But in Ruby 1.8.x, which you're using, this parameter is not available.

因此，您需要直接使用net/http，调用start/get there并根据自己的喜好设置read_timeout.如果仅使用open-uri包装器，则read_timeout仍为60秒，比您想要的时间短.

So, you need to use net/http directly, call start/get there and set read_timeout to your liking. If you just use the open-uri wrapper, the read_timeout remains 60 seconds, which is shorter than what you want.

这篇关于从URL打开大文件时防止超时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！