如何使用 Ruby Eventmachine 编写(大)文件 [英] How to write (large) files with Ruby Eventmachine

问题描述

我已经花了几天时间为 eventmachine 寻找一些非回声服务器示例，但似乎没有.假设我想编写一个接受文件并将其写入临时文件的服务器:

I've spent several days now finding some non-echo-server examples for eventmachine, but there just don't seem to be any. Let's say i want to write a server that accepts a file and writes it to a Tempfile:

require 'rubygems'
require 'tempfile'
require 'eventmachine'

module ExampleServer

  def receive_data(data)
    f = Tempfile.new('random')
    f.write(data)
  ensure
    f.close
  end

end

EventMachine::run {
  EventMachine::start_server "127.0.0.1", 8081, ExampleServer
  puts 'running example server on 8081'
}

写入文件会阻塞反应器，但我不知道如何做到Eventmachine 风格".我是否必须分块读取数据并将每个块写入 Em.next_tick 块内的磁盘?

Writing to the file would block the reactor, but i don't get how to do it 'Eventmachine style'. Would i have to read the data in chunks and write each chunk to disk within an Em.next_tick block?

感谢您的帮助安德烈亚斯

Thanks for any help Andreas

推荐答案

两个答案:

懒惰的答案:只需使用阻塞写入.EM 已经在为您提供离散的数据块，而不是一个巨大的字符串.所以你的示例实现可能有点偏离.您确定要为 EM 交给您的每个块创建一个新的临时文件吗?但是，我将继续假设您的示例代码按预期工作.

Lazy answer: just use a blocking write. EM is already handing you discrete chunks of data, not one gigantic string. So your example implementation may be a bit off. Are you sure you want to make a new tempfile for every single chunk that EM hands you? However, I'll continue on the assumption that your sample code is working as intended.

诚然，懒惰的方法取决于您要写入的设备，但是尝试同时将多个大流写入磁盘将成为主要瓶颈，并且您将失去基于事件的优势反正服务器.您最终只会到处乱搞磁盘搜索，IO 性能会下降，服务器的性能也会下降.RAM 可以同时处理很多事情，但是一旦您开始处理块设备和 IO 调度，无论您在做什么，都会遇到性能瓶颈.

Admittedly, the lazy approach depends on the device you're writing to, but trying to simultaneously write several large streams to disk at the same is going to be a major bottleneck and you'll lose your advantages of having an event based server anyway. You'll just end up with juggling disk seeks all over the place, IO performance will plummet, and so will your server's performance. Handling many things at once is okay with RAM, but once you start dealing with block devices and IO scheduling, you're going to run into performance bottlenecks no matter what you're doing.

但是，我猜您可能希望在对其他非 IO 繁重请求进行低延迟响应的同时对磁盘进行一些长写入.所以，也许是好的答案:

However, I guess you might want to do some long writes to disk at the same time that you want low latency responses to other, non-IO heavy requests. So, perhaps the good answer:

使用延迟.

require 'rubygems'
require 'tempfile'
require 'eventmachine'

module ExampleServer

  def receive_data(data)
    operation = proc do
      begin
        f = Tempfile.new('random')
        f.write(data)
      ensure
        f.close
      end
    end

    callback = proc do
      puts "I wrote a file!"
    end

    EM.defer(operation, callback)
  end

end

EventMachine::run {
  EventMachine::start_server "127.0.0.1", 8081, ExampleServer
  puts 'running example server on 8081'
}

是的，这确实使用了线程.在这种情况下它真的没有那么糟糕:您不必担心线程之间的同步，因为 EM 已经足够好为您处理这个问题.如果您需要响应，请使用回调，当工作线程完成时，它将在主反应器线程中执行.此外，对于这种情况，GIL 不是问题，因为您在这里处理的是 IO 阻塞，而不是试图实现 CPU 并发.

Yes, this does use threading. It's really not that bad in this case: you don't have to worry about synchronization between threads, because EM is nice enough to handle this for you. If you need a response, use the callback, which will be executed in the main reactor thread when the worker thread completes. Also, the GIL is something of a non-issue for this case, since you're dealing with IO blocking here, and not trying to achieve CPU concurrency.

但是如果您确实打算将所有内容写入同一个文件，则必须小心使用 defer，因为会出现同步问题，因为您的线程可能会同时尝试写入同一个文件.

But if you did intend to write everything to the same file, you'll have to be careful with defer, since the synchronization issue will arise as your threads will likely attempt to write to the same file at the same time.

这篇关于如何使用 Ruby Eventmachine 编写(大)文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！