Rails 并发请求 [英] Rails concurrent requests

问题描述

所以我有一个 rails 应用程序来解析一个长 json 文件 - 在此期间它向外部 API 发送请求，这似乎需要永远.我的问题是，是否像将每个请求粘贴到单独的线程中以加快速度一样简单?我不明白为什么我不能说 3 或 4 个线程同时运行发送请求......我看到的唯一问题是目前该方法显示一个错误页面，如果在解析结束时出现任何错误 - 和这可能不会发生，因为其他线程可能仍在运行?..最终我想把这个方法变成一个 rake 任务并使用一个呈现 html 页面的 cron 作业来运行它，这样应该更正确?

So I have a rails app that parses a long json file - during which it sends off requests to an external API, which seem to take forever. My question is, is it as simple as to stick each request into a separate thread in order to speed it up? I don't see why I can't have say 3 or 4 threads running concurrently sending off requests ... the only issue I see is currently the method displays an errors page if there are any errors present at the end of parsing - and this possibly couldn't happen as the other threads might still be running? .. eventually I want to make this method into a rake task and run it using a cron job that renders a html page so that should be better correct?

def load_movies
 @errors = []
 movie_list = get_data

 movie_list.first(50).each do |movie_data|
 ------------ Thread.new do -----------
  movie = Movie.new
  movie.construct_movie movie_data

  if !movie.valid?
    @errors << movie
  else
   movie.save
  end
  ----------- end thread ------------
  unless @errors.count > 0
  redirect_to root_path
end
end

是否像执行上述操作一样简单，并且它会运行打开的多个线程来发送请求(可以假设在construct_movie 中

is it as simple as to do what is above and it will run off open multiple threads to send the requests (which can be assumed to be in construct_movie

推荐答案

你可以做到这一点，是的.我已经做到了，是的.

You can do this, yes. I have done it, yes.

如果您使用 ActiveRecord，您可能会遇到一些使用 ActiveRecord 的多线程问题.为了在 ruby​​ 中做多线程的事情，我推荐并发 ruby​​ gem，它提供线程池和更高级别的抽象，如 Futures.https://github.com/ruby-concurrency/concurrent-ruby

You can run into some gotchas with multi-threaded use of ActiveRecord, if you're using ActiveRecord. For doing multi-threaded stuff in ruby, I recommend the concurrent-ruby gem, which provides thread pools and higher level abstractions like Futures. https://github.com/ruby-concurrency/concurrent-ruby

您可以这样做，不过，是的，这可能会变得棘手——如果后台作业队列有效，那对您来说可能是一个不太需要重新发明轮子的解决方案.无论是后台作业队列还是线程，您都必须弄清楚如何显示结果"——尽管使用直线程方法，我猜您的请求可以等待所有线程并且他们的结果，在显示结果之前(是的，我也这样做了).这可能会导致其自身的请求运行缓慢(您可能希望使用 puma 或乘客企业进行部署，以避免在它们上阻塞整个应用程序进程).

You can do this, although, yeah, it can get tricky -- if a background job queue works, that might be a less reinventing-the-wheel solution for you. Whether a background job queue or threads, you're going to have to figure out how to display 'results' -- although with the straight threads approach, I guess your request can wait on all the threads and their results, before displaying results (yeah, I've done this too). That can cause it's own problems with having slow-running requests (you'll probably want to deploy with puma or passenger enterprise to avoid blocking your entire app process on them).

从我自己做类似事情的经验来看，我没有一个简单的答案，我想无论你走哪条路都会遇到一些挑战.如果你能找到一种方法让后台作业队列工作，我认为这通常会比直接在应用程序进程中创建线程给你带来更少的麻烦.

From my own experience doing similar things, I don't have an easy answer, you'll run into some challenges no matter what route you take, I think. If you can figure out a way to make a background job queue work, I think that will generally give you less gotchas than directly creating threads in the app process.

这篇关于Rails 并发请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！