Rails 4.2自动加载不是线程安全的 [英] Rails 4.2 Autoloading not thread-safe

问题描述

我有以下模型：

  class用户< ActiveRecord :: Base 
 def send_message（content）
 MessagePoro.new（content）.deliver！ 
 end 
 
 def self.send_to_all（content）
线程= [] 
 all.each | user | 
个线程<< Thread.new do 
 user.send_message（content）
结束
结束
 thread.each（&：join）
结束
结束
  

MessagePoro模型可以很简单，例如app / models / message_poro.rb：

  class MessagePoro 
 def initialize（content）
＃.. 
结束
 
 def交付！ 
＃... 
结束
结束
  

现在，当我有例如100个用户，并且我正在运行User.send_to_all（ test），我有时遇到这些错误：

  RuntimeError：自动加载常量MessagePoro 
  

or：

$时检测到循环依赖b
$ b

 参数数量错误（1代表0）
  

我认为一定是因为未加载MessagePoro，并且所有线程都尝试同时加载它，或者类似的事情。由于这些错误仅在某些情况下发生，因此，我敢肯定，只有在存在竞赛条件或与Threading有关的情况下才可能发生。我尝试在启动线程之前初始化MessagePoro，并且尝试过eager_loading，但问题似乎仍然存在。
还有什么我可以尝试缓解的问题？

解决方案

我最近在尝试解决此问题时遇到了一个非常相似的问题使用放置在 [rails_root] / lib 目录中的额外自定义库。

TL; DR：

您可以使用紧急加载解决问题，因为这样可以确保在运行任何实际代码之前，所有常量/模块/类都已在内存中。但是，要使其工作：

1. 您必须设置 config.eager_load = true 在Rails的配置中（默认在生产环境中完成）


2. 要加载的类所在的文件必须位于 config.eager_load_paths ，而不是 config.autoload_paths 。

或

您可以使用 require 或 require_dependency （另一个ActiveSupport功能），以确保在Rails自动加载之前明确加载了您需要的代码。

更多信息

正如digidigo在其答复中所述，循环依赖项错误来自 ActiveSupport :: Dependencies 模块或 Rails自动加载器。此代码不是线程安全的，因为它使用该类/模块变量来存储正在加载的文件。如果两个线程最终一次同时自动加载同一事物，那么其中一个线程可能会因看到文件已在该类变量中加载而引发循环依赖错误而误导了我们。

当使用（线程化的）Puma Web服务器在生产模式下运行Rails时，我遇到了这个问题。我们已经在Rails根目录的 lib 目录中添加了一个小型库，并且最初在<$ c $中添加了 lib c> config.autoload_once_paths 。在开发中一切正常，但在生产中（启用 config.eager_load 和 config.cache_classes ），我们偶尔会几乎同时提出请求时，也会遇到同样的循环依赖问题。几个小时的调试之后，当逐步遍历循环依赖项的ActiveSupport代码并看到不同的线程在代码的不同位置出现时，我最终看到了眼前发生的非线程安全问题。第一个线程将文件添加到 loading 数组中，然后第二个线程在此处找到文件并引发循环依赖错误。

原来为 autoload_paths 或 autoload_once_paths 添加了一些内容 NOT 也意味着它会因急于加载而被拾取。但是相反的情况是正确的-如果禁用了eager_load，则添加到 eager_load_paths 的路径将被考虑自动加载（请参阅这篇文章以获得更多信息）。我们切换到 eager_load_paths ，到目前为止没有其他问题。

有趣的是，就在Rails 4 beta之前，默认情况下，生产环境中的自动加载功能处于禁用状态，这意味着此类问题将导致100％的时间失败，而不是导致5％的古怪的线程失败。但是，此功能已在4.0 beta版本中及时恢复-您可以在此处查看一些（热情的）讨论（包括选择短语老实说，您是要我自己去吗？）。但是从那以后，该还原已在Rails 5.0.0beta1之前还原，因此希望将来会有更少的人再次解决这个令人头疼的问题。

其他注意事项：

Rails自动加载器与Ruby自动加载器完全分开-这似乎是因为Rails在尝试自动加载常量时对目录结构进行了更多推断。 p>

从Ruby 2.0开始，Ruby的自动加载似乎已成为线程安全的，但这与Rails的自动加载代码无关。如前所述，Rails的自动加载器绝对不是不是线程安全的。

I have the following model:

class User < ActiveRecord::Base
  def send_message(content)
    MessagePoro.new(content).deliver!
  end

  def self.send_to_all(content)
    threads = []
    all.each do |user|
      threads << Thread.new do
        user.send_message(content)
      end
    end
    threads.each(&:join)
  end
end

MessagePoro model can be something simple, e.g. app/models/message_poro.rb:

class MessagePoro
  def initialize(content)
    # ...
  end

  def deliver!
    # ...
  end
end

Now, when I'm having e.g. 100 Users, and I'm running User.send_to_all("test") I'm sometimes getting thoser errors:

RuntimeError: Circular dependency detected while autoloading constant MessagePoro

or:

wrong number of arguments (1 for 0)

I figured it must be because MessagePoro isn't loaded and all Threads try to load it simultaneously, or something like that. Since those errors occur only sometimes, I'm pretty sure it's only when there is a 'race condition' or has something todo with Threading. I have tried to initialize MessagePoro before starting the Threads, and I have played around with eager_loading, but the problem seems to persist. What else can I try to mitigate this issue?

解决方案

I recently ran into a very similar issue when trying to use an extra custom library placed in the [rails_root]/lib directory.

TL;DR:

You can use eager loading to around the issue, as that makes sure all constants/modules/classes are in memory before any actual code runs. However for this to work:

1. You must have config.eager_load = true set in the Rails config (this is done by default in the Production environment)
2. The file that your class-to-be-eager-loaded is in must be in the config.eager_load_paths, as opposed to config.autoload_paths.

OR

You can use require or require_dependency (another ActiveSupport feature) to make sure the code you need it explicitly loaded before it would otherwise get autoloaded by Rails.

More info

As digidigo mentioned in his reply, the circular dependency error comes from the ActiveSupport::Dependencies module, or the Rails autoloader in more general terms. This code is not threadsafe, as it uses that class/module variable to store files that it is loading. If two threads end up autoloading the same thing at the same time, one of them can get mislead by seeing the file to load already in that class variable and throwing a 'circular dependency' error.

I ran into this issue when running Rails in production mode with the (threaded) Puma webserver. We had added a small library to the lib directory in our Rails root, and initially added lib to config.autoload_once_paths. Everything was fine in Development, but in Production (with config.eager_load and config.cache_classes enabled), very occasionally we would get these same circular dependency issues with near-simultaneous requests. A few hours of debugging later, I ended up seeing the non-thread-safety happening in front of my eyes, when stepping through the ActiveSupport code around the circular dependency and seeing the different threads pick up at different points in the code. The first thread would add the file to load into the loading array, then the second thread would find it there and raise the circular dependency error.

It turns out adding something to autoload_paths or autoload_once_paths does NOT also mean that it will get picked up by eager loading. However the opposite is true - paths added to eager_load_paths will be considered for autoloading if eager_load is disabled (see this article for more info). We switched to eager_load_paths and have had no further issues so far.

Interestingly enough, just before the Rails 4 beta, autoloading was disabled in the Production environment by default, which meant that an issue like this would have caused a hard fail 100% of the time, rather than a quirky threading fail 5% of the time. However this was reverted in time for the 4.0 beta release - you can see some (passionate) discussion about it here (including the choice phrase 'honestly, you're telling me to go f*** myself?'). Since then though, that revert has been reverted ahead of the Rails 5.0.0beta1, so hopefully less people will have to deal with this headache of an issue again in the future.

Extra notes:

The Rails autoloader is totally separate from the Ruby autoloader - this seems to be because Rails does more inference on directory structure when trying to autoload constants.

Ruby's autoload appears to have been made threadsafe as of Ruby 2.0, however this has nothing to do with the Rails autoloading code. Rails's autoloader appears to be definitely not threadsafe, as previously mentioned.

这篇关于Rails 4.2自动加载不是线程安全的的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！