Rails 中的多线程:自动加载常量时检测到循环依赖 [英] Multithreading in Rails: Circular dependency detected while autoloading constant

问题描述

我有一个 Rails 应用，其中有一个 Rake 任务，该任务使用由 concurrent-ruby gem 提供的多线程功能.

I have a Rails app in which I have a Rake task that uses multithreading functions supplied by the concurrent-ruby gem.

我不时遇到自动加载常量时检测到循环依赖错误.

在谷歌搜索后，我发现这与结合使用线程和加载 Rails 常量有关.

After Googling for a bit I found this to be related to using threading in combination with loading Rails constants.

我偶然发现了以下 GitHub 问题:https://github.com/ruby-concurrency/concurrent-ruby/issues/585 和 https://github.com/rails/rails/issues/26847

I stumbled upon the following GitHub issues: https://github.com/ruby-concurrency/concurrent-ruby/issues/585 and https://github.com/rails/rails/issues/26847

正如这里所解释的，您需要将从新线程调用的任何代码包装在 Rails.application.reloader.wrap do 或 Rails.application.executor.wrap do代码>块，这就是我所做的.然而，这会导致死锁.

As explained here you need to wrap any code that is called from a new thread in a Rails.application.reloader.wrap do or Rails.application.executor.wrap do block, which is what I did. However, this leads to deadlock.

然后建议使用 ActiveSupport::Dependencies.interlock.permit_concurrent_loads 在主线程上包装另一个阻塞调用.但是，我不确定我应该用它包装哪个代码.

The recommendation is then to use ActiveSupport::Dependencies.interlock.permit_concurrent_loads to wrap another blocking call on the main thread. However, I am unsure which code I should wrap with this.

这是我尝试过的，但这仍然会导致死锁:

Here's what I tried, however this still leads to a deadlock:

@beanstalk = Beaneater.new("#{ENV.fetch("HOST", "host")}:#{ENV.fetch("BEANSTALK_PORT", "11300")}")
tube_name = ENV.fetch("BEANSTALK_QUEUE_NAME", "queue")

pool = Concurrent::FixedThreadPool.new(Concurrent.processor_count * 2)

# Process jobs from tube, the body of this block gets executed on each message received
@beanstalk.jobs.register(tube_name) do |job|
    ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
      @logger.info "Received job: #{job.id}"
      Concurrent::Future.execute(executor: pool) do
        Rails.application.reloader.wrap do
          # Stuff that references Rails constants etc
          process_beanstalk_message(job.body)
        end
      end
    end
end

@beanstalk.jobs.process!(reserve_timeout: 10)

谁能解释一下我应该如何解决这个问题?奇怪的是，我在生产中遇到过这种情况，而关于此主题的其他信息似乎暗示它通常只应在开发中发生.

Can anyone shed a light as to how I should solve this? The odd thing is I encounter this in production while other information on this topic seems to imply it should normally only occur in development.

在生产中我使用以下设置:

In production I use the following settings:

config.eager_load = true

config.cache_classes = true.

所有环境的自动加载路径都是 Rails 默认加上两个特定文件夹(models/validators"和jobs/concerns").

Autoload paths for all environments are Rails default plus two specific folders ("models/validators" & "jobs/concerns").

eager_load_paths 未在我的任何配置中修改或设置，因此必须等于 Rails 默认值.

eager_load_paths is not modified or set in any of my configs so must be equal to the Rails default.

我使用的是 Rails 5，所以 enable_dependency_loading 在生产中应该等于 false.

I am using Rails 5 so enable_dependency_loading should equal to false in production.

推荐答案

您可能需要更改 eager_load_paths 以包含引发错误的类或模块的路径.eager_load_paths 记录在在 Rails 指南中.

You likely need to change your eager_load_paths to include the path to the classes or modules that are raising the errors. eager_load_paths is documented in the Rails Guides.

您遇到的问题是应用启动时Rails 没有加载这些常量；当它们被其他一些代码调用时，它会自动加载它们.在多线程 Rails 应用中，两个线程在尝试加载这些常量时可能会出现竞争条件.

The problem you're running into is that Rails is not loading these constants when the app starts; it automatically loads them when they are called by some other piece of code. In a multithreaded Rails app, two threads may have a race condition when they try to load these constants.

告诉 Rails 急切加载这些常量意味着它们将在 Rails 应用程序启动时加载一次.光说 eager_load = true 是不够的；您还必须指定类或模块定义的路径.在 Rails 应用程序配置中，这是 eager_load_paths 下的 Array.例如，要预先加载 ActiveJob 类:

Telling Rails to eagerly load these constants means they will be loaded once when the Rails app is started. It's not enough to say eager_load = true; you have to specify the paths to the class or module definitions as well. In the Rails application configuration, this is an Array under eager_load_paths. For example, to eager load ActiveJob classes:

config.eager_load_paths += ["#{config.root}/app/jobs"]

或者从 lib/ 加载自定义模块:

Or to load a custom module from lib/:

config.eager_load_paths += ["#{config.root}/lib/custom_module"]

更改预先加载设置会影响 Rails 的行为.例如，在 Rails development 环境中，您可能习惯于运行一次 rails server，并且每次重新加载其中一个端点时，它都会反映对代码的任何更改你做了.这不适用于 config.eager_load = true，因为这些类在启动时加载一次.因此，您通常只需更改 production 的 eager_load 设置.

Changing your eager load settings will affect the behavior of Rails. For example, in the Rails development environment, you're probably used to running rails server once, and every time you reload one of the endpoints it will reflect any changes to code you've made. That will not work with config.eager_load = true, because the classes are loaded once, at startup. Therefore, you will typically only change your eager_load settings for production.

更新

您可以从 rails 控制台 检查您现有的 eager_load_paths.例如，这些是新 Rails 5 应用程序的默认值.如您所见，它不会加载 app/**/*.rb；它加载 Rails 应该知道的特定路径.

You can check your existing eager_load_paths from the rails console. For example, these are the default values for a new Rails 5 app. As you can see, it does not load app/**/*.rb; it loads the specific paths that Rails is expected to know about.

Rails.application.config.eager_load_paths
=> ["/app/assets",
 "/app/channels",
 "/app/controllers",
 "/app/controllers/concerns",
 "/app/helpers",
 "/app/jobs",
 "/app/mailers",
 "/app/models",
 "/app/models/concerns"]

这篇关于Rails 中的多线程:自动加载常量时检测到循环依赖的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！