配置 Puma 和 Sidekiq [英] Configuring Puma and Sidekiq

问题描述

可能更多的是试图克服学习曲线 + 实际代码问题.如果它看起来很糟糕，我深表歉意，目前我在生产中遇到了这个错误.

Might be more of trying to overcome a learning curve + actual code question. I apologize if it seems nubish, currently I get this error within production.

基本上，我一直收到这个redis 池太小"的消息，我不知道从哪里开始，实际上我根本无法理解如何使用 puma 准确配置 sidekiq 或在配置之后出现的任何东西，例如缩放等.

Basically I keep recieving this "redis pool is too small" and I'm lost where to start, I'm actually lost on basically understanding how to accurately configure sidekiq with puma or anything that comes after configuration like scaling etc.

下面我的配置遵循了我收到的错误.

Below I have my configuration followed my error I recieve .

程序文件web: bundle exec puma -C config/puma.rb worker: bundle exec sidekiq-e 生产 -C config/sidekiq.yml

ProcFile web: bundle exec puma -C config/puma.rb worker: bundle exec sidekiq -e production -C config/sidekiq.yml

Sidekiq 初始化

Sidekiq init

if Rails.env.production?

  Sidekiq.configure_client do |config|
    config.redis = { url: ENV['REDIS_URL'], size: 2 }
  end

  Sidekiq.configure_server do |config|
    config.redis = { url: ENV['REDIS_URL'], size: 20 }

    Rails.application.config.after_initialize do
      Rails.logger.info("DB Connection Pool size for Sidekiq Server before disconnect is: #{ActiveRecord::Base.connection.pool.instance_variable_get('@size')}")
      ActiveRecord::Base.connection_pool.disconnect!

      ActiveSupport.on_load(:active_record) do
        config = Rails.application.config.database_configuration[Rails.env]
        config['reaping_frequency'] = ENV['DATABASE_REAP_FREQ'] || 10 # seconds
        # config['pool'] = ENV['WORKER_DB_POOL_SIZE'] || Sidekiq.options[:concurrency]
        config['pool'] = 16
        ActiveRecord::Base.establish_connection(config)

        Rails.logger.info("DB Connection Pool size for Sidekiq Server is now: #{ActiveRecord::Base.connection.pool.instance_variable_get('@size')}")
      end
    end
  end

end  

redis 初始化

   if ENV["REDISTOGO_URL"]
    $redis = Redis.new(:url => ENV["REDISTOGO_URL"])
    end

美洲狮.rb

   workers Integer(ENV['WEB_CONCURRENCY'] || 2)  
    threads_count = Integer(ENV['MAX_THREADS'] || 1)  
    threads threads_count, threads_count

    preload_app!

    rackup      DefaultRackup  
    port        ENV['PORT']     || 3000  
    environment ENV['RACK_ENV'] || 'development'

    # Because we are using preload_app, an instance of our app is created by master process (calling our initializers) and then memory space
    # is forked. So we should close DB connection in the master process to avoid connection leaks.
    # https://github.com/puma/puma/issues/303
    # http://stackoverflow.com/questions/17903689/puma-cluster-configuration-on-heroku
    # http://www.rubydoc.info/gems/puma/2.14.0/Puma%2FDSL%3Abefore_fork
    # Dont have to worry about Sidekiq's connection to Redis because connections are only created when needed. As long as we are not
    # queuing workers when rails is booting, there will be no redis connections to disconnect, so it should be fine.
    before_fork do  
      puts "Puma master process about to fork. Closing existing Active record connections."
      ActiveRecord::Base.connection.disconnect!
    end

    on_worker_boot do  
      # Worker specific setup for Rails 4.1+
      # See: https://devcenter.heroku.com/articles/deploying-rails-applications-with-the-puma-web-server#on-worker-boot
      ActiveRecord::Base.establish_connection
    end 

日志错误

             Your Redis connection pool is too small for Sidekiq to work. Your pool has 20 connections but really needs to have at least 22
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq/redis_connection.rb:38:in `verify_sizing'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq/redis_connection.rb:17:in `create'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq.rb:128:in `redis='
10:04:44 worker.1 | /home/ubuntu/workspace/sample_v1/config/initializers/sidekiq.rb:10:in `block in <top (required)>'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq.rb:70:in `configure_server'
10:04:44 worker.1 | /home/ubuntu/workspace/sample_v1/config/initializers/sidekiq.rb:9:in `<top (required)>'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:268:in `load'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:268:in `block in load'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:240:in `load_dependency'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:268:in `load'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/engine.rb:652:in `block in load_config_initializer'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/notifications.rb:166:in `instrument'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/engine.rb:651:in `load_config_initializer'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/engine.rb:616:in `block (2 levels) in <class:Engine>'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/engine.rb:615:in `each'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/engine.rb:615:in `block in <class:Engine>'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:30:in `instance_exec'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:30:in `run'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:55:in `block in run_initializers'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:228:in `block in tsort_each'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:422:in `block (2 levels) in each_strongly_connected_component_from'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:431:in `each_strongly_connected_component_from'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:421:in `block in each_strongly_connected_component_from'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:44:in `each'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:44:in `tsort_each_child'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:415:in `call'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:415:in `each_strongly_connected_component_from'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:349:in `block in each_strongly_connected_component'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:347:in `each'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:347:in `call'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:347:in `each_strongly_connected_component'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:226:in `tsort_each'
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:205:in `tsort_each'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:54:in `run_initializers'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/application.rb:352:in `initialize!'
10:04:44 worker.1 | /home/ubuntu/workspace/sample_v1/config/environment.rb:5:in `<top (required)>'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:274:in `require'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:274:in `block in require'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:240:in `load_dependency'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:274:in `require'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq/cli.rb:237:in `boot_system'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq/cli.rb:50:in `run'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/bin/sidekiq:12:in `<top (required)>'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/bin/sidekiq:23:in `load'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/bin/sidekiq:23:in `<top (required)>'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/cli/exec.rb:63:in `load'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/cli/exec.rb:63:in `kernel_load'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/cli/exec.rb:24:in `run'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/cli.rb:304:in `exec'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/cli.rb:11:in `start'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/exe/bundle:27:in `block in <top (required)>'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/friendly_errors.rb:98:in `with_friendly_errors'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/exe/bundle:19:in `<top (required)>'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/bin/bundle:23:in `load'
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/bin/bundle:23:in `<main>'
10:04:45 worker.1 | exited with code 1
10:04:45 system   | sending SIGTERM to all processes

任何帮助或指导都会很棒.

Any help or sort of guidance will be amazing.

推荐答案

删除 configure_* 块中的 size: 属性.让 Sidekiq 为您管理尺寸.

Remove the size: attribute in your configure_* blocks. Let Sidekiq manage the size for you.

这篇关于配置 Puma 和 Sidekiq的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！