Sidekiq 部署到多个环境 [英] Sidekiq deploy to multiple environments

问题描述

(我的详细配置见下文，这是 Henley Chiu 回答的结果).

(See below for my detailed config, which is the result of Henley Chiu's answer).

我一直在努力思考 Sidekiq 的部署，但我并没有真正理解它.我在同一台服务器上有一个带有临时环境和生产环境的应用程序.我看到的关于 sidekiq 部署的所有内容基本上都是说只需将 sidekiq/capistrano 添加到您的部署文件中"，所以我这样做了.然后说明是这是一个带有选项的 yml 文件"，但似乎没有任何解释.我需要命名空间吗?我在初始化文件中看到了这一点，但这似乎指向服务器外部.

I've been trying to wrap my brain around Sidekiq deploys, and I am not really getting it. I have an app with a staging environment, and a production environment, on the same server. Everything I see about sidekiq deploys basically say "just add sidekiq/capistrano to your deploy file", so I did that. And then the instructions are "here's a yml file with options" but nothing seems to be explained. Do I need namespaces? I see that in an initialize file, but that seems to be to point outside the server.

我之前部署过，每个阶段似乎都在适当的环境下启动 sidekiq，但它们都从相同的队列中进行处理.我的生产电子邮件试图由阶段 sidekiq 处理，但失败了.我暂时停止了我的舞台，但最终我需要再次使用它.我希望我不是很密​​集，我真的试图理解这一点，只是很难找到一个明确的这是如何完成的".

I deployed earlier, and each stage seems to boot sidekiq up with the proper environment, but they both process from the same queues. My emails from production were trying to be processed by the stage sidekiq, and failing. I stopped my stage for now, but eventually I will need to use it again. I hope I'm not being dense, I've really tried to understand this and am just having a hard time with finding a definitive "here's how it's done".

就其价值而言，这里是 config/sidekiq.yml(在部署期间加载良好):

For what it's worth, here is config/sidekiq.yml (which is loaded fine during the deploy):

:concurrency: 5
:verbose: false
:pidfile: ./tmp/pids/sidekiq.pid
:logfile: ./log/sidekiq.log
:queues:
  - [carrierwave, 7]
  - [client_emails, 5]
  - [default, 3]
staging:
  :concurrency: 10
production:
  :concurrency: 25

日志文件和 pid 似乎在正确的位置，但队列只是合并了.任何帮助都会很棒！

Log files, and pids seem to be in the right spot, but the queues are just merged. Any help would be GREAT!

另外，如果有关系:

Rails 3.2.11, passenger, nginx, rvm, Ubuntu 12.10, and Ruby 1.9.3

详细配置(答案):

首先，我在端口 7777(或除默认 6379 之外的任何端口)设置了一个新的 redis 服务器.几乎遵循了我第一次使用的 redis 快速入门指南.

然后我制作了initilizer文件；这有客户端和服务器配置.两者都需要使 sidekiq 多阶段工作.

Then I made the initilizer file; this has both the client and the server config. Both are required to make sidekiq work multistage.

请注意，我使用的是外部 YAML 文件进行设置.我正在使用 SettingsLogic 来让事情变得更简单，但你也可以同样轻松地通过包含文件自己做.通过使用 yaml 文件，我们不必接触我们的环境/暂存或生产文件.

Note that I am using an external YAML file for the settings. I am using SettingsLogic for this to make things easier, but you can just as easily do this yourself by including the file. By using a yaml file, we don't have to touch our environments/staging or production files.

# config/initializers/sidekiq.rb
server = Settings.redis.server
port = Settings.redis.port
db_num = Settings.redis.db_num
namespace = Settings.redis.namespace

Sidekiq.configure_server do |config|  
  config.redis = { url: "redis://#{server}:#{port}/#{db_num}", namespace: namespace  }
end

我正在使用乘客 - sidekiq wiki 的故障排除页面 建议在使用独角兽或乘客时更改设置，所以我在那里添加了客户端设置的代码:

I am using passenger - the troubleshooting page of the sidekiq wiki recommends a change for the setup when using unicorn or passenger, so I added the code there for the client setup:

# config/initializers/sidekiq.rb (still)
if defined?(PhusionPassenger)
  PhusionPassenger.on_event(:starting_worker_process) do |forked|
    Sidekiq.configure_client do |config|
      config.redis = { url: "redis://#{server}:#{port}/#{db_num}", namespace: namespace }
    end if forked
  end
end

这是我的设置文件(显然值已更改):

This is my Settings file (obviously values changed):

#config/settings.yml
defaults: &defaults
  redis: &redis_defaults
    server: 'localhost'
    port: 6379
    db_num: 0
    namespace: 'sidekiq_development'

development:
  <<: *defaults

test:
  <<: *defaults

staging:
  <<: *defaults
  redis:
    <<: *redis_defaults
    port: 8888
    namespace: 'sidekiq_staging'

production:
  <<: *defaults
  redis:
    <<: *redis_defaults
    port: 7777
    namespace: 'sidekiq_production'

我发现将命名空间添加到 config/sidekiq.yml 文件似乎不起作用 - sidekiq 会在部署时使用正确的端口启动，但实际上不会处理任何事情.但是由于 wiki 推荐使用命名空间，我最终只是将它添加到 init 文件中.

I found that adding the namespace to the config/sidekiq.yml file didn't seem to work - sidekiq would boot on deploy using the right port, but wouldn't actually process anything. But since the wiki recommends using a namespace, I ended up just adding it to the init file.

我希望这对其他人有帮助，因为这对我来说真的很难理解，因为之前没有做过很多这样的设置.

I hope this helpful for others, because this was really hard for me to understand, having not done a lot of this kind of setup before.

推荐答案

在您的 initializers/sidekiq.rb 文件中，您指定所有环境启动时使用的 Redis 队列.对我来说是:

In your initializers/sidekiq.rb file, you specify the Redis queue all environments boot up with. For mine it is:

redisServer = "localhost"
Sidekiq.configure_server do |config|
  config.redis = { :url => 'redis://' + redisServer + ':6379/0' }
end

如果您希望每个环境从单独的队列中进行处理，您可以在每个环境的环境文件夹中拥有特定的 sidekiq.rb 文件.每个都有不同的 redis 服务器.

If you want each environment to process from separate queues, you can have specific sidekiq.rb files in the environments folder for each environment. Each with different redis servers.

这篇关于Sidekiq 部署到多个环境的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！