Heroku + Sidekiq：ActiveRecord :: StatementInvalid：PG :: UnableToSend：SSL SYSCALL错误：检测到EOF [英] Heroku + Sidekiq: ActiveRecord::StatementInvalid: PG::UnableToSend: SSL SYSCALL error: EOF detected

问题描述

我们在Heroku的Cedar堆叠上运行Unicorn和Sidekiq。我们间歇性地得到以下错误：

  BurnThis ActiveRecord :: StatementInvalid：PG :: UnableToSend：SSL SYSCALL错误：EOF检测到
 
 ActiveRecord :: StatementInvalid：PG :: ConnectionBad：PQconsumeInput（）SSL SYSCALL错误：连接超时
  

有没有人有任何认识这些错误的直接原因是什么？与我们的数据库连接太多了吗？我们已经按照以下方式建立了分叉：

unicorn.rb

  worker_processes整数（ENV [WEB_CONCURRENCY] || 3）
超时30 
 preload_app true 
 
 before_fork do | server，worker | 
 Signal.trap'TERM'do 
把'Unicorn master拦截TERM并发送自己QUIT'，而不是'
 Process.kill'QUIT'，Process.pid 
 end 
 
定义了？（ActiveRecord :: Base）和
 ActiveRecord :: 
 
 Base.connection.disconnect！ 
结束
 
 after_fork | server，worker | 
 Signal.trap'TERM'do 
让'独角兽工人拦截TERM并且什么也不做。等待master发送QUIT'
 end 
 
＃其他设置
（ActiveRecord :: Base）
 config = Rails.application.config.database_configuration [ Rails.env] 
 config ['adapter'] ='postgis'
 config ['pool'] = ENV ['DB_POOL'] || 5 
 config ['reaping_frequency'] = ENV ['DB_REAP_FREQ'] || 10＃秒
 ActiveRecord :: Base.establish_connection（config）
 end 
 end 
  

和sidekiq.rb

  Sidekiq.configure_server do | config | 
 config.redis = {：url => ENV ['REDIS_URL']，：namespace => 'btsidekiq'} 
 
如果定义了？（ActiveRecord :: Base）
 config = Rails.application.config.database_configuration [Rails.env] 
 config ['adapter'] = 'postgis'
 config ['pool'] = ENV ['DB_POOL'] || 5 
 config ['reaping_frequency'] = ENV ['DB_REAP_FREQ'] || 10＃秒
 ActiveRecord :: Base.establish_connection（config）
 end 
 end 
 
 Sidekiq.configure_client do | config | 
 config.redis = {：url => ENV ['REDIS_URL']，：namespace => 'btsidekiq'} 
 end 
  

我们的数据库池非常大DB_POOL = 100，我们在PG数据库上，显然同时支持500个连接。

解决方案

这个错误是由你的 postgis 适配器尝试使用来自ActiveRecord连接池的陈旧/死亡连接。有两种方法可以解决此问题：
$ b $ ol

调整连接池的大小以匹配线程数/进程数

降低连接收割频率（Reaper为死连接检查池，每N秒）

要实现＃1，您需要设置适合Unicorn和Sidekiq的池大小，可能有不同的需求。
$ b Unicorn是单线程的，所以默认池大小每个流程的5 连接对您而言是正确的。这将为每个 WEB_CONCURRENCY 后端独角兽工作者分配最多5个连接。您应该重置默认池大小并使用您现有的 unicorn.rb ：

  $> heroku config：set DB_POOL = 5 
  

然而Sidekiq使用了一个非常不同的模型。默认情况下，Sidekiq有一个进程和N个线程。您希望比Sidekiq线程的数量稍大的数据库池大小。您可以在 config / initializers / sidekiq.rb 中实现这一点，如下所示：

  Sidekiq.configure_server do | config | 
 pool_size = Sidekiq.options [：concurrency] + 2 
 
 config.redis = {：url => ENV ['REDIS_URL']，：namespace => 'btsidekiq'，：size => pool_size} 
 
如果已定义？（ActiveRecord :: Base）
 config = Rails.application.config.database_configuration [Rails.env] 
 config ['adapter'] ='postgis '
 config ['pool'] = pool_size 
 config ['reaping_frequency'] = ENV ['DB_REAP_FREQ'] || 10＃秒
 ActiveRecord :: Base.establish_connection（config）
 end 
 end 
  

我最好的猜测是，使用如此大的100个连接池，您更可能累积无用的连接。如果这不起作用，你应该尝试减少你的 DB_REAP_FREQ 到5秒。

Hi we are running on Heroku's Cedar stack with Unicorn and Sidekiq. We intermittently get the following errors

BurnThis ActiveRecord::StatementInvalid: PG::UnableToSend: SSL SYSCALL error: EOF detected

ActiveRecord::StatementInvalid: PG::ConnectionBad: PQconsumeInput() SSL SYSCALL error: Connection timed out

Does anyone have any insight what the direct cause of these errors? Is it too many connections to our database? We have our forking set up already in the following way:

unicorn.rb

worker_processes Integer(ENV["WEB_CONCURRENCY"] || 3)
timeout 30
preload_app true

before_fork do |server, worker|
  Signal.trap 'TERM' do
    puts 'Unicorn master intercepting TERM and sending myself QUIT instead'
    Process.kill 'QUIT', Process.pid
  end

  defined?(ActiveRecord::Base) and
    ActiveRecord::

Base.connection.disconnect!
end

after_fork do |server, worker|
  Signal.trap 'TERM' do
    puts 'Unicorn worker intercepting TERM and doing nothing. Wait for master to send QUIT'
  end

  # other setup
  if defined?(ActiveRecord::Base)
    config = Rails.application.config.database_configuration[Rails.env]
    config['adapter'] = 'postgis'
    config['pool']              = ENV['DB_POOL'] || 5
    config['reaping_frequency'] = ENV['DB_REAP_FREQ'] || 10 # seconds
    ActiveRecord::Base.establish_connection(config)
  end
end

And sidekiq.rb

Sidekiq.configure_server do |config|
  config.redis = { :url => ENV['REDIS_URL'], :namespace => 'btsidekiq' }

  if defined?(ActiveRecord::Base)
    config = Rails.application.config.database_configuration[Rails.env]
    config['adapter'] = 'postgis'
    config['pool']              = ENV['DB_POOL'] || 5
    config['reaping_frequency'] = ENV['DB_REAP_FREQ'] || 10 # seconds
    ActiveRecord::Base.establish_connection(config)
  end
end

Sidekiq.configure_client do |config|
  config.redis = { :url => ENV['REDIS_URL'], :namespace => 'btsidekiq' }
end

Our database pool size is pretty large DB_POOL=100 and we are on a PG database that apparently supports 500 connections concurrently.

解决方案

This error is caused by your postgis adapter trying to use a stale/dead connection from the ActiveRecord connection pool. There are two ways to address this issue:

1. Size your connection pool to match the number of threads/process
2. Lower connection reaping frequency (Reaper checks pool for dead connections, every N secs)

To implement #1, you need to set your pool size appropriate for Unicorn and for Sidekiq, which likely have different needs.

Unicorn is single threaded, so the default pool size of 5 connections per process is correct for you. This will allocate up to 5 connections for each of WEB_CONCURRENCY backend unicorn workers. You should reset the default pool size and use your existing unicorn.rb:

$> heroku config:set DB_POOL=5

Sidekiq however uses a very different model. By default, Sidekiq has a single process and N threads. You want a slightly larger DB pool size than the number of Sidekiq threads. You can implement this in your config/initializers/sidekiq.rb as follows:

Sidekiq.configure_server do |config|
  pool_size = Sidekiq.options[:concurrency] + 2

  config.redis = { :url => ENV['REDIS_URL'], :namespace => 'btsidekiq', :size => pool_size }

  if defined?(ActiveRecord::Base)
    config = Rails.application.config.database_configuration[Rails.env]
    config['adapter'] = 'postgis'
    config['pool']              = pool_size
    config['reaping_frequency'] = ENV['DB_REAP_FREQ'] || 10 # seconds
    ActiveRecord::Base.establish_connection(config)
  end
end

My best guess is that using such a large 100 connection pool, you are more likely to accrue dead connections. Sizing the pool appropriately should fix this.

If this does not work, you should try decreasing your DB_REAP_FREQ to 5 seconds.

这篇关于Heroku + Sidekiq：ActiveRecord :: StatementInvalid：PG :: UnableToSend：SSL SYSCALL错误：检测到EOF的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！