在Rails中的多个数据库之间切换而不会中断事务 [英] Switching between multiple databases in Rails without breaking transactions

问题描述

我正在设置带有多个数据库的Rails应用程序.它使用ActiveRecord::Base.establish_connection db_config在数据库之间切换(所有数据库均在database.yml中配置).

I am setting up a Rails app with multiple databases. It uses ActiveRecord::Base.establish_connection db_config to switch between the databases (all of which are configured in database.yml).

establish_connection显然会中断每个呼叫的未决事务.负面影响之一是测试，必须禁用use_transactional_tests(导致不希望的缓慢测试).

establish_connection apparently breaks pending transactions on each call. One negative consequence is testing, where use_transactional_tests must be disabled (leading to undesirably slow tests).

那么... Rails应用程序如何同时在不同数据库上维护多个事务? (为澄清起见，我并不是在寻找花哨的跨数据库事务.这是数据库客户端(即Rails应用程序)同时维护多个事务的一种方法，每个数据库一个.)

So ... how can a Rails app maintain multiple transactions on different databases at the same time? (To clarify, I'm not looking for a fancy cross-database transaction. Just a way for the database client, ie the Rails app, to maintain multiple transactions simultaneously, one per database.)

我看到的唯一解决方案是直接放入establish_connection在类定义中，但是假设您有一个专门用于特定类的数据库.我正在应用一种基于用户的分片策略，该策略将单个记录类型分布在多个数据库中，因此需要在代码中动态切换数据库.

The only solution I've seen is putting establish_connection directly in the class definition, but that assumes you have a database dedicated to specific classes. I am applying a user-based sharding strategy, where a single record type is distributed across multiple databases, so the database needs to be switched dynamically in code.

推荐答案

这是一个棘手的问题，因为ActiveRecord内部紧密耦合，但是我设法创建了一些可行的概念证明.或至少看起来像是可行.

This is a tricky problem, because of tight coupling inside ActiveRecord, but I've managed to create some proof of concept that works. Or at least it looks like it works.

某些背景

ActiveRecord使用一个ActiveRecord::ConnectionAdapters::ConnectionHandler类，该类负责按模型存储连接池.默认情况下，所有型号都只有一个连接池，因为通常的Rails应用程序已连接到一个数据库.

ActiveRecord uses a ActiveRecord::ConnectionAdapters::ConnectionHandler class that is responsible for storing connection pools per model. By default there is only one connection pool for all models, because usual Rails app is connected to one database.

对于特定模型中的其他数据库执行establish_connection之后，将为该模型创建新的连接池.以及所有可能从中继承的模型.

After executing establish_connection for different database in particular model, new connection pool is created for that model. And also for all models that may inherit from it.

在执行任何查询之前，ActiveRecord首先为相关模型检索连接池，然后从该池中检索连接.

Before executing any query, ActiveRecord first retrieves connection pool for relevant model and then retrieves the connection from the pool.

请注意，以上解释可能并非100％准确，但应接近.

Note that above explanation may not be 100% accurate, but it should be close.

解决方案

因此，我们的想法是将默认连接处理程序替换为自定义连接处理程序，该处理程序将根据提供的分片描述返回连接池.

So the idea is to replace the default connection handler with custom one that will return connection pool based on provided shard description.

这可以通过许多不同的方式来实现.我是通过创建将碎片名称作为伪装的ActiveRecord类传递的代理对象来实现的.连接处理程序期望获得AR模型，并查看name属性，还查看superclass以遍历模型的层次结构链.我已经实现了DatabaseModel类，该类基本上是分片名称，但是它的行为类似于AR模型.

This can be implemented in many different ways. I did it by creating the proxy object that is passing shard names as disguised ActiveRecord classes. Connection handler is expecting to get AR model and looks at name property and also at superclass to walk the hierarchy chain of model. I've implemented DatabaseModel class that is basically shard name, but it is behaving like AR model.

实施

这里是示例实现.为了简化起见，我使用了sqlite数据库，您无需任何设置即可运行此文件.您还可以查看此要点

Here is example implementation. I've used sqlite database for simplicity, you can just run this file without any setup. You can also take a look at this gist

# Define some required dependencies
require "bundler/inline"
gemfile(false) do
  source "https://rubygems.org"
  gem "activerecord", "~> 4.2.8"
  gem "sqlite3"
end

require "active_record"

class User < ActiveRecord::Base
end

DatabaseModel = Struct.new(:name) do
  def superclass
    ActiveRecord::Base
  end
end

# Setup database connections and create databases if not present
connection_handler = ActiveRecord::ConnectionAdapters::ConnectionHandler.new
resolver = ActiveRecord::ConnectionAdapters::ConnectionSpecification::Resolver.new({
  "users_shard_1" => { adapter: "sqlite3", database: "users_shard_1.sqlite3" },
  "users_shard_2" => { adapter: "sqlite3", database: "users_shard_2.sqlite3" }
})

databases = %w{users_shard_1 users_shard_2}
databases.each do |database|
  filename = "#{database}.sqlite3"

  ActiveRecord::Base.establish_connection({
    adapter: "sqlite3",
    database: filename
  })

  spec = resolver.spec(database.to_sym)
  connection_handler.establish_connection(DatabaseModel.new(database), spec)

  next if File.exists?(filename)

  ActiveRecord::Schema.define(version: 1) do
    create_table :users do |t|
      t.string :name
      t.string :email
    end
  end
end

# Create custom connection handler
class ShardHandler
  def initialize(original_handler)
    @original_handler = original_handler
  end

  def use_database(name)
    @model= DatabaseModel.new(name)
  end

  def retrieve_connection_pool(klass)
    @original_handler.retrieve_connection_pool(@model)
  end

  def retrieve_connection(klass)
    pool = retrieve_connection_pool(klass)
    raise ConnectionNotEstablished, "No connection pool for #{klass}" unless pool
    conn = pool.connection
    raise ConnectionNotEstablished, "No connection for #{klass} in connection pool" unless conn
    puts "Using database \"#{conn.instance_variable_get("@config")[:database]}\" (##{conn.object_id})"
    conn
  end
end

User.connection_handler = ShardHandler.new(connection_handler)

User.connection_handler.use_database("users_shard_1")
User.create(name: "John Doe", email: "[e-mail]")
puts User.count

User.connection_handler.use_database("users_shard_2")
User.create(name: "Jane Doe", email: "[e-mail]")
puts User.count

User.connection_handler.use_database("users_shard_1")
puts User.count

我认为这应该为如何实施生产就绪解决方案提供一个思路.我希望我不会在这里错过任何明显的事情.我可以建议几种不同的方法:

I think this should give an idea how to implement production ready solution. I hope I didn't miss anything obvious here. I can suggest couple of different approaches:

1. 子类ActiveRecord::ConnectionAdapters::ConnectionHandler并覆盖负责检索连接池的那些方法
2. 创建全新的类，实现与ConnectionHandler
相同的api
3. 我想也可以只覆盖retrieve_connection方法.我不记得它的定义位置，但是我认为它在ActiveRecord::Core中.

1. Subclass ActiveRecord::ConnectionAdapters::ConnectionHandler and overwrite those methods responsible for retrieving connection pools
2. Create completely new class implementing the same api as ConnectionHandler
3. I guess it is also possible to just overwrite retrieve_connection method. I don't remember where it is defined, but I think it is in ActiveRecord::Core.

我认为方法1和方法2是可行的方法，并且在处理数据库时应涵盖所有情况.

I think approaches 1 and 2 are the way to go and should cover all cases when working with databases.

这篇关于在Rails中的多个数据库之间切换而不会中断事务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！