ActiveRecord的：处理DB种族工人 [英] ActiveRecord: Handling DB races among workers
问题描述
我对PostgreSQL的9.0上运行一个Rails 3的项目。
I have a Rails 3 project running on top of PostgreSQL 9.0.
使用案例：用户可以要求按照艺术家的名字。要做到这一点，他们提交的名称的列表，以一个REST资源。如果我不能找到艺术家的名字在当地集合，我咨询last.fm有关它们的信息，并在本地缓存信息。这个过程可能需要一些时间，因此委托给所谓的后台作业 IndexArtistJob 。
Use Case: Users can request to follow Artists by name. To do this, they submit a list of names to a REST resource. If I can't find the Artist by name in the local collection, I consult last.fm for information about them, and cache that information locally. This process can take some time, so it is delegated to a background job called IndexArtistJob.
问题： IndexArtistJob 将并行运行。因此，它可能是两个用户可以请求添加相同艺术家在同一时间。两个用户都应该有艺术家添加到他们的收藏，但只有一个艺术家应该结束了在本地数据库。
Problem: IndexArtistJob will be run in parallel. Thus, it is possible that two users may request to add the same Artist at the same time. Both users should have the Artist added to their collection, but only one Artist should end up in the local database.
在艺术家模型的相关部分：
require 'services/lastfm'
class Artist < ActiveRecord::Base
validates_presence_of :name
validates_uniqueness_of :name, :case_sensitive => false
def self.lookup(name)
artist = Artist.find_by_name(name)
return artist if not artist.nil?
info = LastFM.get_artist_info(name)
return if info.nil?
# Check local DB again for corrected name.
if name.downcase != info.name.downcase
artist = Artist.find_by_name(info.name)
return artist if not artist.nil?
end
Artist.new(
:name => info.name,
:image_url => info.image_url,
:bio => info.bio
)
end
end
在 IndexArtistJob 类定义为：
class IndexArtistJob < Struct.new(:user_id, :artist_name)
def perform
user = User.find(user_id)
# May return a new, uncommitted Artist model, or an existing, committed one.
artist = Artist.lookup(artist_name)
return if artist.nil?
# Presume the thread is pre-empted here for a long enough time such that
# the work done by this worker violates the DB's unique constraint.
user.artists << artist
rescue ActiveRecord::RecordNotUnique # Lost race, defer to winning model
user.artists << Artist.lookup(artist_name)
end
end
我想在这里做什么是让每个员工提交新艺术家发现，最好的希望。如果有冲突发生，我希望慢工（S）放弃他们赞成做工作的艺术家这是刚插入，并添加艺术家来指定用户。
What I'm trying to do here is let each worker commit the new Artist it finds, hoping for the best. If a conflict does occur, I want the slower worker(s) to abandon the work they did in favor of the Artist that was just inserted, and add that Artist to the specified user.
我所知道的是Rails的验证并不能代替实际的数据完整性检查在数据库级别的事实。为了解决这个问题，我加上了艺术家表的小写名称字段一个唯一索引来处理这个问题（和用于搜索）。现在，如果我理解正确的文件，一个AR的关联集合提交被添加到项目更改（艺术家在这种情况下）和底层集合在一个事务中。但我不能保证艺术家将被添加。
I'm aware of the fact that Rails validators are no substitute for actual data integrity checking at the level of the database. To handle this, I added a unique index on the Artist table's lowercased name field to handle this (and to use for searching). Now, if I understand the documentation correctly, an AR's association collection commits changes to the item being added (Artist in this case) and the underlying collection in a transaction. But I can't be guaranteed the Artist will be added.
我是否在做正确吗？如果是这样，有没有更好的方式来做到这一点？我觉得构建它周围的异常突出的事实，问题是并发的，因此有点微妙。
Am I doing this correctly? If so, is there a nicer way to do it? I feel like structuring it around exceptions accentuates the fact that the problem is one of concurrency, and thus a bit subtle.
推荐答案
听起来像是你可以使用一个简单的排队机制。你可以使用一个数据库表做到这一点：
Sounds like you could use a simple queuing mechanism. You could do this using a database table:
-
在前端线程发现一个丢失的艺术家，把它写作者姓名表中状态为等待（对艺术家姓名的唯一索引所以这只能发生一次）。
When a "front-end" thread discovers a missing Artist, have it write the Artist name to the table with status "waiting" (have a unique index on Artist name so this can only happen once).
同时在后台线程/进程位于一个循环，并查询表新的就业机会：
一）开始交易
二）发现第一位艺术家，状态=等待
三）更新艺术家的状态为处理
D）交易结束
Meanwhile a background thread/process sits in a loop and queries the table for new jobs:
a) start transaction
b) find first Artist with status="waiting"
c) update Artist status to "processing"
d) end transaction
后台线程，然后索引的艺术家。否则没有人会尝试，因为他们可以看到状态为处理。
The background thread then indexes the Artist. Noone else will try because they can see the status as "processing".
在完成后，后台线程删除表中的艺术家。
When finished, the background thread deletes the Artist from the table.
使用这种方法，你可以运行多个后台线程来提高你的艺术家索引的并发性。
Using this method, you could run multiple background threads to increase concurrency on your Artist indexing.
另外，也要看看像魔豆来管理这一过程。请参见 http://railscasts.com/episodes/243-beanstalkd-and-stalker。
Also look at something like beanstalk to manage this process. See http://railscasts.com/episodes/243-beanstalkd-and-stalker.
这篇关于ActiveRecord的：处理DB种族工人的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
