Twisted + SQLAlchemy及其最佳方法 [英] Twisted + SQLAlchemy and the best way to do it

问题描述

所以我要编写另一个基于Twisted的守护程序。它会像往常一样具有xmlrpc接口，因此我可以轻松地与它进行通信，并让其他进程根据需要与它交换数据。

So I'm writing yet another Twisted based daemon. It'll have an xmlrpc interface as usual so I can easily communicate with it and have other processes interchange data with it as needed.

该守护程序需要访问数据库。我们一直在使用SQL Alchemy代替用于最新项目的硬编码SQL字符串，这些字符串主要用于Pylons中的Web应用程序。

This daemon needs to access a database. We've been using SQL Alchemy in place of hard coding SQL strings for our latest projects - those mostly done for web apps in Pylons.

与此应用程序相同，并重复使用利用SQL Alchemy的库代码。那么该怎么办？当然，由于该库是为在Pylons应用中使用而编写的，因此所有人都已经习惯了所有简单明了的阻塞样式代码，并且所有非阻塞都由Pylons通过线程，线程局部变量，作用域会话等神奇地处理。

We'd like to do the same for this app and re-use library code that makes use of SQL Alchemy. So what to do? Well of course since that library was written for use in a Pylons app it's all the straight-forward blocking style code that everyone is accustomed to and all of the non-blocking is magically handled by Pylons via threading, thread locals, scoped sessions and so on.

所以对于Twisted，我想我有点卡住了。我可以：

So now for Twisted I guess I'm a bit stuck. I could:

1. 只要它很小，就直接写我需要的sql，并在需要时使用dbapi池扭曲执行runInteractions等。


2. 在我们的库中使用对象和固有的阻塞方法，并在我的Twisted守护进程中不时阻塞。 Bah。


3. 使用sAsync，该方法最近一次更新是在2008年，可以重用我们已经定义的模型，但实际上不是，并且不能解决库代码也需要在Pylons中工作的问题。 。这甚至可以与最新版本的SQL Alchemy一起使用吗？谁知道。这个项目看起来很棒-为什么它显然被放弃了？


4. 产生一个单独的子进程，使其处理库代码及其所有阻塞，当对象通过xmlrpc通过YAML编组时，结果准备好时返回到我的守护程序。


5. 使用deferToThread，然后清除已返回的对象并确保进行急切的加载，这样我就可以拥有所有需要的东西。

1. Just write the sql I need directly if it's minimal and use the dbapi pool in twisted to do runInteractions etc when I need to hit the db.
2. Use the objects and inherently blocking methods in our library and block now and then in my Twisted daemon. Bah.
3. Use sAsync which was last updated in 2008 and kind of reuse the models we have defined already but not really and this doesn't address that the library code needs to work in Pylons too. Does that even work with the latest version SQL Alchemy? Who knows. That project looked great though - why was it apparently abandoned?
4. Spawn a separate subprocess and have it deal with the library code and all it's blocking, the results being returned back to my daemon when ready as objects marshalled via YAML over xmlrpc.
5. Use deferToThread and then expunge the objects returned having made sure to do eager loads so that I have all my stuff that I might need. Seems kind of ugha to me.

我也被Python 2.5.4 atm困住了，所以还没有2.6，但我没有我想我可以从将来进行导入就可以访问那里很酷的新多处理模块。

I'm also stuck using Python 2.5.4 atm so no 2.6 yet and I don't think I can just do an import from future to get access to the cool new multiprocessing module stuff in there. That's OK though I guess as we've got dealing with interprocess communication down pretty well.

所以我认为我们可以很好地处理进程间通信，所以我倾向于使用选项4，因为这样可以避免致命的错误。使用选项1进行逻辑复制，同时远离线程。

So I'm leaning towards option 4 mostly as that would avoid the mortal sin of logic duplication with option 1 while also staying the heck away from threads.

我的第一个尝试是将选项放开，然后将对库代码的调用分离出来，如果看起来像一个很有可能阻止某件事花费太长时间。伤心。也许在这里将Stackless Python和Twisted结合起来会很有趣。

My first attempt though will be option 2 to just get the thing going and then separate out the calls to the library code perhaps into a separate process if it looks like there's a good chance that something might take a bit too long to block on. Sad. Maybe a combination of Stackless Python and Twisted would be interesting here.

还有更好的主意吗？

推荐答案

首先，不幸的是，我只能赞同你的观点，即扭曲和
SQLAlchemy表现不佳。我曾经和
一起工作过，并且有些担心
将它们放在一起会带来的复杂性。

Firstly, I can unfortunately only second your opinion that twisted and SQLAlchemy don't play along very well. I have worked some with both and would be somewhat afraid of the complexity that would arise from putting them together.

所有数据库集成层我到目前为止所知道的使用
twisteds线程集成层，并且如果您想避免在
的所有花费，则您的列表中第4点几乎固定不变。

All the database integration layers that I know of to date use twisteds threading integration layer, and if you want to avoid that at all costs you are pretty much stuck with point 4 in your list.

另一方面，我看到了使用deferToThread（）和朋友工作得很好的数据库连接代码
的示例。

On the other hand, I have seen examples of database connecting code using deferToThread() and friends that worked very well.

无论如何，如果您愿意考虑使用除SQLAlchemy之外的其他框架
，则可以使用一些指针：

Anyway, some pointers if you'd be ready to consider other frameworks than SQLAlchemy:

DivMod伙计们一直在对绞线进行一些尝试性工作-
基于Storm ORM的数据库集成（Google表示 storm orm）。

The DivMod guys have been doing some tentative work on twisted - database integration based on the Storm ORM (google for "storm orm").

有关示例，请参见以下链接：

See this link for an example:

http://divmod.readthedocs.org / en / latest / products / nev ow / storm-approach.html

另外，请转到DivMod的站点，并查看其Axiom db层的
来源（因为
仅是Sqlite，可能对您没有直接用处，但是它的原理可能会有用）。

Also, head over to DivMod's site and have a look at the sources of their Axiom db layer (probably not of any use to you directly since it's Sqlite only, but it's principles might be useful).

这篇关于Twisted + SQLAlchemy及其最佳方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！