Twisted + SQLAlchemy 和最好的方法 [英] Twisted + SQLAlchemy and the best way to do it

问题描述

所以我正在编写另一个基于 Twisted 的守护进程.它将像往常一样有一个 xmlrpc 接口，这样我就可以轻松地与它通信，并让其他进程根据需要与其交换数据.

So I'm writing yet another Twisted based daemon. It'll have an xmlrpc interface as usual so I can easily communicate with it and have other processes interchange data with it as needed.

这个守护进程需要访问一个数据库.我们一直在使用 SQL Alchemy 来代替我们最新项目的硬编码 SQL 字符串——这些项目主要是为 Pylons 中的网络应用程序完成的.

This daemon needs to access a database. We've been using SQL Alchemy in place of hard coding SQL strings for our latest projects - those mostly done for web apps in Pylons.

我们希望对这个应用程序做同样的事情，并重用利用 SQL Alchemy 的库代码.那么该怎么办?当然，因为该库是为在 Pylons 应用程序中使用而编写的，所以它是每个人都习惯的所有直接阻塞式代码，并且所有非阻塞都由 Pylons 通过线程、线程局部变量、作用域会话等神奇地处理

We'd like to do the same for this app and re-use library code that makes use of SQL Alchemy. So what to do? Well of course since that library was written for use in a Pylons app it's all the straight-forward blocking style code that everyone is accustomed to and all of the non-blocking is magically handled by Pylons via threading, thread locals, scoped sessions and so on.

所以现在对于 Twisted 我想我有点卡住了.我可以:

So now for Twisted I guess I'm a bit stuck. I could:

1. 如果它是最小的，只需直接编写我需要的 sql，并在需要访问数据库时使用扭曲的 dbapi 池来执行 runInteractions 等.
2. 在我们的库中使用对象和固有的阻塞方法，并在我的 Twisted 守护进程中时不时地阻塞.呸.
3. 使用上次更新于 2008 年的 sAsync 并重新使用我们已经定义的模型，但实际上并非如此，这并没有解决库代码也需要在 Pylons 中工作的问题.这甚至适用于最新版本的 SQL Alchemy 吗?谁知道.不过那个项目看起来很棒——为什么它显然被放弃了?
4. 生成一个单独的子进程并让它处理库代码及其所有阻塞，准备好后将结果作为通过 xmlrpc 上的 YAML 编组的对象返回给我的守护进程.
5. 使用 deferToThread，然后清除返回的对象，确保执行急切加载，以便我拥有可能需要的所有内容.对我来说似乎有点恶心.

我也一直在使用 Python 2.5.4 atm，所以还没有 2.6，我认为我不能只从未来进行导入来访问那里很酷的新多处理模块的东西.不过我想这没问题，因为我们已经很好地处理了进程间通信.

I'm also stuck using Python 2.5.4 atm so no 2.6 yet and I don't think I can just do an import from future to get access to the cool new multiprocessing module stuff in there. That's OK though I guess as we've got dealing with interprocess communication down pretty well.

所以我主要倾向于选项 4，因为这样可以避免选项 1 的逻辑重复的致命罪恶，同时也远离线程.

So I'm leaning towards option 4 mostly as that would avoid the mortal sin of logic duplication with option 1 while also staying the heck away from threads.

我的第一次尝试将是选项 2，只是让事情继续进行，然后将库代码的调用分离出来，如果看起来很可能需要很长时间才能阻止某些事情在.伤心.也许 Stackless Python 和 Twisted 的组合在这里会很有趣.

My first attempt though will be option 2 to just get the thing going and then separate out the calls to the library code perhaps into a separate process if it looks like there's a good chance that something might take a bit too long to block on. Sad. Maybe a combination of Stackless Python and Twisted would be interesting here.

有更好的想法吗?

推荐答案

首先，很遗憾，我只能赞同你那扭曲和SQLAlchemy 不能很好地配合.我和两者都工作过并且会有点害怕由此产生的复杂性把它们放在一起.

Firstly, I can unfortunately only second your opinion that twisted and SQLAlchemy don't play along very well. I have worked some with both and would be somewhat afraid of the complexity that would arise from putting them together.

迄今为止我所知道的所有数据库集成层都在使用扭曲线程集成层，如果你想避免在所有费用您几乎都停留在清单中的第 4 点.

All the database integration layers that I know of to date use twisteds threading integration layer, and if you want to avoid that at all costs you are pretty much stuck with point 4 in your list.

另一方面，我看过数据库连接代码的例子使用 deferToThread() 和效果很好的朋友.

On the other hand, I have seen examples of database connecting code using deferToThread() and friends that worked very well.

无论如何，如果您准备考虑其他框架，请提供一些建议比 SQLAlchemy:

Anyway, some pointers if you'd be ready to consider other frameworks than SQLAlchemy:

DivMod 的家伙们一直在做一些关于扭曲的试探性工作——基于 Storm ORM 的数据库集成(谷歌为storm orm").

The DivMod guys have been doing some tentative work on twisted - database integration based on the Storm ORM (google for "storm orm").

查看此链接以获取示例:

See this link for an example:

http://divmod.readthedocs.org/en/latest/products/nevow/storm-approach.html

另外，前往 DivMod 的网站并查看来源他们的 Axiom db 层(可能对你没有任何用处，因为它只是 Sqlite，但它的原则可能有用).

Also, head over to DivMod's site and have a look at the sources of their Axiom db layer (probably not of any use to you directly since it's Sqlite only, but it's principles might be useful).

这篇关于Twisted + SQLAlchemy 和最好的方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！