如何使Python对象在Web应用程序中持久化？ [英] How would one make Python objects persistent in a web-app?
问题描述
我在写一个相当复杂的网络应用程序。 Python后端运行一种算法，其状态取决于存储在几个相互关联的数据库表中的数据，这些数据库不会经常更改，以及经常更改的用户特定数据。当用户使用应用程序时，算法的每个用户状态经历许多小的改变。由于性能原因，对来自（半标准化）数据库的每个请求重新初始化状态，因此，每个用户的工作中经常使用此算法来做出某些重要的决定。
<数据迅速变得不可行。例如，高度优选地以某种方式高速缓存状态的Python对象，使得它可以在必要时被简单地使用和/或更新。但是，由于这是一个Web应用程序，因此有几个进程为请求提供服务，因此使用全局变量是不成问题的。
我已经尝试序列化相关对象通过pickle）并将序列化数据保存到DB，现在我正在通过memcached缓存序列化数据。然而，这仍然有很大的开销序列化和反序列化对象经常。
我已经看过共享内存解决方案，但我发现唯一相关的事情是< a href =http://poshmodule.sourceforge.net/ =nofollow noreferrer> POSH 。然而，POSH似乎没有被广泛使用，我不觉得很容易集成这样的实验组件到我的应用程序。
我需要一些建议！这是我开发Web应用程序的第一枪，所以我希望这是一个常见的问题，有这样的问题的知名解决方案。在这一点上，假定Python后端在单个服务器上运行的解决方案就足够了，但对于扩展到多个服务器的解决方案来说，额外的点：）
注意：
- 我有此应用程序正在运行，目前正在运行，并与活动用户。我开始没有做任何过早的优化，然后根据需要进行优化。我做了测量和测试，以确保上述问题是实际的瓶颈。我相信我肯定可以从当前的设置中挤出更多的性能，但我想问一个更好的方法。
- 设置本身仍然是一个进行中的工作;假设系统的架构可以是任何适合您的解决方案。
a href =http://docs.python.org/library/multiprocessing.html =nofollow noreferrer> multiprocessing 框架有可能适用于此处 - 即共享ctypes模块。
多处理对Python来说是新的，所以它可能有一些奇怪。我不太确定解决方案是否适用于未通过 multiprocessing 生成的进程。
I'm writing a reasonably complex web application. The Python backend runs an algorithm whose state depends on data stored in several interrelated database tables which does not change often, plus user specific data which does change often. The algorithm's per-user state undergoes many small changes as a user works with the application. This algorithm is used often during each user's work to make certain important decisions.
For performance reasons, re-initializing the state on every request from the (semi-normalized) database data quickly becomes non-feasible. It would be highly preferable, for example, to cache the state's Python object in some way so that it can simply be used and/or updated whenever necessary. However, since this is a web application, there several processes serving requests, so using a global variable is out of the question.
I've tried serializing the relevant object (via pickle) and saving the serialized data to the DB, and am now experimenting with caching the serialized data via memcached. However, this still has the significant overhead of serializing and deserializing the object often.
I've looked at shared memory solutions but the only relevant thing I've found is POSH. However POSH doesn't seem to be widely used and I don't feel easy integrating such an experimental component into my application.
I need some advice! This is my first shot at developing a web application, so I'm hoping this is a common enough issue that there are well-known solutions to such problems. At this point solutions which assume the Python back-end is running on a single server would be sufficient, but extra points for solutions which scale to multiple servers as well :)
Notes:
- I have this application working, currently live and with active users. I started out without doing any premature optimization, and then optimized as needed. I've done the measuring and testing to make sure the above mentioned issue is the actual bottleneck. I'm sure pretty sure I could squeeze more performance out of the current setup, but I wanted to ask if there's a better way.
- The setup itself is still a work in progress; assume that the system's architecture can be whatever suites your solution.
I think that the multiprocessing framework has what might be applicable here - namely the shared ctypes module.
Multiprocessing is fairly new to Python, so it might have some oddities. I am not quite sure whether the solution works with processes not spawned via multiprocessing.
这篇关于如何使Python对象在Web应用程序中持久化？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
