SqlAlchemy 对只读对象模型的优化 [英] SqlAlchemy optimizations for read-only object models

问题描述

我有一个使用 sqlalchemy ORM 映射从 sqlite 数据库生成的复杂对象网络.我有很多深度嵌套:

I have a complex network of objects being spawned from a sqlite database using sqlalchemy ORM mappings. I have quite a few deeply nested:

for parent in owner.collection: 
    for child in parent.collection: 
        for foo in child.collection: 
            do lots of calcs with foo.property 

我的分析显示 sqlalchemy 检测在这个用例中花费了大量时间.

My profiling is showing me that the sqlalchemy instrumentation is taking a lot of time in this use case.

问题是:我从不在运行时更改对象模型(映射属性)，因此一旦加载它们，我就不需要检测，甚至根本不需要任何 sqlalchemy 开销.经过大量研究，我想我可能需要从我已经加载的检测对象"中克隆一组纯 python"对象，但这会很痛苦.

The thing is: I don't ever change the object model (mapped properties) at runtime, so once they are loaded I don't NEED the instrumentation, or indeed any sqlalchemy overhead at all. After much research, I'm thinking I might have to clone a 'pure python' set of objects from my already loaded 'instrumented objects', but that would be a pain.

性能在这里真的很重要(它是一个模拟器)，所以也许最好直接使用 sqlite api 将这些层编写为 C 扩展.有什么想法吗?

Performance is really crucial here (it's a simulator), so maybe writing those layers as C extensions using sqlite api directly would be best. Any thoughts?

推荐答案

如果您多次引用单个实例的单个属性，一个简单的技巧就是将其存储在局部变量中.

If you reference a single attribute of a single instance lots of times, a simple trick is to store it in a local variable.

如果你想要一种创建廉价纯 python 克隆的方法，请与原始对象共享 dict 对象:

If you want a way to create cheap pure python clones, share the dict object with the original object:

class CheapClone(object):
    def __init__(self, original):
        self.__dict__ = original.__dict__

创建这样的副本花费大约一半的检测属性访问和属性查找与正常一样快.

Creating a copy like this costs about half of the instrumented attribute access and attribute lookups are as fast as normal.

也许还有一种方法可以让映射器创建未检测类的实例而不是检测类.如果我有时间，我可能会看看填充实例与检测类属于同一类型的假设是多么根深蒂固.

There might also be a way to have the mapper create instances of an uninstrumented class instead of the instrumented one. If I have some time, I might take a look how deeply ingrained is the assumption that populated instances are of the same type as the instrumented class.

找到一种快速而肮脏的方法，它似乎至少在 0.5.8 和 0.6 上有些工作.没有使用继承或其他可能交互不良的功能对其进行测试.另外，这涉及到一些非公开的 API，所以在更改版本时要注意损坏.

Found a quick and dirty way that seems to at least somewhat work on 0.5.8 and 0.6. Didn't test it with inheritance or other features that might interact badly. Also, this touches some non-public API's, so beware of breakage when changing versions.

from sqlalchemy.orm.attributes import ClassManager, instrumentation_registry

class ReadonlyClassManager(ClassManager):
    """Enables configuring a mapper to return instances of uninstrumented 
    classes instead. To use add a readonly_type attribute referencing the
    desired class to use instead of the instrumented one."""
    def __init__(self, class_):
        ClassManager.__init__(self, class_)
        self.readonly_version = getattr(class_, 'readonly_type', None)
        if self.readonly_version:
            # default instantiation logic doesn't know to install finders
            # for our alternate class
            instrumentation_registry._dict_finders[self.readonly_version] = self.dict_getter()
            instrumentation_registry._state_finders[self.readonly_version] = self.state_getter()

    def new_instance(self, state=None):
        if self.readonly_version:
            instance = self.readonly_version.__new__(self.readonly_version)
            self.setup_instance(instance, state)
            return instance
        return ClassManager.new_instance(self, state)

Base = declarative_base()
Base.__sa_instrumentation_manager__ = ReadonlyClassManager

使用示例:

class ReadonlyFoo(object):
    pass

class Foo(Base, ReadonlyFoo):
    __tablename__ = 'foo'
    id = Column(Integer, primary_key=True)
    name = Column(String(32))

    readonly_type = ReadonlyFoo

assert type(session.query(Foo).first()) is ReadonlyFoo

这篇关于SqlAlchemy 对只读对象模型的优化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！