Django 动态模型字段 [英] Django dynamic model fields
问题描述
我正在开发一个多租户应用程序，其中一些用户可以定义自己的数据字段(通过管理员)以收集表单中的其他数据并报告数据.后者使 JSONField 不是一个很好的选择，所以我有以下解决方案:
class CustomDataField(models.Model):"""任意数据字段的抽象规范.不用于保存数据本身，而是用于保存字段的元数据."""site = models.ForeignKey(Site, default=settings.SITE_ID)name = models.CharField(max_length=64)元类:抽象 = 真类 CustomDataValue(models.Model):"""任意数据的抽象规范."""值=models.CharField(max_length=1024)元类:抽象 = 真请注意 CustomDataField 如何具有站点的外键 - 每个站点将具有一组不同的自定义数据字段，但使用相同的数据库.那么各个具体的数据字段可以定义为:
class UserCustomDataField(CustomDataField):经过类 UserCustomDataValue(CustomDataValue):custom_field = models.ForeignKey(UserCustomDataField)用户 = 模型.外键(用户，related_name='custom_data')元类:unique_together=(('user','custom_field'),)这导致以下用途:
custom_field = UserCustomDataField.objects.create(name='zodiac', site=my_site) #可能是在管理员中创建的user = User.objects.create(username='foo')user_sign = UserCustomDataValue(custom_field=custom_field, user=user, data='Libra')user.custom_data.add(user_sign) #实际上，这到底是做什么的?但这感觉很笨重，尤其是需要手动创建相关数据并将其与具体模型相关联.有没有更好的方法?
被抢先丢弃的选项:
- 自定义 SQL 可即时修改表.部分是因为这不会扩展，部分是因为它太黑客了.
- 无架构解决方案，如 NoSQL.我不反对他们，但他们仍然不合适.最终，这些数据是输入的，并且存在使用第三方报告应用程序的可能性.
- JSONField，如上所述，因为它不适用于查询.
截至今天，有四种可用的方法，其中两种需要特定的存储后端:
Django-eav(原包是不再维护，但有一些蓬勃发展的分叉)
此解决方案基于实体属性值数据模型，本质上，它使用多个表来存储对象的动态属性.此解决方案的重要部分在于:
- 使用几个纯粹简单的Django模型来表示动态字段，这使得它易于理解且与数据库无关；
允许您使用以下简单命令有效地将动态属性存储附加/分离到 Django 模型:
eav.unregister(遭遇)eav.register(患者)同时非常强大.
缺点:
- 效率不高.这更多是对 EAV 模式本身的批评，它需要手动将数据从列格式合并到模型中的一组键值对.
- 更难维护.维护数据完整性需要多列唯一键约束，这在某些数据库上可能效率低下.
- 您需要选择其中一个fork，因为官方软件包没有维护时间更长，没有明确的领导者.
用法非常简单:
导入eav从 app.models 导入 Patient, Encountereav.register(遭遇)eav.register(患者)Attribute.objects.create(name='age', datatype=Attribute.TYPE_INT)Attribute.objects.create(name='height', datatype=Attribute.TYPE_FLOAT)Attribute.objects.create(name='weight', datatype=Attribute.TYPE_FLOAT)Attribute.objects.create(name='city', datatype=Attribute.TYPE_TEXT)Attribute.objects.create(name='country', datatype=Attribute.TYPE_TEXT)self.yes = EnumValue.objects.create(value='yes')self.no = EnumValue.objects.create(value='no')self.unkown = EnumValue.objects.create(value='unkown')ynu = EnumGroup.objects.create(name='Yes/No/Unknown')ynu.enums.add(self.yes)ynu.enums.add(self.no)ynu.enums.add(self.unkown)Attribute.objects.create(name='fever', datatype=Attribute.TYPE_ENUM,enum_group=ynu)# 当您在 EAV 中注册模型时，# 您可以访问所有 EAV 属性:Patient.objects.create(name='Bob', eav__age=12,eav__fever=no, eav__city='New York',eav__country='美国')# 您可以根据 EAV 字段过滤查询:query1 = Patient.objects.filter(Q(eav__city__contains='Y'))query2 = Q(eav__city__contains='Y') |Q(eav__fever=no)PostgreSQL 中的 Hstore、JSON 或 JSONB 字段
PostgreSQL 支持几种更复杂的数据类型.大多数都通过第三方包提供支持，但近年来 Django 已将它们纳入 django.contrib.postgres.fields.
HStoreField:
Django-hstore 原本是第三方包，但 Django 1.8 添加了HStoreField 作为内置，以及其他几种 PostgreSQL 支持的字段类型.
从某种意义上说，这种方法很好，它可以让您两全其美:动态字段和关系数据库.但是，hstore 不是理想的性能，尤其是如果您最终将在一个字段中存储数千个项目.它也只支持字符串作为值.
#app/models.py从 django.contrib.postgres.fields 导入 HStoreField类东西(模型.模型):name = models.CharField(max_length=32)数据 = 模型.HStoreField(db_index=True)在 Django 的 shell 中你可以这样使用它:
<预><代码>>>>实例 =Something.objects.create(名称='某物',数据={'a':'1'，'b':'2'})>>>实例.数据['a']'1'>>>空 = Some.objects.create(name='empty')>>>空数据{}>>>empty.data['a'] = '1'>>>空.save()>>>Some.objects.get(name='something').data['a']'1'您可以针对 hstore 字段发出索引查询:
# 等价Some.objects.filter(data={'a': '1', 'b': '2'})# 键/值映射的子集Some.objects.filter(data__a='1')# 键列表的子集Some.objects.filter(data__has_keys=['a', 'b'])# 单键子集Some.objects.filter(data__has_key='a')JSONField:
JSON/JSONB 字段支持任何 JSON 可编码的数据类型，不仅仅是键/值对，而且往往比 Hstore 更快并且(对于 JSONB)更紧凑.几个包实现了 JSON/JSONB 字段，包括 django-pgfields，但从 Django 1.9 开始，JSONField 是一个内置的使用 JSONB 进行存储.JSONField 与 HStoreField 类似，对于大型字典可能会表现得更好.它还支持字符串以外的类型，例如整数、布尔值和嵌套字典.
#app/models.py从 django.contrib.postgres.fields 导入 JSONField类东西(模型.模型):name = models.CharField(max_length=32)数据 = JSONField(db_index=True)在shell中创建:
<预><代码>>>>实例 =Something.objects.create(名称='某物',数据={'a':1，'b':2，'嵌套':{'c':3}})索引查询几乎与 HStoreField 相同，只是可以嵌套.复杂的索引可能需要手动创建(或脚本化迁移).
<预><代码>>>>Some.objects.filter(data__a=1)>>>Some.objects.filter(data__nested__c=3)>>>Some.objects.filter(data__has_key='a')或其他 NoSQL Django 改编版——通过它们，您可以拥有完全动态的模型.
NoSQL Django 库很棒，但请记住，它们不是 100% 与 Django 兼容的，例如，迁移到 Django-nonrel 来自标准 Django，您需要将 ManyToMany 替换为 ListField 等等.
查看这个 Django MongoDB 示例:
from djangotoolbox.fields import DictField类图像(模型.模型):exif = DictField()...>>>图像 = Image.objects.create(exif=get_exif_data(...))>>>图像文件{u'camera_model':'Spamcams 4242'，'exposure_time':0.3，...}您甚至可以创建任何 Django 模型的嵌入式列表:
class Container(models.Model):东西 = ListField(EmbeddedModelField())类 FooModel(models.Model):foo = 模型.IntegerField()类 BarModel(models.Model):bar = 模型.CharField()...>>>Container.objects.create(东西=[FooModel(foo=42), BarModel(bar='spam')])Django-mutant:基于 syncdb 和 South-hooks 的动态模型
Django-mutant 实现了完全动态的外键和 m2m 字段.并受到 Will Hardy 和 Michael Hall 的令人难以置信但有点骇人听闻的解决方案的启发.
所有这些都基于 Django South hooks，根据 Will Hardy 在 DjangoCon 2011 上的演讲 (注意！) 尽管如此健壮并在生产中经过测试(相关源代码).
是的，这很神奇，通过这些方法，您可以使用任何关系数据库后端实现完全动态的 Django 应用程序、模型和字段.但代价是什么?大量使用会影响应用程序的稳定性吗?这些都是需要考虑的问题.您需要确保保持适当的锁允许同时更改数据库的请求.
如果您使用的是 Michael Halls 库，您的代码将如下所示:
来自 dynamo 导入模型test_app, created = models.DynamicApp.objects.get_or_create(名称='发电机')测试，创建 = models.DynamicModel.objects.get_or_create(名称='测试'，verbose_name='测试模型'，app=test_app)foo, created = models.DynamicModelField.objects.get_or_create(名称 = 'foo',verbose_name = 'Foo 字段',模型 = 测试，field_type = 'dynamiccharfield',空 = 真，空白 = 真，唯一 = 错误，help_text = 'Foo 的测试字段',)bar, created = models.DynamicModelField.objects.get_or_create(名称 = '酒吧',verbose_name = '酒吧字段',模型 = 测试，field_type = '动态整数字段',空 = 真，空白 = 真，唯一 = 错误，help_text = 'Bar 的测试字段',)
I'm working on a multi-tenanted application in which some users can define their own data fields (via the admin) to collect additional data in forms and report on the data. The latter bit makes JSONField not a great option, so instead I have the following solution:
class CustomDataField(models.Model):
"""
Abstract specification for arbitrary data fields.
Not used for holding data itself, but metadata about the fields.
"""
site = models.ForeignKey(Site, default=settings.SITE_ID)
name = models.CharField(max_length=64)
class Meta:
abstract = True
class CustomDataValue(models.Model):
"""
Abstract specification for arbitrary data.
"""
value = models.CharField(max_length=1024)
class Meta:
abstract = True
Note how CustomDataField has a ForeignKey to Site - each Site will have a different set of custom data fields, but use the same database. Then the various concrete data fields can be defined as:
class UserCustomDataField(CustomDataField):
pass
class UserCustomDataValue(CustomDataValue):
custom_field = models.ForeignKey(UserCustomDataField)
user = models.ForeignKey(User, related_name='custom_data')
class Meta:
unique_together=(('user','custom_field'),)
This leads to the following use:
custom_field = UserCustomDataField.objects.create(name='zodiac', site=my_site) #probably created in the admin
user = User.objects.create(username='foo')
user_sign = UserCustomDataValue(custom_field=custom_field, user=user, data='Libra')
user.custom_data.add(user_sign) #actually, what does this even do?
But this feels very clunky, particularly with the need to manually create the related data and associate it with the concrete model. Is there a better approach?
Options that have been pre-emptively discarded:
- Custom SQL to modify tables on-the-fly. Partly because this won't scale and partly because it's too much of a hack.
- Schema-less solutions like NoSQL. I have nothing against them, but they're still not a good fit. Ultimately this data is typed, and the possibility exists of using a third-party reporting application.
- JSONField, as listed above, as it's not going to work well with queries.
As of today, there are four available approaches, two of them requiring a certain storage backend:
Django-eav (the original package is no longer mantained but has some thriving forks)
This solution is based on Entity Attribute Value data model, essentially, it uses several tables to store dynamic attributes of objects. Great parts about this solution is that it:
- uses several pure and simple Django models to represent dynamic fields, which makes it simple to understand and database-agnostic;
allows you to effectively attach/detach dynamic attribute storage to Django model with simple commands like:
eav.unregister(Encounter) eav.register(Patient)At the same time being really powerful.
Downsides:
- Not very efficient. This is more of a criticism of the EAV pattern itself, which requires manually merging the data from a column format to a set of key-value pairs in the model.
- Harder to maintain. Maintaining data integrity requires a multi-column unique key constraint, which may be inefficient on some databases.
- You will need to select one of the forks, since the official package is no longer maintained and there is no clear leader.
The usage is pretty straightforward:
import eav from app.models import Patient, Encounter eav.register(Encounter) eav.register(Patient) Attribute.objects.create(name='age', datatype=Attribute.TYPE_INT) Attribute.objects.create(name='height', datatype=Attribute.TYPE_FLOAT) Attribute.objects.create(name='weight', datatype=Attribute.TYPE_FLOAT) Attribute.objects.create(name='city', datatype=Attribute.TYPE_TEXT) Attribute.objects.create(name='country', datatype=Attribute.TYPE_TEXT) self.yes = EnumValue.objects.create(value='yes') self.no = EnumValue.objects.create(value='no') self.unkown = EnumValue.objects.create(value='unkown') ynu = EnumGroup.objects.create(name='Yes / No / Unknown') ynu.enums.add(self.yes) ynu.enums.add(self.no) ynu.enums.add(self.unkown) Attribute.objects.create(name='fever', datatype=Attribute.TYPE_ENUM, enum_group=ynu) # When you register a model within EAV, # you can access all of EAV attributes: Patient.objects.create(name='Bob', eav__age=12, eav__fever=no, eav__city='New York', eav__country='USA') # You can filter queries based on their EAV fields: query1 = Patient.objects.filter(Q(eav__city__contains='Y')) query2 = Q(eav__city__contains='Y') | Q(eav__fever=no)Hstore, JSON or JSONB fields in PostgreSQL
PostgreSQL supports several more complex data types. Most are supported via third-party packages, but in recent years Django has adopted them into django.contrib.postgres.fields.
HStoreField:
Django-hstore was originally a third-party package, but Django 1.8 added HStoreField as a built-in, along with several other PostgreSQL-supported field types.
This approach is good in a sense that it lets you have the best of both worlds: dynamic fields and relational database. However, hstore is not ideal performance-wise, especially if you are going to end up storing thousands of items in one field. It also only supports strings for values.
#app/models.py from django.contrib.postgres.fields import HStoreField class Something(models.Model): name = models.CharField(max_length=32) data = models.HStoreField(db_index=True)In Django's shell you can use it like this:
>>> instance = Something.objects.create( name='something', data={'a': '1', 'b': '2'} ) >>> instance.data['a'] '1' >>> empty = Something.objects.create(name='empty') >>> empty.data {} >>> empty.data['a'] = '1' >>> empty.save() >>> Something.objects.get(name='something').data['a'] '1'You can issue indexed queries against hstore fields:
# equivalence Something.objects.filter(data={'a': '1', 'b': '2'}) # subset by key/value mapping Something.objects.filter(data__a='1') # subset by list of keys Something.objects.filter(data__has_keys=['a', 'b']) # subset by single key Something.objects.filter(data__has_key='a')JSONField:
JSON/JSONB fields support any JSON-encodable data type, not just key/value pairs, but also tend to be faster and (for JSONB) more compact than Hstore. Several packages implement JSON/JSONB fields including django-pgfields, but as of Django 1.9, JSONField is a built-in using JSONB for storage. JSONField is similar to HStoreField, and may perform better with large dictionaries. It also supports types other than strings, such as integers, booleans and nested dictionaries.
#app/models.py from django.contrib.postgres.fields import JSONField class Something(models.Model): name = models.CharField(max_length=32) data = JSONField(db_index=True)Creating in the shell:
>>> instance = Something.objects.create( name='something', data={'a': 1, 'b': 2, 'nested': {'c':3}} )Indexed queries are nearly identical to HStoreField, except nesting is possible. Complex indexes may require manually creation (or a scripted migration).
>>> Something.objects.filter(data__a=1) >>> Something.objects.filter(data__nested__c=3) >>> Something.objects.filter(data__has_key='a')-
Or other NoSQL Django adaptations -- with them you can have fully dynamic models.
NoSQL Django libraries are great, but keep in mind that they are not 100% the Django-compatible, for example, to migrate to Django-nonrel from standard Django you will need to replace ManyToMany with ListField among other things.
Checkout this Django MongoDB example:
from djangotoolbox.fields import DictField class Image(models.Model): exif = DictField() ... >>> image = Image.objects.create(exif=get_exif_data(...)) >>> image.exif {u'camera_model' : 'Spamcams 4242', 'exposure_time' : 0.3, ...}You can even create embedded lists of any Django models:
class Container(models.Model): stuff = ListField(EmbeddedModelField()) class FooModel(models.Model): foo = models.IntegerField() class BarModel(models.Model): bar = models.CharField() ... >>> Container.objects.create( stuff=[FooModel(foo=42), BarModel(bar='spam')] ) Django-mutant: Dynamic models based on syncdb and South-hooks
Django-mutant implements fully dynamic Foreign Key and m2m fields. And is inspired by incredible but somewhat hackish solutions by Will Hardy and Michael Hall.
All of these are based on Django South hooks, which, according to Will Hardy's talk at DjangoCon 2011 (watch it!) are nevertheless robust and tested in production (relevant source code).
First to implement this was Michael Hall.
Yes, this is magic, with these approaches you can achieve fully dynamic Django apps, models and fields with any relational database backend. But at what cost? Will stability of application suffer upon heavy use? These are the questions to be considered. You need to be sure to maintain a proper lock in order to allow simultaneous database altering requests.
If you are using Michael Halls lib, your code will look like this:
from dynamo import models test_app, created = models.DynamicApp.objects.get_or_create( name='dynamo' ) test, created = models.DynamicModel.objects.get_or_create( name='Test', verbose_name='Test Model', app=test_app ) foo, created = models.DynamicModelField.objects.get_or_create( name = 'foo', verbose_name = 'Foo Field', model = test, field_type = 'dynamiccharfield', null = True, blank = True, unique = False, help_text = 'Test field for Foo', ) bar, created = models.DynamicModelField.objects.get_or_create( name = 'bar', verbose_name = 'Bar Field', model = test, field_type = 'dynamicintegerfield', null = True, blank = True, unique = False, help_text = 'Test field for Bar', )
这篇关于Django 动态模型字段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
