在Django框架中使用动态模型 [英] Using dynamic models in Django framework
问题描述
我目前正在使用Django框架（包括其Models机制）来抽象化数据库模式声明和常规db访问，这在大多数情况下都可以正常工作。
但是，我的应用程序还需要使用表来就运行时而言，动态创建和访问该文件是不可行的，据我所知，Django尚不支持。
这些表通常具有相同的结构，并且基本上可以由同一Model类抽象，但是Django不允许您更改某个模型查询的基础db_table，因为它在Model上声明了类，而不是在Manager上。
I am currently using Django framework including its Models mechanism to abstract the database schema declaration and general db access, which is working fine for most scenarios.
However, my application also requires tables to be created and accessed dynamically during runtime, which as far as I can see, is not supported by Django out of the box.
These tables usually have an identical structure, and can basically be abstracted by the same Model class, but Django doesn't let you change the underlying db_table of a certain model query, as it is declared on the Model class and not on the Manager.
为此，我的解决方案是在需要创建，填充和访问新表时执行此过程：
My solution for this is to do this process whenever I need a new table to be created, populated and accessed:
- 使用原始sql创建并填充表
- 使用原始sql向表添加索引
-
当我需要访问表时（使用django queryset api），我通过使用以下代码动态声明一个新类型，并将其作为查询模型返回：
- Create and populate the table using raw sql
- Add indexes to the table using raw sql
When I need to access the table (using django queryset api), I declare a new type dynamically and return it as the model for the query, by using this code:
table_name = # name of the table created by sql
model_name = '%d_%s' % (connection.tenant.id, table_name)
try:
model = apps.get_registered_model('myapp', model_name)
return model
except LookupError:
pass
logger.debug("no model exists for model %s, creating one" % model_name)
class Meta:
db_table = table_name
managed = False
attrs = {
'field1' : models.CharField(max_length=200),
'field2' : models.CharField(max_length=200),
'field3' : models.CharField(max_length=200)
'__module__': 'myapp.models',
'Meta':Meta
}
model = type(str(model_name), (models.Model,), attrs)
return model
请注意，我确实检查了该模型是否已经在django中注册，并且我正在使用现有模型，以防万一。每个表的模型名称始终是唯一的。由于我使用的是多租户，因此租户名称也是模型名称的一部分，以避免与在不同架构上声明的相似表发生冲突。
Note that I do check if the model is already registered in django and I'm using an existing model in case it does. The model name is always unique for each table. Since I'm using multi tenants, the tenant name is also part of the model name to avoid conflict with similar tables declared on different schemas.
该解决方案到目前为止对我来说还不错。
但是，应用程序将需要支持大量这些表。即10,000-100,000个这样的表（和相应的模型类），每个表最多一百万行。
假设底层数据库在这种负载下很好，我的问题是：
This solution works fine for me so far.
However, the application will need to support a large number of these tables. i.e. 10,000 - 100,000 such tables(and corresponding model classes), with up to a million rows per table.
Assuming the underlying db is fine with this load, my questions are:
-
无论是否考虑预期规模，您是否都认为此解决方案有任何问题？
Do you see any problem with this solution, with and without regards to the expected scale ?
对于这种情况，每个人都有更好的解决方案吗？
Anybody has a better solution for this scenario ?
谢谢。
推荐答案
有一个关于动态创建模型的Wiki页面，尽管自上次更新以来已经有一段时间了：
There is a wiki page on creating models dynamically, although it has been a while since it was last updated:
也有一些针对该用例设计的应用程序，但我认为其中没有任何一个正在积极维护：
There are also a few apps that are designed for this use case, but I don't think any of them is being actively maintained:
我明白了d，如果您已经承诺使用Django，这并不是很有帮助，但这对于Django来说并不是一个很好的用例。与仅使用psycopg2或适用于您的数据的任何其他适配器相比，与Django模型层提供的抽象进行斗争可能会更加昂贵。
I understand that if you are already committed to Django this isn't very helpful, but this a use case for which Django isn't really good. It might be more costly to fight against the abstractions provided by Django's model layer, than to just use psycopg2 or whatever other adapter is appropriate for your data.
取决于哪种排序对于将要对数据执行的操作，使用带有索引字段的单个模型可能更合理，该模型允许您区分该行将在哪个表中，然后按该列对数据进行分片。
Depending on what sort of operations you are going to perform on your data, it may be also more reasonable to use a single model with an indexed field that allows you to distinguish in which table that row would be and then sharding the data by that column.
如果您仍然需要这样做，通常的想法是：
If you still need to do this, the general idea would be:
-
创建一个扩展Django ModelBase的元类。该元类将用作实际模型的工厂。
Create a metaclass that extends Django's ModelBase. This metaclass you would use as a factory for your actual models.
考虑该Wiki页面上提到的内容，例如绕过app_label问题。
Consider stuff mentioned on that wiki page, like circumventing the app_label issue.
生成并执行sql，以创建模型，如Wiki页面上所示。
Generate and execute the sql for the creation of the model as also shown on the wiki page.
这篇关于在Django框架中使用动态模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
