Django 模型:动态更改 db_table [英] Django model: change db_table dynamically

问题描述

我有大量的数据集.每组数据由几个数据库表组成.数据库表集的模式是相同的.每组表可以有超过一百万行.每组数据属于一个作业，作业之间没有关系.一个或多个作业属于不同的用户.表集被导入并最终作为一组表被删除.从性能的角度来看，最好将它们保存为单独的表集.

I have a large number of sets of data. Each set of data comprises of several database tables. The schema for the sets of database tables is identical. Each set of tables can have over a million rows. Each set of data belongs to one job, there are no relations between jobs. One or more jobs belong to a different user. Sets of tables get imported and eventually deleted as a set of tables. From a performance point of view it is better to keep them as separate sets of tables.

所以我想为几个表中的每一个创建几个通用的 Django 模型.我已经在我的 views.py 文件中使用类似这样的代码实现了它:

So I would like to have several generic Django models one for each of the several tables. I have achieved it in my views.py file by using code similar to this:

from foobar.models import Foo, Bar

def my_view(request):
  prefix = request.GET.get('prefix')
  Foo._meta.db_table = prefix + '_foo'
  Bar._meta.db_table = prefix + '_bar'
  ....

  foobar_list = Foo.objects.filter(bar_id=myval)
  ...

我的问题是:对于基于 Django 的 Web 应用程序的并发多个用户使用此代码是否安全?模型对象是否在用户之间共享?如果同时有两个请求会发生什么?

My questions are: Is it safe to use this code with concurrent multiple users of a Django based web application? Are the models objects shared across users? What would happen if there were two requests simultaneously?

编辑二:我已经考虑了 Lie Ryan 的回答和评论，并提出了这个代码:

EDIT NO 2: I have considered Lie Ryan's answer and the comments and come up with this code:

from django.http import HttpResponse, HttpResponseNotFound
from django.db import models
from django.template import RequestContext, loader

def getModels(prefix):
    table_map = {}

    table_map["foo"] = type(str(prefix + '_foo'), (models.Model,), {
        '__module__': 'foobar.models',
        'id' : models.IntegerField(primary_key=True),
        'foo' : models.TextField(blank=True),
        })
    table_map["foo"]._meta.db_table = prefix + '_foo'

    table_map["bar"] = type(str(prefix + '_bar'), (models.Model,), {
        '__module__': 'foobar.models',
        'id' : models.IntegerField(primary_key=True), 
        'foo' : models.ForeignKey(prefix + '_foo', null=True, blank=True),
        })
    table_map["bar"]._meta.db_table = prefix + '_bar'

    return table_map

def foobar_view(request):  
    prefix = request.GET.get('prefix')
    if prefix != None and prefix.isdigit():
        table_map = getModels(prefix)
        foobar_list = table_map["bar"].objects.filter.order_by('foo__foo')
        template = loader.get_template('foobar/foobar.html')
        context = RequestContext(request, {
            'foobar_list': foobar_list,
        })
        return HttpResponse(template.render(context))
    else:
        return HttpResponseNotFound('<h1>Page not found</h1>')

现在我的问题是，编辑后的代码的第二稿对于并发多个用户是否安全?

Now my question is, is this second draft of the edited code safe with concurrent multiple users?

推荐答案

这种技术称为分片.不，如果您使用线程处理并发请求，那么这样做是不安全的.

This technique is called sharding. No, it is not safe to do this if you serve concurrent requests with threads.

你可以做的是动态构造多个指向不同db_tables的类，并使用一个工厂来选择正确的类.

What you can do is to dynamically construct multiple classes pointing to different db_tables, and use a factory to select the right class.

tables = ["foo", "bar"]
table_map = {}
for tbl in tables:
    class T(models.Model):
        db_table = tbl
        ... table definition ...
    table_map[tbl] = T

然后创建一个函数，根据您的数据分片方式选择正确的 table_map.

And then create a function that selects the right table_map based on how you shard your data.

如果你接受用户输入的表名，也要小心注入.

Also be careful of injection if you accept table name from user input.

或者，某些数据库系统(例如 PostgrSQL)允许每个数据库使用多个模式，这可能是在某些情况下分离数据的更好方法.

Alternatively, some database systems like PostgrSQL allows multiple schemas per database, which might be a better way to separate your data in certain circumstances.

这篇关于Django 模型:动态更改 db_table的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！