如何强制 Django 忽略任何缓存并重新加载数据? [英] How do I force Django to ignore any caches and reload data?

问题描述

我正在使用来自不是从 HTTP 请求调用的进程的 Django 数据库模型.该过程应该每隔几秒钟轮询一次新数据并对其进行一些处理.我有一个循环，它会休眠几秒钟，然后从数据库中获取所有未处理的数据.

我看到的是，在第一次获取之后，该进程再也看不到任何新数据.我运行了一些测试，看起来 Django 正在缓存结果，即使我每次都在构建新的 QuerySets.为了验证这一点，我从 Python shell 中做到了这一点:

<预><代码>>>>MyModel.objects.count()885#(这里我添加了来自另一个进程的更多数据.)>>>MyModel.objects.count()885>>>MyModel.objects.update()0>>>MyModel.objects.count()1025

如您所见，添加新数据不会改变结果计数.但是，调用管理器的 update() 方法似乎可以解决问题.

我找不到有关该 update() 方法的任何文档，也不知道它可能会做什么其他坏事.

我的问题是，为什么我会看到这种缓存行为，这与 Django 文档 说?我该如何防止它发生?

解决方案

遇到这个问题并找到了两个明确的解决方案后，我认为值得发布另一个答案.

这是MySQL默认事务模式的问题.Django 在开始时打开一个事务，这意味着默认情况下您不会看到数据库中所做的更改.

这样演示

在终端 1 中运行 django shell

<预><代码>>>>MyModel.objects.get(id=1).my_field你老了

另一个在2号航站楼

<预><代码>>>>MyModel.objects.get(id=1).my_field你老了>>>a = MyModel.objects.get(id=1)>>>a.my_field = "新">>>a.save()>>>MyModel.objects.get(id=1).my_field你是新人>>>

回到终端 1 来演示问题 - 我们仍然从数据库中读取旧值.

<预><代码>>>>MyModel.objects.get(id=1).my_field你老了

现在在终端 1 中演示解决方案

<预><代码>>>>从 django.db 导入事务>>>>>>@transaction.commit_manually... def flush_transaction():...事务.commit()...>>>MyModel.objects.get(id=1).my_field你老了>>>冲洗交易()>>>MyModel.objects.get(id=1).my_field你是新人>>>

现在读取新数据

这是带有文档字符串的易于粘贴的代码块

from django.db 导入事务@transaction.commit_manuallydef flush_transaction():"""刷新当前事务，这样我们就不会读取过时的数据在长时间运行的进程中使用以确保从中读取新数据数据库.这是 MySQL 和默认的问题交易模式.您可以通过设置修复它"transaction-isolation = READ-COMMITTED" 在 my.cnf 或通过调用在适当的时候这个功能"""交易.commit()

另一种解决方案是更改mysql的my.cnf以更改默认事务模式

transaction-isolation = READ-COMMITTED

请注意，这是 Mysql 的一个相对较新的功能，并且具有 二进制日志/从属的一些后果.如果需要，您也可以将其放在 django 连接序言中.

3 年后更新

现在 Django 1.6 有 在 MySQL 中启用自动提交 这不再是问题.无论您的 MySQL 是在 REPEATABLE-READ(默认)还是 READ-COMMITTED 事务中，上面的示例现在都可以在没有 flush_transaction() 代码的情况下正常工作隔离模式.

在非自动提交模式下运行的以前版本的 Django 中发生的事情是第一个 select 语句打开了一个事务.由于 MySQL 的默认模式是 REPEATABLE-READ，这意味着后续的 select 语句不会读取对数据库的更新 - 因此需要 flush_transaction() 上面的代码停止事务并开始新的事务.

尽管如此，您仍然有可能想要使用 READ-COMMITTED 事务隔离的原因.如果您要将终端 1 放入事务中，并且想查看终端 2 的写入，则需要 READ-COMMITTED.

flush_transaction() 代码现在会在 Django 1.6 中产生弃用警告，因此我建议您将其删除.

I'm using the Django database models from a process that's not called from an HTTP request. The process is supposed to poll for new data every few seconds and do some processing on it. I have a loop that sleeps for a few seconds and then gets all unhandled data from the database.

What I'm seeing is that after the first fetch, the process never sees any new data. I ran a few tests and it looks like Django is caching results, even though I'm building new QuerySets every time. To verify this, I did this from a Python shell:

>>> MyModel.objects.count()
885
# (Here I added some more data from another process.)
>>> MyModel.objects.count()
885
>>> MyModel.objects.update()
0
>>> MyModel.objects.count()
1025

As you can see, adding new data doesn't change the result count. However, calling the manager's update() method seems to fix the problem.

I can't find any documentation on that update() method and have no idea what other bad things it might do.

My question is, why am I seeing this caching behavior, which contradicts what Django docs say? And how do I prevent it from happening?

解决方案

Having had this problem and found two definitive solutions for it I thought it worth posting another answer.

This is a problem with MySQL's default transaction mode. Django opens a transaction at the start, which means that by default you won't see changes made in the database.

Demonstrate like this

Run a django shell in terminal 1

>>> MyModel.objects.get(id=1).my_field
u'old'

And another in terminal 2

>>> MyModel.objects.get(id=1).my_field
u'old'
>>> a = MyModel.objects.get(id=1)
>>> a.my_field = "NEW"
>>> a.save()
>>> MyModel.objects.get(id=1).my_field
u'NEW'
>>> 

Back to terminal 1 to demonstrate the problem - we still read the old value from the database.

>>> MyModel.objects.get(id=1).my_field
u'old'

Now in terminal 1 demonstrate the solution

>>> from django.db import transaction
>>> 
>>> @transaction.commit_manually
... def flush_transaction():
...     transaction.commit()
... 
>>> MyModel.objects.get(id=1).my_field
u'old'
>>> flush_transaction()
>>> MyModel.objects.get(id=1).my_field
u'NEW'
>>> 

The new data is now read

Here is that code in an easy to paste block with docstring

from django.db import transaction

@transaction.commit_manually
def flush_transaction():
    """
    Flush the current transaction so we don't read stale data

    Use in long running processes to make sure fresh data is read from
    the database.  This is a problem with MySQL and the default
    transaction mode.  You can fix it by setting
    "transaction-isolation = READ-COMMITTED" in my.cnf or by calling
    this function at the appropriate moment
    """
    transaction.commit()

The alternative solution is to change my.cnf for MySQL to change the default transaction mode

transaction-isolation = READ-COMMITTED

Note that that is a relatively new feature for Mysql and has some consequences for binary logging / slaving. You could also put this in the django connection preamble if you wanted.

Update 3 years later

Now that Django 1.6 has turned on autocommit in MySQL this is no longer a problem. The example above now works fine without the flush_transaction() code whether your MySQL is in REPEATABLE-READ (the default) or READ-COMMITTED transaction isolation mode.

What was happening in previous versions of Django which ran in non autocommit mode was that the first select statement opened a transaction. Since MySQL's default mode is REPEATABLE-READ this means that no updates to the database will be read by subsequent select statements - hence the need for the flush_transaction() code above which stops the transaction and starts a new one.

There are still reasons why you might want to use READ-COMMITTED transaction isolation though. If you were to put terminal 1 in a transaction and you wanted to see the writes from the terminal 2 you would need READ-COMMITTED.

The flush_transaction() code now produces a deprecation warning in Django 1.6 so I recommend you remove it.

这篇关于如何强制 Django 忽略任何缓存并重新加载数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！