线程 Django 任务不会自动处理事务或数据库连接? [英] Threaded Django task doesn't automatically handle transactions or db connections?
问题描述
我已经将 Django 设置为在它们自己的线程中运行一些重复性任务，我注意到它们总是留下未完成的数据库连接进程(pgsql事务空闲").
I've got Django set up to run some recurring tasks in their own threads, and I noticed that they were always leaving behind unfinished database connection processes (pgsql "Idle In Transaction").
我查看了 Postgres 日志，发现事务没有完成(没有回滚).我尝试在我的函数上使用各种事务装饰器，但没有成功.
I looked through the Postgres logs and found that the transactions weren't being completed (no ROLLBACK). I tried using the various transaction decorators on my functions, no luck.
我切换到手动事务管理并手动进行回滚，这可行，但仍使进程处于空闲"状态.
I switched to manual transaction management and did the rollback manually, that worked, but still left the processes as "Idle".
然后我调用了 connection.close()，一切都很好.
So then I called connection.close(), and all is well.
但我想知道，为什么 Django 的典型事务和连接管理不适用于这些从 Django 主线程产生的线程任务?
But I'm left wondering, why doesn't Django's typical transaction and connection management work for these threaded tasks that are being spawned from the main Django thread?
推荐答案
经过数周的测试和阅读 Django 源代码后，我找到了自己问题的答案:
After weeks of testing and reading the Django source code, I've found the answer to my own question:
交易
Django 的默认自动提交行为仍然适用于我的线程函数.但是，它在 Django 文档中说明:
Django's default autocommit behavior still holds true for my threaded function. However, it states in the Django docs:
只要您执行需要写入数据库的操作，Django 就会生成 INSERT/UPDATE/DELETE 语句，然后执行 COMMIT.没有隐式回滚.
As soon as you perform an action that needs to write to the database, Django produces the INSERT/UPDATE/DELETE statements and then does the COMMIT. There’s no implicit ROLLBACK.
最后一句话非常直白.它不会发出 ROLLBACK 命令，除非 Django 中的某些内容设置了脏标志.由于我的函数只执行 SELECT 语句，因此它从不设置脏标志，也不会触发 COMMIT.
That last sentence is very literal. It DOES NOT issue a ROLLBACK command unless something in Django has set the dirty flag. Since my function was only doing SELECT statements it never set the dirty flag and didn't trigger a COMMIT.
这与 PostgreSQL 认为事务需要 ROLLBACK 的事实背道而驰，因为 Django 为时区发出了 SET 命令.在查看日志时，我放弃了，因为我一直看到这些 ROLLBACK 语句并假设 Django 的事务管理是来源.事实证明不是这样，没关系.
This goes against the fact that PostgreSQL thinks the transaction requires a ROLLBACK because Django issued a SET command for the timezone. In reviewing the logs, I threw myself off because I kept seeing these ROLLBACK statements and assumed Django's transaction management was the source. Turns out it's not, and that's OK.
连接
连接管理是事情变得棘手的地方.事实证明，Django 使用 signals.request_finished.connect(close_connection) 来关闭它通常使用的数据库连接.由于 Django 中通常不会发生任何不涉及请求的事情，因此您认为这种行为是理所当然的.
The connection management is where things do get tricky. It turns out Django uses signals.request_finished.connect(close_connection) to close the database connection it normally uses. Since nothing normally happens in Django that doesn't involve a request, you take this behavior for granted.
不过，就我而言，没有请求，因为作业已安排好.没有请求意味着没有信号.没有信号意味着数据库连接从未关闭.
In my case, though, there was no request because the job was scheduled. No request means no signal. No signal means the database connection was never closed.
回到事务，事实证明，在事务管理没有任何更改的情况下，简单地调用 connection.close() 会在 PostgreSQL 日志中发出 ROLLBACK 语句，我'一直在寻找.
Going back to transactions, it turns out that simply issuing a call to connection.close() in the absence of any changes to the transaction management issues the ROLLBACK statement in the PostgreSQL log that I'd been looking for.
解决方案
解决方案是让正常的 Django 事务管理照常进行，只需通过以下三种方式之一关闭连接:
The solution is to allow the normal Django transaction management to proceed as normal and to simply close the connection one of three ways:
- 编写一个装饰器来关闭连接并将必要的功能包装在其中.
- 连接现有的请求信号，让 Django 关闭连接.
- 在函数结束时手动关闭连接.
这三个中的任何一个都将(并且确实)起作用.
Any of those three will (and do) work.
这让我疯狂了好几个星期.我希望这能在未来对其他人有所帮助！
This has driven me crazy for weeks. I hope this helps someone else in the future!
这篇关于线程 Django 任务不会自动处理事务或数据库连接?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
