线程Django任务不会自动处理事务或数据库连接？ [英] Threaded Django task doesn't automatically handle transactions or db connections?

问题描述

我已经设置Django在自己的线程中运行一些重复的任务，我注意到他们总是留下未完成的数据库连接进程（pgsqlIdle In Transaction）。

我浏览了Postgres日志，发现事务没有完成（没有ROLLBACK）。我尝试在我的函数上使用各种事务装饰器，没有运气。

我切换到手动事务管理，并手动进行了回滚， Idle。

所以我调用connection.close（），一切都很好。

我仍然想知道，为什么Django的典型的事务和连接管理工作的这些线程任务是从主Django线程派生的？

解决方案

经过几个星期的测试和阅读Django源代码后，我发现了自己的问题的答案：

事务

Django的默认自动提交行为仍然适用于我的线程函数。但是，它在Django文档中声明：

一旦执行了需要写入数据库的操作，Django就会生成INSERT / UPDATE / DELETE语句，然后执行COMMIT。没有隐式的ROLLBACK。

最后一句话很文字。它不发出ROLLBACK命令，除非在Django中有东西设置了脏标志。因为我的函数只是做SELECT语句，所以从来没有设置脏标志并且没有触发COMMIT。

这违背了PostgreSQL认为事务需要ROLLBACK因为Django为时区发出了SET命令。在审查日志，我抛开了，因为我一直看到这些ROLLBACK语句，并假设Django的事务管理是源。

连接管理是事情变得棘手的地方。结果Django使用 signals.request_finished.connect（close_connection）关闭它通常使用的数据库连接。因为Django中通常不会发生请求，所以你认为这个行为是正确的。

在我的例子中，没有请求，预定。没有请求意味着没有信号。没有信号意味着数据库连接从未关闭。

回到事务，结果是发出对 connection.close 在没有任何更改事务管理问题的PostgreSQL日志中的ROLLBACK语句，我一直在寻找。

解决方案

解决方案是允许正常的Django事务管理正常进行，并以三种方式简单地关闭连接：

1. 编写一个装饰器，关闭连接并将必要的函数包装起来。


2. 挂钩到现有的请求信号

<

这使我疯狂了几个星期。我希望这将有助于未来的人！

I've got Django set up to run some recurring tasks in their own threads, and I noticed that they were always leaving behind unfinished database connection processes (pgsql "Idle In Transaction").

I looked through the Postgres logs and found that the transactions weren't being completed (no ROLLBACK). I tried using the various transaction decorators on my functions, no luck.

I switched to manual transaction management and did the rollback manually, that worked, but still left the processes as "Idle".

So then I called connection.close(), and all is well.

But I'm left wondering, why doesn't Django's typical transaction and connection management work for these threaded tasks that are being spawned from the main Django thread?

解决方案

After weeks of testing and reading the Django source code, I've found the answer to my own question:

Transactions

Django's default autocommit behavior still holds true for my threaded function. However, it states in the Django docs:

As soon as you perform an action that needs to write to the database, Django produces the INSERT/UPDATE/DELETE statements and then does the COMMIT. There’s no implicit ROLLBACK.

That last sentence is very literal. It DOES NOT issue a ROLLBACK command unless something in Django has set the dirty flag. Since my function was only doing SELECT statements it never set the dirty flag and didn't trigger a COMMIT.

This goes against the fact that PostgreSQL thinks the transaction requires a ROLLBACK because Django issued a SET command for the timezone. In reviewing the logs, I threw myself off because I kept seeing these ROLLBACK statements and assumed Django's transaction management was the source. Turns out it's not, and that's OK.

Connections

The connection management is where things do get tricky. It turns out Django uses signals.request_finished.connect(close_connection) to close the database connection it normally uses. Since nothing normally happens in Django that doesn't involve a request, you take this behavior for granted.

In my case, though, there was no request because the job was scheduled. No request means no signal. No signal means the database connection was never closed.

Going back to transactions, it turns out that simply issuing a call to connection.close() in the absence of any changes to the transaction management issues the ROLLBACK statement in the PostgreSQL log that I'd been looking for.

Solution

The solution is to allow the normal Django transaction management to proceed as normal and to simply close the connection one of three ways:

1. Write a decorator that closes the connection and wrap the necessary functions in it.
2. Hook into the existing request signals to have Django close the connection.
3. Close the connection manually at the end of the function.

Any of those three will (and do) work.

This has driven me crazy for weeks. I hope this helps someone else in the future!

这篇关于线程Django任务不会自动处理事务或数据库连接？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！