跨多个线程使用不同文件的SQLite死锁 [英] Sqlite deadlock across multiple threads with different files

问题描述

我有一个OS X应用程序(Yosemite，10.10)，该应用程序执行长期运行的工作，涉及跨多个线程大量使用sqlite.而且我遇到了8个线程的僵局，所有这些线程都陷入了连接到不同数据库文件的sqlite代码中.它们之间没有明显的与资源相关的连接.我正在新Mac Pro(2013年末)上对其进行调试.

I've got an OS X app (Yosemite, 10.10) that performs long-running jobs involving heavy use of sqlite across multiple threads. And I've hit a deadlock across 8 threads, all of which are stuck in sqlite code connecting to different database files. There is no apparent resource-related connection between them. I'm debugging it on a new Mac Pro (late 2013).

其中有四个在此堆栈中.其中三个在同一表上操作(同样，不同的数据库文件)；三是更新，一是查询.

Four of them are at this stack. Of those, three are operating on the same table (again, different database files); three are updating and one is querying.

__psynch_mutexwait
_pthread_mutex_lock
unixLock
sqlite3PagerSharedLock
sqlite3BtreeBeginTrans
sqlite3VdbeExec
sqlite3_step

该堆栈中有一个表，与上一个堆栈中的三个表更新了同一张表.

One is at this stack, updating the same table as three of the ones at the stack above.

guarded_close_np
nolockClose
pager_end_transaction
sqlite3BtreeCommitPhaseTwo
sqlite3VdbeHalt
sqlite3VdbeExec
sqlite3_step

该堆栈中有两个，在不同位置打开具有相同名称的数据库文件.打开模式为SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_FULLMUTEX.

Two are at this stack, opening database files with the same name in different locations. The open mode is SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_FULLMUTEX.

__psynch_mutexwait
_pthread_mutex_lock
sqlite3ParseUri
openDatabase

一个在这个堆栈中，结束了交易.

One is at this stack, ending a transaction.

__psynch_mutexwait
_pthread_mutex_lock
unixLock
sqlite3VdbeHalt
sqlite3VdbeExec
sqlite3_step

所以，问题是:什么会导致sqlite在不涉及任何共享资源的情况下死锁?

更新:我现在锁定了七个线程，分别调用sqlite3_open_v2和一个在sqlite3_close上，所有线程都在不同的数据库文件上(几个具有相同的名称，但位于不同的文件夹中).堆栈是:

Update: I've now got seven threads locked calling sqlite3_open_v2 and one on sqlite3_close, all on different database files (several with the same name, but in different folders). The stack is:

__psynch_mutexwait
_pthread_mutex_lock
sqlite3ParseUri
openDatabase

紧密的堆栈是:

guarded_close_np
unixClose
sqlite3PagerClose
sqlite3BtreeClose
sqlite3LeaveMutexAndCloseZombie
sqlite3Close

通过修复一些内存泄漏(这在ARC中不运行)并删除了一些事务语句，我可以使它在锁定之前持续更长的时间.

I was able to get it to go longer before locking up by fixing some memory leaks (this isn't running with ARC) and removing some transaction statements.

更新2:我已经通过sqlite3_config连接了SQLITE_LOG(

Update 2: I've hooked up SQLITE_LOG via sqlite3_config (documentation) and I'm seeing a fair amount of logging of code 28 (sqlite_warning) with the message "file renamed while open:".

更新3:我擦掉了机器并重新安装了Yosemite，以排除文件系统问题.我仍然以同样的方式锁定.它会运行几分钟，然后线程一一锁定.在guarded_close_np处有一个，被卡在指令jae <address here>中的汇编中，其中要跳转的地址具有指令retq.我问了一个单独的问题关于文件的sqlite日志消息被重新命名，希望与之相关.

Update 3: I wiped the machine and did a fresh install of Yosemite in an attempt to rule out file system issues. I'm still locking up the same way. It'll run for several minutes and then one by one the threads lock up. There's one at guarded_close_np, stuck in assembly at an instruction jae <address here> where the address being jumped to has the instruction retq. I've asked a separate question about the sqlite log message about files being renamed in hopes that it's related.

推荐答案

我的应用通过链接到动态库来使用sqlite.我从此处(撰写本文时为3.8.7.4)下载了最新的SQLite源合并.进入应用程序，一切开始正常工作.因此，也许这是3.8.5中的错误.显然，直接将源代码编译到应用程序中的是

My app was using sqlite by linking against the dynamic library. I downloaded the latest SQLite source amalgamation from here (3.8.7.4 as of writing) and compiled it right into the app, and everything started working. So perhaps it was a bug in 3.8.5. Apparently compiling the source into the app directly is the recommended way to use sqlite anyway.

我仍然不知道是什么原因引起的.我唯一能想到的是，这与我创建数据库文件的方式有关:我正在使用NSFileManager createFileAtPath创建一个空文件，然后将其与SQLITE_OPEN_CREATE一起传递给sqlite3_open_v2.标志参数.因此，它将数据库写入现有文件中，而不是在指定位置创建数据库文件.

I still don't know exactly what caused the issue. The only thing I can think of is that it's something to do with how I'm creating the database files: I'm creating an empty file using NSFileManager createFileAtPath and then passing it to sqlite3_open_v2 with SQLITE_OPEN_CREATE as part of the flags parameter. So it's writing a database into an existing file instead of creating a database file at the specified location.

这篇关于跨多个线程使用不同文件的SQLite死锁的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！