在 Windows 上使用多线程文件 IO 的 SHARING_VIOLATION [英] SHARING_VIOLATION with multi-threaded file IO on Windows

问题描述

我有一些类似于这个最小复制示例的代码(真实版本生成一些代码并编译它):

I have some code that resembles this minimal reproduction example (the real version generates some code and compiles it):

#include <fstream>
#include <string>
#include <thread>
#include <vector>

void write(unsigned int thread)
{
    std::ofstream stream("test_" + std::to_string(thread) + ".txt");
    stream << "test" << std::endl;
    stream << "thread" << std::endl;
    stream << "bad" << std::endl;
}

void test(unsigned int thread)
{
    write(thread);
#ifdef _WIN32
    const std::string command = "rename test_" + std::to_string(thread) + ".txt test_renamed_" + std::to_string(thread) + ".txt";
#else
    const std::string command = "mv test_" + std::to_string(thread) + ".txt test_renamed_" + std::to_string(thread) + ".txt";
#endif
    system(command.c_str());
}

int main()
{
    std::vector<std::thread> threads;
    for(unsigned int i = 0; i < 5; i++) {
        // Remove renamed file
        std::remove(("test_renamed_" + std::to_string(i) + ".txt").c_str());

        threads.emplace_back(test, i);
    }

    // Join all threads
    for(auto &t : threads) {
        t.join();
    }
    return EXIT_SUCCESS;
}

我的理解是 std::ofstream 应该以良好的 RAII 方式运行，并在写入函数结束时关闭和刷新.在 Linux 上，它似乎就是这样做的.但是，在 Windows 10 上，我偶尔会收到该进程无法访问该文件，因为它正被另一个进程使用"的消息.错误.我已经用 procmon 研究了它，看起来文件没有被父进程 (22224) 关闭，导致 SHARING_VIOLATION 这可能会导致错误:尽管 procmon 跟踪看起来问题出在我的进程中，但我已尝试关闭病毒扫描程序.我还尝试使用 C 风格的 fopen、fprintf、fclose 并确保我使用 system 生成的进程不会通过清除底层文件句柄上的 HANDLE_FLAG_INHERIT 以某种方式继承文件句柄......这让我有点想不通了！有什么想法吗?

My understanding is that std::ofstream should behave in a nice RAII manner and close and flush at the end of the write function. On Linux, it appears to do just this. However, on Windows 10 I get sporadic "The process cannot access the file because it is being used by another process" errors. I've dug into it with procmon and it looks like the file isn't getting closed by the parent process (22224) resulting in the SHARING_VIOLATION which presumably causes the error: Although the procmon trace looks like the problem is within my process, I have tried turning off the virus scanner. I have also tried using C-style fopen,fprintf,fclose and also ensuring that the process I'm spawning with system isn't inheriting file handles somehow by clearing HANDLE_FLAG_INHERIT on the underlying file handle...which leaves me somewhat out of ideas! Any thoughts SO?

推荐答案

我们可以使用Win32 API重写文件写入:

We can rewrite the file writing using Win32 API:

void writeRaw(unsigned int thread)
{
    const auto str = "test_" + std::to_string(thread) + ".txt";
    auto hFile = CreateFileA(str.c_str(), GENERIC_WRITE, 
        FILE_SHARE_WRITE, nullptr, CREATE_ALWAYS, 0, nullptr);
    DWORD ret{};
    WriteFile(hFile, str.data(), str.size(), &ret, nullptr);
    CloseHandle(hFile);
}

由于 Windows 的工作方式，运行测试仍然会导致文件共享冲突.当最后一个句柄关闭时，文件系统驱动程序执行 IRP_MJ_CLEANUP IOCTL 以完成与文件相关的任何处理.例如，防病毒软件会尝试扫描文件(并顺便锁定它=)).另外文档 MSDN IRP_MJ_CLEANUP 指出:

Running the test still gives a file share violation due to the way windows works. When the last handle is closed, filesystem driver performs IRP_MJ_CLEANUP IOCTL to finish processing anything related to the file. Antivirus software, for instance, would attempt to scan the file (and incidentally holds the lock on it =) ). Additionally documentation MSDN IRP_MJ_CLEANUP states that:

需要注意的是，当一个文件对象的所有句柄都已关闭时，这并不一定意味着不再使用该文件对象.系统组件，例如缓存管理器和内存管理器，可能持有对文件对象的未完成引用.即使在收到 IRP_MJ_CLEANUP 请求之后，这些组件仍然可以读取或写入文件.

It is important to note that when all handles to a file object have been closed, this does not necessarily mean that the file object is no longer being used. System components, such as the Cache Manager and the Memory Manager, might hold outstanding references to the file object. These components can still read to or write from a file, even after an IRP_MJ_CLEANUP request is received.

结论:由于底层系统组件仍在处理文件关闭请求，因此如果进程在关闭句柄后不久尝试对文件执行某些操作，则预计会在 Windows 中收到文件共享冲突.

Conclusion: It is expected that to receive a file share violation in windows if a process tries to do something with the file shortly after closing the handle as the underlying system components are still processing the file close request.

这篇关于在 Windows 上使用多线程文件 IO 的 SHARING_VIOLATION的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！