文件内容损坏的原因 [英] Cause of corrupted file contents

问题描述

我在野外经常遇到应用程序问题.

I'm having a recurring problem with an app in the wild.

它有一个相当简单的XML文件，它时不时地转储，大约每30分钟一次.

It has a fairly simple XML file it dumps out every now and then, something like every 30 minutes.

数据文件通常很小-例如< 5KB.

The data files are often quite small - e.g. < 5KB.

它没有对文件进行锁定-每次都只是从头开始重新创建它.

It doesn't hold a lock on the file - it just recreates it from scratch each time.

我很幸运地看到问题在测试计算机上发生，并且我观察到该文件已损坏并设置为空"(即十六进制为00).真正奇怪的是，与应有的长度相比，它是正确的长度.

I was lucky enough to see the problem occur on a test machine, and what I observed was that the file was corrupted and set to "nulls" (i.e. 00 in Hex). What's really weird is that it is exactly the correct length compared to what it should have been.

在保存过程中，我尝试非常小心:

I've tried to be really careful in my saving process:

1. 我将xml写入与要真正保存的目录相同的临时文件中
2. 我执行了一个具有MOVEFILE_WRITE_THROUGH设置的Win32 MoveFile()(因此它应该应该阻塞，直到移动真正完成为止)，然后移动文件以替换现有的数据文件

1. I write the xml to a temp file in the same directory as I'm going to really save it
2. I perform a Win32 MoveFile() with the MOVEFILE_WRITE_THROUGH set (so it should block until the move is really and truly complete), to move the file to replace the existing data file

我什至锁定互斥锁以确保这不是线程问题.

I even lock on a Mutex to make sure this isn't a threading issue.

这种情况很少发生，例如大约每千名用户中就有一个.

It doesn't happen that often, like maybe 1 in 1000 users.

现在，我过去曾经观察到数据文件在写入过程中由于电源故障或BSOD损坏，而且我看到诸如文件的32kb全部为NULL之类的东西.

Now I have in the past observed data files being corrupted by a power failure or BSOD during writing, and I've seen things like the 32kb of a file being all NULL.

但是，考虑到写入期间可能发生电源故障，尤其是因为我使用的是MOVEFILE_WRITE_THROUGH，这似乎比我预期的要多.

But it just seems like it's happening more than I'd expect, given the chances of a power failure during the write, and espcecially since I'm using MOVEFILE_WRITE_THROUGH.

有什么想法吗?

约翰

回答一些问题:

• 问:为什么不直接写入文件 答:我避免了这种情况，以使软件不易受到电源故障问题的影响.例如.您在写入文件和崩溃/powerfail/BSOD的过程中途中途，那么您肯定有损坏的文件.进行临时文件写入然后移动是确保您执行原子文件操作的一种常用且简单的方法(并且，在不使用NTFS特定API的情况下，尽可能合理地关闭).我应该说该软件是一个归档/备份系统，所以与其他应用程序相比，我必须更加注意数据的一致性.

• Q: Why not write to the file directly A: I avoided this to make the software less vulnerable to power failure issues. E.g. you're halfway through writing the file and crash/powerfail/BSOD then you definitely have a corrupted file. Doing a temp file write and then a move is a commonly used and simple way of ensuring that you do an atomic file operation as possible (well, as close as is reasonable without using NTFS specific APIs). I should say that the software is an archiving/backup system, so I have to take more care with data consistency than other apps might.

问:这是在正常操作期间发生的吗?

Q: Does this happen during normal operation?

A:由于这个问题很普遍，因此我仅提供一些线索，因此我不确定.我可以说该软件的正常运行时间为99.9％.我想这就是我的问题:这仅仅是由BSOD/电源故障引起的随机倒霉还是一个错误?

A: As this issue occurs in the wild, I'm only working with a few clues, so I don't know for sure. I can say that the software works reliabely 99.9% of the time. I guess that's the nub of my question: is this just random unluckyness caused by BSOD/power failure or is it a bug?

问:什么环境/操作系统:

Q: What environment/OS:

A:XP，Vista，7，Server 200X.最可能是NTFS，但可能是FAT32

A: XP, Vista, 7, Server 200X. Most likely NTFS, but could be FAT32

问:我要在移动前关闭文件吗

Q: Am I closing the file before moving

A:是的.我正在使用C ++流并在执行MoveFile之前调用close()

A: Yes. I'm using C++ streams and calling close() before I do the MoveFile

问:其他哪些进程正在访问文件?

Q: What other processses are accessing the file?

推荐答案

根据我的经验，这可能是由于Windows中的文件缓存引起的.您应该尝试使用CreateFile()和FILE_FLAG_WRITE_THROUGH传入来保存文件.通过这种方式保存文件可以确保文件将进入硬盘.

As my experience, it is possibly cause by file cache in windows. You should try to save file using CreateFile() with FILE_FLAG_WRITE_THROUGH pass in. Saving file by this way can make sure the file will land in hard disk.

我编写了一个小程序来对此进行测试.如果程序使用std::ofstream创建文件并使用MoveFileEx()和MOVEFILE_WRITE_THROUGH来移动该文件，则几乎在每次文件移动完成后立即关闭(不关闭)VM电源时，文件几乎都会损坏；否则，如果程序将CreateFile()与FILE_FLAG_WRITE_THROUGH结合使用来创建文件并再次执行相同的操作，则该文件没有损坏(我测试了大约10次，但没有发生).

I had wroten a little program to test this. If the program create file with std::ofstream and use MoveFileEx() with MOVEFILE_WRITE_THROUGH to move that file, the file corrupt almost every time if power off (not shutdown) the VM immediately after file move finished; Otherwise, if the program use CreateFile() with FILE_FLAG_WRITE_THROUGH to create file and do the same thing again, The file didn't corrupt (I tested for about 10 times but it didn't happened).

经过这些简单的测试，我认为您应该尝试将CreateFile()与FILE_FLAG_WRITE_THROUGH一起使用来解决您的问题.

After those simple tests, I think you should try to use CreateFile() with FILE_FLAG_WRITE_THROUGH to solve your problem.

更多信息:
文件缓存(Windows)
Windows Internals，第6版，第11章缓存管理器

More information:
File Caching (Windows)
Windows Internals, 6th edition, Chapter 11 Cache Manager

这篇关于文件内容损坏的原因的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！