如何实现文件系统和数据库之间的协调? [英] How to bring coordination between file system and database?

问题描述

我正在从事一个在线文件管理项目.我们将数据库(sql server)上的引用和文件系统上的文件数据存储起来.

I am working on a online file management project. We are storing references on the database (sql server) and files data on the on file system.

我们在上传文件和删除文件时都面临着文件系统和数据库之间的协调问题.首先，我们在数据库中创建一个引用或在文件系统上存储文件.

We are facing a problem of coordination between file system and database while we are uploading a file and also in case of deleting a file. First we create a reference in the database or store files on file system.

问题是，如果我先在数据库中创建一个引用，然后在文件系统上存储一个文件，但是在文件系统上存储文件时发生任何类型的错误，然后在数据库中创建该文件的引用但文件系统上不存在文件数据.

The problem is that if I create a reference in the database first and then store a file on file system, but while storing files on the file system any type of error occur, then the reference for that file is created in the database but no file data exist on the file system.

请给我一些如何处理这种情况的解决方案.我非常需要它.

Please give me some solution how to deal with such situation. I am badly in need of it.

我们删除文件时也会出现这种情况?

This case happens also while we deleting a file?

推荐答案

对文件系统的访问确实不是事务性的.您需要自己模拟一个全有或全无分布式事务:如果数据库中的提交失败，请删除文件系统上的文件.相反，如果写入文件失败，则回滚数据库事务(这会更复杂一些，但这是一个粗略的草图).

Access to the file system is indeed not transactional. You will need to simulate an all-or-nothing distributed transaction yourself: if the commit in database fails, delete the file on file-system. Inversely, if writing file fails, rollback database transaction (That will be a bit more complicated, but that's a rough sketch).

请注意，更新文件时可能会变得非常复杂.您首先需要复制它，这样如果在覆盖文件后数据库事务失败，您仍然可以恢复文件的旧版本.您是否要这样做取决于所需的稳健性水平.

Note that it can get pretty complicated when a file is updated. You need first to copy it, so that if the database transaction fails after you've overwritten the file you can still restore the old version of the file. Whether you want to do this depends on the level of robustness that is desired.

尝试强制所有操作都通过您的应用程序(创建、写入、删除文件).如果您不能这样做并且您无法阻止直接在文件系统上访问文件(并且可能被删除)，我认为除了定期将数据库与文件系统同步之外别无他法:检查哪个文件被删除并删除数据库中的条目.您可以为此创建一个每 X 分钟运行一次的作业.

Try to enforce that all manipulations go through your application (create, write, delete of files). If you can not do this and you can not prevent a file from being accessed directly on the file system (and maybe deleted), I see no other way than to periodically synchronize the database with the file system: check which file was removed and delete the entry in database. You could create a job that runs each X minute for that.

我还建议在数据库中存储文件的哈希(例如 MD5).花一点时间来计算它，但这对我检测问题非常有用，例如如果文件因错误在文件系统上重命名，但不在数据库中.这也允许定期运行一些完整性检查，以验证没有任何问题.

I would also suggest storing a hash (e.g. MD5) of the file in database. Take a bit of time to compute it, but that has been immensely useful for me to detect problems, e.g. if the file is renamed on file system by error but not in database. That also allows to run some integrity check periodically, to verify nothing was screwed.

如果这种方法还不够(例如，您希望它更健壮)，我认为除了将二进制文件存储在 LOB 中的数据库中之外别无他法.那么它将真正具有交易性和安全性.

If this approach is not sufficient (e.g. you want it to be more robust), I see no other way than to store the binary in the database in LOB. Then it will be really transactional and safe.

这篇关于如何实现文件系统和数据库之间的协调?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！