在存储器和存储受限系统上加密和/或解密大文件（AES），具有“灾难恢复” [英] encrypting and/or decrypting large files (AES) on a memory and storage constrained system, with "catastrophe recovery"

问题描述

我有一个相当普通的问题，所以请原谅，如果它有点模糊。

所以，假设一个1GB的文件，需要加密

问题是系统具有少于512 MB的可用内存和大约1.5 GB的存储空间（给或取），因此，与文件onboard我们有约〜500 MB的硬盘临时空间和小于512 mb RAM玩。

系统不是不可能在加密或解密期间的任何时刻经历非预定的掉电，并且需要能够在被再次加电之后成功地恢复加密/解密过程（并且这似乎是一个令人不快的危险解决）。

问题是：

1）完全是可行的）？

2）最好的策略是什么。

加密/解密，只需要这么少的临时空间（不能有整个文件躺在解密/加密，需要截断它在飞行中不知何故...）

和

b）实施灾难恢复，在这种受限的环境中工作？

PS：
使用的密码必须是AES。

具体来说，但它似乎并没有那么好的灾难恢复shenanigan在一个环境，你不能保存整个解密文件，直到结束...

解决方案

它

1. 保存AES状态和文件位置P档案STAGE1和STAGE2


2. 读取一个区块（例如10 MB）的加密/解密数据


3. 将解密/加密的区块写入外部暂存区


4. 记录SCRATCH已完成的事实


5. 在同一位置对原始档案写入SCRATCH


6. 记录SCRATCH已成功复制的事实


7. 转到1

碰撞后阶段1，和STAGE1和STAGE2不同意，你只是重新启动，并采取与最早的阶段P是好的。
如果在阶段2期间或之后发生严重崩溃，您将丢失10兆字节的工作量：但AES和P是好的，所以您只需重复阶段2.
如果您在阶段3崩溃，然后在恢复你将找不到阶段4的标记，因此将知道SCRATCH是不可靠的，必须重新生成。有STAGE1 / STAGE2，你可以这样做。
如果你在阶段4崩溃，你会相信，SCRATCH必须被重新生成，即使你可以避免这一点 - 但是除了一点时间之外，你不会失去任何东西。
同样的道理，如果你在5或6之前崩溃提交到磁盘，你只需重复第5步和第6步。你知道你不必重新生成SCRATCH，因为第4阶段提交到磁盘。

这一切都假设10 MB是一个缓存的（OS +硬盘，如果回写的话） ）的数据。如果不是，请提高到32或64 MB。

如果这些函数可用，在每个写入阶段完成后，flush（）和sync（）可能会有帮助。 p>

总写入时间比正常的多一点，因为需要写两次才能确定。

I have a fairly generic question, so please pardon if it is a bit vague.

So, let's a assume a file of 1GB, that needs to be encrypted and later decrypted on a given system.

Problem is that the system has less than 512 mb of free memory and about 1.5 GB storage space (give or take), so, with the file "onboard" we have about ~500 MB of "hard drive scratch space" and less than 512 mb RAM to "play with".

The system is not unlikely to experience an "unscheduled power down" at any moment during encryption or decryption, and needs to be able to successfully resume the encryption/decryption process after being powered up again (and this seems like an extra-unpleasant nut to tackle).

The questions are:

1) is it at all doable :) ?

2) what would be the best strategy to go about

a) encrypting/decrypting with so little scratch space (can't have the entire file lying around while decrypting/encrypting, need to truncate it "on the fly" somehow...)

and

b) implementing a disaster recovery that would work in such a constrained environment?

P.S.: The cipher used has to be AES.

I looked into AES-CTR specifically but it does not seem to bode all that well for the disaster recovery shenanigan in an environment where you can't keep the entire decrypted file around till the end...

[edited to add] I think I'll be doing it the Iserni way after all.

解决方案

It is doable, provided you have a means to save the AES status vector together with the file position.

1. Save AES status and file position P to files STAGE1 and STAGE2
2. Read one chunk (say, 10 megabytes) of encrypted/decrypted data
3. Write the decrypted/encrypted chunk to external scratch SCRATCH
4. Log the fact that SCRATCH is completed
5. Write SCRATCH over the original file at the same position
6. Log the fact that SCRATCH has been successfully copied
7. Goto 1

If you get a hard crash after stage 1, and STAGE1 and STAGE2 disagree, you just restart and assume the stage with the earliest P to be good. If you get a hard crash during or after stage 2, you lose 10 megabytes worth of work: but the AES and P are good, so you just repeat stage 2. If you crash at stage 3, then on recovery you won't find the marker of stage 4, and so will know that SCRATCH is unreliable and must be regenerated. Having STAGE1/STAGE2, you are able to do so. If you crash at stage 4, you will BELIEVE that SCRATCH must be regenerated, even if you could avoid this -- but you lose nothing in regenerating except a little time. By the same token, if you crash during 5, or before 6 is committed to disk, you just repeat stages 5 and 6. You know you don't have to regenerate SCRATCH because stage 4 was committed to disk. If you crash after stage 1, you will still have a good SCRATCH to copy.

All this assumes that 10 MB is more than a cache's (OS + hard disk if writeback) worth of data. If it is not, raise to 32 or 64 MB. Recovery will be proportionately slower.

It might help to flush() and sync(), if these functions are available, after every write-stage has been completed.

Total write time is a bit more than twice normal, because of the need of "writing twice" in order to be sure.

这篇关于在存储器和存储受限系统上加密和/或解密大文件（AES），具有“灾难恢复”的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！