使用“灾难恢复"在内存和存储受限的系统上加密和/或解密大文件(AES) [英] encrypting and/or decrypting large files (AES) on a memory and storage constrained system, with "catastrophe recovery"
问题描述
我有一个相当笼统的问题，如果有点含糊，请见谅.
I have a fairly generic question, so please pardon if it is a bit vague.
所以，让我们假设一个 1GB 的文件，需要在给定的系统上加密然后解密.
So, let's a assume a file of 1GB, that needs to be encrypted and later decrypted on a given system.
问题是系统只有不到 512 MB 的可用内存和大约 1.5 GB 的存储空间(给予或接受)，因此，对于板载"文件，我们有大约 500 MB 的硬盘暂存空间"和小于 512 mb RAM 可以玩".
Problem is that the system has less than 512 mb of free memory and about 1.5 GB storage space (give or take), so, with the file "onboard" we have about ~500 MB of "hard drive scratch space" and less than 512 mb RAM to "play with".
系统在加密或解密过程中的任何时候都不太可能遇到计划外掉电"，并且需要在再次上电后能够成功恢复加密/解密过程(这似乎是一个额外的 -难以解决的问题).
The system is not unlikely to experience an "unscheduled power down" at any moment during encryption or decryption, and needs to be able to successfully resume the encryption/decryption process after being powered up again (and this seems like an extra-unpleasant nut to tackle).
问题是:
1) 是否可行:) ?
2) 最好的策略是什么
a) 使用如此小的暂存空间进行加密/解密(在解密/加密时不能将整个文件放在周围，需要以某种方式即时"截断它......)
a) encrypting/decrypting with so little scratch space (can't have the entire file lying around while decrypting/encrypting, need to truncate it "on the fly" somehow...)
和
b) 实施可在这种受限环境中工作的灾难恢复?
b) implementing a disaster recovery that would work in such a constrained environment?
附:使用的密码必须是 AES.
P.S.: The cipher used has to be AES.
我专门研究了 AES-CTR，但在无法将整个解密文件保留到最后的环境中，灾难恢复恶作剧似乎并不是那么好......
I looked into AES-CTR specifically but it does not seem to bode all that well for the disaster recovery shenanigan in an environment where you can't keep the entire decrypted file around till the end...
我想我还是会按照 Iserni 的方式来做.
[edited to add] I think I'll be doing it the Iserni way after all.
推荐答案
这是可行的，只要您有办法将 AES 状态向量与文件位置一起保存.
It is doable, provided you have a means to save the AES status vector together with the file position.
- 将 AES 状态和文件位置 P 保存到文件 STAGE1 和 STAGE2
- 读取一大块(例如 10 兆字节)的加密/解密数据
- 将解密/加密的块写入外部暂存器 SCRATCH
- 记录 SCRATCH 已完成的事实
- 在原文件的相同位置写入 SCRATCH
- 记录 SCRATCH 已成功复制的事实
- 转到 1
如果您在第 1 阶段之后遇到严重崩溃，并且 STAGE1 和 STAGE2 不同意，您只需重新启动并假设 P 最早的阶段是好的.如果您在第 2 阶段期间或之后发生严重崩溃，您将损失 10 兆字节的工作:但是 AES 和 P 很好，所以您只需重复第 2 阶段.如果您在第 3 阶段崩溃，那么在恢复时您将找不到第 4 阶段的标记，因此会知道 SCRATCH 不可靠，必须重新生成.拥有 STAGE1/STAGE2，您可以这样做.如果你在第 4 阶段崩溃，你会相信 SCRATCH 必须重新生成，即使你可以避免这种情况——但你在重新生成过程中不会损失什么，除了一点点时间.出于同样的原因，如果您在 5 期间崩溃，或者在 6 被提交到磁盘之前，您只需重复第 5 和 6 阶段.您知道您不必重新生成 SCRATCH，因为第 4 阶段已提交到磁盘.如果您在第 1 阶段后崩溃，您仍然可以复制一个好的 SCRATCH.
If you get a hard crash after stage 1, and STAGE1 and STAGE2 disagree, you just restart and assume the stage with the earliest P to be good. If you get a hard crash during or after stage 2, you lose 10 megabytes worth of work: but the AES and P are good, so you just repeat stage 2. If you crash at stage 3, then on recovery you won't find the marker of stage 4, and so will know that SCRATCH is unreliable and must be regenerated. Having STAGE1/STAGE2, you are able to do so. If you crash at stage 4, you will BELIEVE that SCRATCH must be regenerated, even if you could avoid this -- but you lose nothing in regenerating except a little time. By the same token, if you crash during 5, or before 6 is committed to disk, you just repeat stages 5 and 6. You know you don't have to regenerate SCRATCH because stage 4 was committed to disk. If you crash after stage 1, you will still have a good SCRATCH to copy.
所有这些都假设 10 MB 大于缓存(如果写回，则为操作系统 + 硬盘)的数据价值.如果不是，请提高到 32 或 64 MB.恢复会相应地变慢.
All this assumes that 10 MB is more than a cache's (OS + hard disk if writeback) worth of data. If it is not, raise to 32 or 64 MB. Recovery will be proportionately slower.
如果这些函数可用，则在每个写入阶段完成后刷新()和同步()可能会有所帮助.
It might help to flush() and sync(), if these functions are available, after every write-stage has been completed.
总的写入时间比正常的两倍多一点，因为需要写入两次"才能确定.
Total write time is a bit more than twice normal, because of the need of "writing twice" in order to be sure.
这篇关于使用“灾难恢复"在内存和存储受限的系统上加密和/或解密大文件(AES)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
