使用文件缓冲区循环加密文件 [英] Encrypt file using file buffer loop

问题描述

去年，我使用AES 256 GCM使用C ++和crypto ++ lib进行了加密程序。今年我想将其升级到QT，并更改我在文件中阅读的方式。旧的方式是将整个文件读入一个char *，然后加密并写出来。我注意到大文件不起作用，所以我需要将其切换到缓冲区。

Last year I made an encryption program using AES 256 GCM using C++ and the crypto++ lib. This year I wanted to upgrade it to QT and change the way I was reading in the file. The old way was reading the entire file into a char* and then encrypting it and writing it out. I noticed that big files did not work, so I needed to switch this to a buffer.

我将其切换到读取8kb，加密，写入重复系统，但现在每次循环时，它会向输出添加一个额外的33bytes，我不知道为什么。这意味着如果文件大小< 8KB的工作原理，如果文件大小在8KB和16KB之间，输出会增加一个额外的33bytes，如果文件大小在16KB到24KB之间，输出会增加一个额外的66bytes等。

I switched it to a read 8kb, encrypt, write repeat system, but now every time it loops, it adds an additional 33bytes to the output, and I am not sure why. This means that if the file size < 8KB it works, if the filesize is between 8KB and 16KB the output adds an extra 33bytes, if the filesize is between 16KB and 24KB the output adds an extra 66bytes etc.

到目前为止我已经能够弄清楚，它不是加密代码，因为它适用于小于8KB的文件，它不是文件循环代码，因为我用简单的复制文件代码替换了加密代码，而且正确复制文件。

What I have been able to figure out so far is it is not the encryption code since it works on files less than 8KB, and it is not the file loop code, since I replaced the encryption code with a simple copy file code, and it copied the file correctly.

我认为问题是我没有重置变量，而是以每个循环的方式搞乱加密代码的数据Feed。

I think the problem is I am not resetting a variable and it is somehow messing up the data feed to the encryption code every loop.

这是我的代码

void encryptfile(double progressbarfilecount, bool& threadstatus) {    

// variables for file data
int buffersize = 8192;
string fullfilename;
string filepath;
string filename;
char memblock[8192];
streampos size;
double filesize;
double encryptedfilesize;
string datastring;
CryptoPP::SecByteBlock initializationvector(32);
string initializationvectorstring;
string cipher;
string encoded;
QMessageBox msgBox;

// encrypt the file
// get the filepath and filename
fullfilename = listbox1->item(progressbarfilecount)->text().toUtf8().constData();
size_t found = fullfilename.find_last_of("/\\");
filepath = fullfilename.substr(0,found);
filename = fullfilename.substr(found + 1);

// get the file size
//QFile myFile(QString::fromStdString(fullfilename));
//filesize = myFile.size();
//myFile.close();
filesize = getfilesize(fullfilename);
 qDebug() << "filesize:" << QString::number(filesize);

// setup the file data
ifstream originalfile(fullfilename, ios::in | ios::binary | ios::ate);
ofstream encryptedfile(fullfilename + ".txt", ios::app);

// get random initializationvector
randomnumber.GenerateBlock(initializationvector, initializationvector.size());

// convert it to a string for the text filee
initializationvectorstring = string((char *)initializationvector.begin(),32);

// check if we should get the checksum of the original file
if (testencryptiontogglebuttonguisetting == "On") {
    originalfilechecksum << checksum(fullfilename);
}



// here is the loop where the problem maybe



// encrypt the file 8KB at a time
for (encryptedfilesize = 0; encryptedfilesize < filesize; encryptedfilesize+= buffersize) {
    // check if the data left to write is less than the buffer size
    if (filesize - encryptedfilesize < buffersize) {
        buffersize = filesize - encryptedfilesize;
        qDebug() << "new buffersize:" << QString::number(buffersize);
    }

    // read the file into a memory block
    originalfile.seekg(encryptedfilesize);
    originalfile.read(memblock, buffersize);

    // convert the memoryblock to readable hexadecimal
    datastring = stringtohexadecimal(string(memblock, buffersize), true);

    // encrypt
    try
    {
    GCM< AES >::Encryption e;
    e.SetKeyWithIV(key, sizeof(key), initializationvector,initializationvector.size());
    // Not required for GCM mode (but required for CCM mode)
    // e.SpecifyDataLengths( adata.size(), pdata.size(), 0 );

    AuthenticatedEncryptionFilter ef(e,new StringSink(cipher), false, TAG_SIZE); // AuthenticatedEncryptionFilter

    // AuthenticatedEncryptionFilter::ChannelPut
    //  defines two channels: "" (empty) and "AAD"
    //   channel "" is encrypted and authenticated
    //   channel "AAD" is authenticated
    ef.ChannelPut("AAD", (const byte*)adata.data(), adata.size());
    ef.ChannelMessageEnd("AAD");

    // Authenticated data *must* be pushed before
    //  Confidential/Authenticated data. Otherwise
    //  we must catch the BadState exception
    ef.ChannelPut("", (const byte*)datastring.data(), datastring.size());
    ef.ChannelMessageEnd("");

    // Pretty print
    StringSource(cipher, true,new HexEncoder(new StringSink(encoded), true, 16, " "));
    }
    catch (CryptoPP::BufferedTransformation::NoChannelSupport&)
    {
    // The tag must go in to the default channel:
    //  "unknown: this object doesn't support multiple channels"
        if (operatingsystem() == "Linux") {
            system("error_message_encrypt_file_error.sh");
        }
        if (operatingsystem() == "Windows") {
            ShellExecute(0, L"open", L"error_message_encrypt_file_error.vbs", 0, 0, SW_NORMAL);
        }
    //msgBox.setText("No Channel Support");
    //msgBox.exec();
    return;
    }
    catch (CryptoPP::AuthenticatedSymmetricCipher::BadState&)
    {
    // Pushing PDATA before ADATA results in:
    //  "GMC/AES: Update was called before State_IVSet"
        if (operatingsystem() == "Linux") {
            system("error_message_encrypt_file_error.sh");
        }
        if (operatingsystem() == "Windows") {
            ShellExecute(0, L"open", L"error_message_encrypt_file_error.vbs", 0, 0, SW_NORMAL);
        }
    //msgBox.setText("Data was read before adata");
    //msgBox.exec();
    return;
    }
    catch (CryptoPP::InvalidArgument&)
    {
        if (operatingsystem() == "Linux") {
            system("error_message_encrypt_file_invalid.sh");
        }
        if (operatingsystem() == "Windows") {
            ShellExecute(0, L"open", L"error_message_encrypt_file_invalid.vbs", 0, 0, SW_NORMAL);
        }
    //msgBox.setText("Invalid Argument");
    //msgBox.exec();
    return;
    }

    // convert the cipher to hexadecimal string
    cipher = stringtohexadecimal(cipher, true);

    // write the encrypted file to a text file with the original file extension
    // check to see if we need to write the initialization vector
    if (encryptedfilesize == 0) {
        initializationvectorstring = stringtohexadecimal(initializationvectorstring, true);
        encryptedfile << initializationvectorstring;
        qDebug() << "wrote the initilization vector";
    }
    encryptedfile << encoded;        
    qDebug() << "encrypted filesize:" << QString::number(encryptedfilesize);

    // clear the variables
    encoded = "";
    cipher = "";
    initializationvectorstring = "";
    keys = "";

}

// close the file data
originalfile.close();
encryptedfile.close();

如果有人可以帮助我弄清楚代码有什么问题，我会很感激。 p>

If anyone could help me figure out what is wrong with the code, I would appreciate it.

推荐答案

去年，我使用C ++和crypto ++ lib进行AES 256 GCM的加密程序。今年我想将其升级到QT，并更改我在文件中阅读的方式。旧的方式是将整个文件读入一个char *，然后加密并写出来。我注意到大文件不起作用，所以我需要将其切换到缓冲区...

Last year I made an encryption program using AES 256 GCM using C++ and the crypto++ lib. This year I wanted to upgrade it to QT and change the way I was reading in the file. The old way was reading the entire file into a char* and then encrypting it and writing it out. I noticed that big files did not work, so I needed to switch this to a buffer...

在最高级别，你似乎有两个设计要求。首先，您需要在避免密文扩展的同时块数据。第二，您需要集成经过身份验证的加密方案。

At the highest levels, you appear to have two design requirements. First, you need to chunk your data while avoiding cipher text expansion. Second, you need to integrate an authenticated encryption scheme.

每个循环的额外16个字节大小都是由于将身份验证标签添加到每个加密块中。相信与否，这有时是一个理想的财产。例如，下载4.7 GB Gentoo图像并查找整个图像的图像已损坏，并最终被拒绝。它由于：

The extra 16 bytes or so on each loop are due to the authentication tag being added to each encrypted chunk. Believe it or not, this is sometimes a desirable property. For example, image downloading a 4.7 GB Gentoo image and finding out the entire image is corrupt and eventually rejected. Its due to:

for (encryptedfilesize = 0; encryptedfilesize < filesize; encryptedfilesize+= buffersize)
{
    ...
    AuthenticatedEncryptionFilter ef(e,new StringSink(cipher), false, TAG_SIZE); // AuthenticatedEncryptionFilter    
    ...
}

为了实现你的目标，我认为你将需要做两件事情。首先，为了回答如何阻止或块块数据的问题，您将需要 Pump 您的数据（Crypto ++将其称为管道说明）。这实际上已经被覆盖了，但它并不明显：

To achieve your goals, I think you are going to need to do two things. First, to answer the question of how to block or chunk the data, you are going to need to Pump your data (as Crypto++ calls it in Pipeline parlance). This has actually been covered previously, but its not readily apparent:

• 无法使用CryptoPP FileSink在线程运行方法中运行Qt线程


• Crypto ++例外调用messageEnd


• 使用管道加密文件

• Cannot kill Qt thread with CryptoPP FileSink running in the thread run method
• Crypto++ exception calling messageEnd
• Use of Pipelines to encrypt a file

以上处理Crypto ++中的数据的阻塞或分块。第二个问题如何避免每个区块上的认证标签，这里还没有被问到（如果内存服务器我正确的话）。

The above handles the blocking or chunking of data in Crypto++. The second issue, how to avoid an authentication tag on each block, has not been asked here (if memory server me correctly).

关于第二个问题的答案可以在 Init-Update-Final 上找到加密++维基。不足之处是，不要在每个循环迭代中创建一个新的 AuthenticatedEncryptionFilter 。相反，使用单个过滤器并调用 MaxRetrievable（）来确定是否有任何密码准备就绪。如果存在，则在可用时检索它。否则，过滤器将无限期地缓冲。

The answer to the second question can be found at Init-Update-Final on the Crypto++ wiki. The short of it is, don't create a new AuthenticatedEncryptionFilter on each loop iteration. Rather, use a single filter and call MaxRetrievable() to determine if there's any cipher text ready. If there is, then retrieve it as it becomes available. Otherwise, the filter will buffer it indefinitely.

Init-Update-Final 页面有一个例子。以下是更新功能的功能。我相信它主要是像你所期望的那样工作，例如Java（这就是为什么我们称之为 JavaCipher ）：

The Init-Update-Final page has an example. Here's how the update function looks. I believe it mostly works as you expect from, say, Java (that's why we called it JavaCipher):

size_t JavaCipher::update(const byte* in, size_t isize, byte* out, size_t osize)
{
    if(in && isize)
        m_filter.get()->Put(in, isize);

    if(!out || !osize || !m_filter.get()->AnyRetrievable())
        return 0;

    size_t t = STDMIN(m_filter.get()->MaxRetrievable(), (word64)osize);
    return m_filter.get()->Get(out, t);
}

当您致电 final ，那就是生成认证标签的时候。虽然它不明显，标签是在调用 MessageEnd（）中生成的：

When you call final, that's when the authentication tag is generated. While its not readily apparent, the tag is generated in the call to MessageEnd():

size_t JavaCipher::final(byte* out, size_t osize)
{
    m_filter.get()->MessageEnd();

    if(!out || !osize || !m_filter.get()->AnyRetrievable())
        return 0;

    size_t t = STDMIN(m_filter.get()->MaxRetrievable(), (word64)osize);
    return m_filter.get()->Get(out, t);
}

我没有使用认证加密模式，如EAX，CCM或GCM。我们可以解决您遇到的任何问题，同时更新维基页面以获得别人的利益。

I have not tested this with an authenticated encryption mode like EAX, CCM or GCM. We can work through any issues you experience while updating the wiki page for the benefit of others.

我已经知道您将需要转换 JavaCiper 成员 StreamTransformationFilter 为 AuthenticatedEncryptionFilter 进行加密，而 AuthenticatedDecryptionFilter 进行解密。 Artjom还在他的评论中详细说明了一些潜在的问题。

I already know you are going to need to swap-out JavaCiper member StreamTransformationFilter for a AuthenticatedEncryptionFilter for encryption, and an AuthenticatedDecryptionFilter for decryption. Artjom also details some potential issues in his comments.

我没有提供很多代码，我很抱歉。在我心目中，你的设计需要一些小的工作，所以你还没有准备好代码（还）。

My apologies for not providing a lot of code. In my mind's eye, your design needs some minor work, so you are not ready for code (yet).

我猜你会

I'm guessing you will be ready for code in your next set of questions (if you ask them here).

这篇关于使用文件缓冲区循环加密文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！