当大小太大时，vector.resize 函数会破坏内存 [英] vector.resize function corrupting memory when size is too large
问题描述
发生的事情是我正在读取加密数据包，我遇到一个损坏的数据包，该数据包返回一个非常大的长度随机数.
what's happening is i'm reading encryption packets, and I encounter a corrupted packet that gives back a very large random number for the length.
size_t nLengthRemaining = packet.nLength - (packet.m_pSource->GetPosition() - packet.nDataOffset);
seckey.SecretValues.m_data.resize(nLengthRemaining);
在这段代码中，m_data 是一个 std::vector.由于数据包损坏，nLengthRemaining 太大，因此调整大小函数抛出.问题不在于 resize 抛出(我们处理异常)，而是 resize 已经损坏了内存，这会导致以后出现更多异常.
In this code m_data is a std::vector<unsigned char>. nLengthRemaining is too large due to a corrupted data packet, therefore the resize function throws. The problem isn't that resize throws (we handle the exceptions), but that resize has corrupted memory already and this leads to more exceptions later.
我想做的是在我调用resize之前知道长度是否太大，然后只有在没问题的情况下才调用resize.我尝试在调用调整大小之前放置此代码:
What I want to do is know if the length is too large before I call resize, then only call resize if it's ok. I have tried putting this code before the call to resize:
std::vector<unsigned char>::size_type nMaxSize = seckey.SecretValues.m_data.max_size();
if(seckey.SecretValues.m_data.size() + nLengthRemaining >= nMaxSize) {
throw IHPGP::PgpException("corrupted packet: length too big.");
}
seckey.SecretValues.m_data.resize(nLengthRemaining);
此代码使用 std::vector max_size 成员函数来测试 nLengthRemaining 是否更大.但这肯定不可靠，因为 nLengthRemaining 仍然小于 nMaxSize，但显然仍然足够大，导致调整大小有问题(nMaxSize 是 4xxxxxxxxx，nLengthRemaining 是 3xxxxxxxxx).
This code is using the std::vector max_size member function to test if the nLengthRemaining is larger. That must not be reliable though, because nLengthRemaining is still less than nMaxSize, but apparently still large enough to cause resize to have a problem (nMaxSize was 4xxxxxxxxx and nLengthRemaining is 3xxxxxxxxx).
此外，我还不确定 resize 会抛出什么异常.它不是 std::length_error 也不是 std::bad_alloc.它抛出什么异常对我来说真的不太重要，但我很想知道.
Also, I haven't determine what exception resize is throwing. It's not a std::length_error and it's not a std::bad_alloc. What exception it's throwing really isn't too important to me, but i'm curious to know.
顺便说一句，正如您所知，此代码在正常情况下确实可以正常工作.这种数据包损坏的情况是它发疯的唯一地方.请帮忙！谢谢.
btw, just so you know, this code does work correctly in normal cases. This case of a corrupted data packet is the only place where it goes crazy. Please help! thanks.
更新:
@迈克尔.现在，如果数据包大于 5 MB，我将忽略它.我将与其他团队成员讨论验证数据包的可能性(它可能已经在那里，但我不知道).我开始认为这确实是我们 STL 版本中的一个错误，它抛出的异常甚至不是 std::exception，这让我感到惊讶.我会尝试从我的主管那里找出我们正在运行的 STL 版本(我将如何检查?).
@Michael. For now i'll just ignore the packet if it's larger than 5 MB. I'll discuss with other team members about possibly validating the packets (it may already be there and I just don't know it). I'm starting to think it really is a bug in our version of STL, the exception it throws isn't even a std::exception, which surprized me. I'll try to find out from my supervisor what version of STL we're running too (how would I check?).
另一个更新:我只是证明这是我在 Visual Studio 6 开发机器上使用的 STL 版本中的一个错误.我编写了这个示例应用程序:
ANOTHER UPDATE: I just prove that it is a bug in the STL version I'm using on my Visual Studio 6 development machine. I wrote this sample app:
//VectorMaxSize.cpp : 定义控制台应用程序的入口点.//
// VectorMaxSize.cpp : Defines the entry point for the console application. //
#include "stdafx.h"
#include <vector>
#include <iostream>
#include <math.h>
#include <typeinfo>
typedef std::vector<unsigned char> vector_unsigned_char;
void fill(vector_unsigned_char& v) {
for (int i=0; i<100; i++) v.push_back(i);
}
void oput(vector_unsigned_char& v) {
std::cout << "size: " << v.size() << std::endl;
std::cout << "capacity: " << v.capacity() << std::endl;
std::cout << "max_size: " << v.max_size() << std::endl << std::endl;
}
void main(int argc, char* argv[]) {
{
vector_unsigned_char v;
fill(v);
try{
v.resize(static_cast<size_t>(3555555555));
}catch(std::bad_alloc&) {
std::cout << "caught bad alloc exception" << std::endl;
}catch(const std::exception& x) {
std::cerr << typeid(x).name() << std::endl;
}catch(...) {
std::cerr << "unknown exception" << std::endl;
}
oput(v);
v.reserve(500);
oput(v);
v.resize(500);
oput(v);
}
std::cout << "done" << std::endl;
}
在我的 VS6 开发机器上，它具有与加密项目相同的行为，它会造成各种破坏.当我在我的 Visual Studio 2008 机器上构建并运行它时，resize 将抛出 std::bad_alloc 异常并且向量不会被破坏，正如我们所期望的！是时候体验一下 EA Sport NCAA 足球了，嘿嘿！
On my VS6 dev machine it has the same behavior has the encryption project, it causes all kinds of havoc. When I build and run it on my Visual Studio 2008 machine, resize will throw a std::bad_alloc exception and the vector will not be corrupted, just as we would have expected! Time for some EA Sport NCAA football hehe!
推荐答案
我认为 vector::max_size() 几乎总是一个硬编码"的东西 - 它与多少内存无关系统/库准备动态分配.您的问题似乎是向量实现中的一个错误，当分配失败时，它会破坏事物.
I think that vector::max_size() is pretty much always a 'hard coded' thing - it's independent of how much memory the system/library is prepared to dynamically allocate. Your problem seems to be a bug in the vector implementation that corrupts things when an allocation fails.
Bug"这个词可能太强了.vector::resize() 是根据 vector::insert() 定义的，并且标准是关于 vector::insert() 的:
'Bug' might be too strong of a word. vector::resize() is defined in terms of vector::insert() and the standard says this about vector::insert():
如果异常不是由 T 的复制构造函数或赋值运算符引发的，则没有任何影响
If an exception is thrown other than by the copy constructor or assignment operator of T there are no effects
所以看起来可能有时 resize() 操作被允许破坏向量，但如果操作是异常安全的，它仍然会很好(我认为它不会期待图书馆这样做是不合时宜的，但也许这比我想象的要难).
So it seems like there may be times when the resize() operation is allowed to corrupt a vector, but it would still be nice if the operation were exception safe (and I think it wouldn't be out of line to expect the library to do that, but maybe it's harder than I imagine).
您似乎有几个合理的选择:
You seem to have a couple reasonable options:
- 更改或更新到没有损坏错误的库(您使用的是什么编译器/库版本?)
- 不要检查
vector::max_size()将nMaxSize设置为您自己合理的最大值，然后执行上述操作，但使用该阈值.
- change or update to a library that doesn't have the corruption bug (what compiler/library version are you using?)
- instead of checking against
vector::max_size()setnMaxSizeto your own reasonable maximum and do what you have above but using that threshold instead.
我看到您正在使用 VC6 - vector::resize() 中肯定存在一个错误，这可能与您的问题有关，但老实说我没有看补丁看看如何(实际上它是 vector::insert() 中的一个错误，但如前所述，resize() 调用 insert()).我想访问 Dinkumwares 的 VC6 错误修复页面 并应用修复.
I see that you're using VC6 - there's definitely a bug in vector::resize() that might have something to do with your problem, though looking at the patch I honestly don't see how (actually it's a bug in vector::insert(), but as mentioned, resize() calls insert()). I'd guess it would be worthwhile to visit Dinkumwares' page for bug fixes to VC6 and apply the fixes.
该问题也可能与该页面上的 <xmemory> 补丁有关 - 目前尚不清楚那里讨论的错误是什么，但是 vector::insert() 确实调用了 _Destroy() 并且 vector<> 确实定义了名称 _Ty 所以你可能会遇到这个问题.一件好事 - 您不必担心管理对标题的更改，因为 Microsoft 永远不会再接触它们.只需确保补丁进入版本控制并记录.
The problem might also have something to do with the <xmemory> patch on that page - it's unclear what the bug is that's discussed there, but vector::insert() does call _Destroy() and vector<> does define the name _Ty so you might be running into that problem. One nice thing - you won't have to worry about managing the changes to the headers, as Microsoft is never touching them again. Just make sure the patches make it into version control and get documented.
请注意，Effective STL"中的 Scott Meyers 建议使用 SGI's 或 <一个 href="http://www.stlport.org/" rel="noreferrer">STLPort 的 库以获得比 VC6 更好的 STL 支持.我还没有这样做，所以我不确定这些库的工作情况(但我也没有经常使用带有 STL 的 VC6).当然，如果您可以选择迁移到更新版本的 VC，请务必这样做.
Note that Scott Meyers in "Effective STL" suggests using SGI's or STLPort's library to get better STL support than comes with VC6. I haven't done that so I'm not sure how well those libraries work (but I also haven't used VC6 with STL very much). Of course, if you have the option to move to a newer version of VC, by all means do it.
再次
感谢您的测试程序...
Thanks for the test program...
VC6 对默认分配器的 _Allocate() 实现(在 _Allocate() 函数强制请求的分配大小为零并继续.请注意，零大小的分配请求几乎总是会成功(不是 vector 无论如何都会检查失败)，因此 vector::resize() 函数会愉快地尝试将其内容移动到新块中，至少可以说它不够大.所以堆被破坏了，它很可能会遇到一个无效的内存页面，无论如何 - 你的程序被软管了.
VC6's _Allocate() implementation for the default allocator (in <xmemory>) uses a signed int to specify the number of elements to allocate, and if the size passed in is negative (which apparently is what you're doing - certainly in the test program you are) the _Allocate() function forces the requested allocation size to zero and proceeds. Note that a zero-sized allocation request will pretty much always succeed (not that vector checks for a failure anyway), so the vector::resize() function merrily tries to move its contents into the new block, which isn't quite big enough to say the least. So the heap gets corrupted, it'll likely hit a invalid memory page, and regardless - your program is hosed.
所以底线是永远不要要求 VC6 一次性分配超过 INT_MAX 个对象.在大多数情况下(VC6 或其他情况)可能不是一个好主意.
So the bottom line is don't ever ask VC6 to allocate more than INT_MAX objects in one go. Probably not a great idea in most circumstances (VC6 or otherwise).
另外，你应该记住，当分配失败时，VC6 使用一个标准前的习惯用法，即从 new 返回 0 而不是抛出 bad_alloc.
Also, you should keep in mind that VC6 uses a pre-standard idiom of returning 0 from new when an allocation fails rather than throwing bad_alloc.
这篇关于当大小太大时，vector.resize 函数会破坏内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
