为什么 malloc 有时不起作用? [英] Why does malloc not work sometimes?

问题描述

我正在将一个 C 项目从 Linux 移植到 Windows.在 Linux 上它是完全稳定的.在 Windows 上，它大部分时间都运行良好，但有时我会遇到分段错误.

我正在使用 Microsoft Visual Studio 2010 进行编译和调试，看起来有时我的 malloc 调用根本不分配内存，返回 NULL.机器有空闲内存；它已经通过该代码一千次了，但它仍然发生在不同的位置.

就像我说的，它不会一直发生，也不会发生在同一个地方；它看起来像一个随机错误.

在 Windows 上我需要比在 Linux 上更小心吗?我做错了什么?

解决方案

malloc() 在无法处理内存请求时返回无效的 NULL 指针.在大多数情况下，当调用 malloc() 并且没有块时，C 内存分配例程通过调用操作系统来管理可用内存的列表或堆以分配额外的内存块在列表或堆上以满足请求.

所以 malloc() 失败的第一种情况是内存请求无法满足，因为 (1) C 运行时的列表或堆上没有可用的内存块，并且(2)当C运行时内存管理向操作系统请求更多内存时，请求被拒绝.

这是一篇关于指针分配策略的文章.p>

这篇论坛文章给出了一个malloc由于内存碎片而失败的例子.p>

malloc() 可能失败的另一个原因是内存管理数据结构已损坏，这可能是由于缓冲区溢出，其中分配的内存区域用于大于分配的内存大小.malloc() 的不同版本可以使用不同的内存管理策略，并确定调用 malloc() 时提供多少内存.例如，一个 malloc() 可能会准确地为您提供请求的字节数，或者它可能会给您比您要求的更多的信息，以适应在内存边界内分配的块或使内存管理更容易.

使用现代操作系统和虚拟内存，除非您正在做一些非常大的内存驻留存储，否则内存很难用完.但是，正如用户 Yeow_Meng 在下面的评论中提到的那样，如果您正在做算术来确定要分配的大小并且结果是负数，您最终可能会请求大量内存，因为 malloc() 用于分配的内存量是无符号的.

在进行指针运算以确定某些数据需要多少空间时，您可能会遇到负大小的问题.这种错误对于对意外文本进行的文本解析很常见.例如，下面的代码会导致一个非常大的 malloc() 请求.

char pathText[64] = "./dir/prefix";//带有路径的文本缓冲区，使用点 (.) 表示当前目录char *pFile = strrchr (pathText, '/');//查找文件名开始的最后一个斜杠char *pExt = strrchr (pathText, '.');//寻找文件扩展名//此时程序员期望//- pFile 指向路径名中的最后一个斜杠//- pExt 指向文件扩展名中的点 (.) 或 NULL//然而，有了这些数据，我们反而有以下指针，因为而不是//绝对路径，是相对路径//- pFile 指向路径名中的最后一个斜杠//- pExt 指向路径名中的第一个点 (.)，因为没有文件扩展名//结果是非 NULL pExt 值大于 pFile，//对于这个特定的数据，它反而更小.字符 *pNameNoExt;if (pExt) {//这应该是 if (pExt && pFile < pExt) {//指定扩展名，所以只为名称分配空间，没有扩展名//只为没有扩展名的文件名分配空间//因为 pExt 小于 pFile，所以我们得到一个负值，然后变成//一个非常大的无符号值.pNameNoExt = malloc ((pExt - pFile + 1) * sizeof(char));} 别的 {pNameNoExt = malloc ((strlen(pFile) + 1) * sizeof(char));}

良好的运行时内存管理将尝试合并已释放的内存块，以便在释放时将许多较小的块组合成较大的块.这种内存块的组合减少了无法使用 C 内存管理运行时管理的列表或内存堆上已经可用的内存来服务内存请求的机会.

您可以重用已分配的内存越多，对 malloc() 和 free() 的依赖越少越好.如果你不做 malloc() 那么它很难失败.

您可以将许多对 malloc() 的小型调用更改为对 malloc() 的更少大型调用，您对内存进行碎片化和扩展的机会就越少内存列表或堆的大小，其中包含许多无法组合的小块，因为它们彼此不相邻.

您可以同时malloc() 和free() 连续块越多，内存管理运行时就越有可能合并块.

没有规定你必须使用对象的特定大小执行 malloc()，提供给 malloc() 的大小参数可以大于您为其分配内存的对象所需的大小.因此，您可能希望对 malloc () 的调用使用某种规则，以便通过四舍五入到某个标准内存量来分配标准大小的块.因此，您可以使用 ((size/16) + 1) * 16 或更可能 ((size >> 4) + 1) << 之类的公式分配 16 个字节的块.4. 许多脚本语言使用类似的东西来增加重复调用 malloc() 和 free() 能够将请求与空闲块匹配的机会在内存列表或堆上.

这是一个尝试减少分配和释放块数量的简单示例.假设我们有一个可变大小的内存块的链表.所以链表中节点的结构看起来像:

typedef struct __MyNodeStruct {结构 __MyNodeStruct *pNext;无符号字符 *pMegaBuffer;} 我的节点结构；

可能有两种方法可以为特定缓冲区及其节点分配此内存.第一个是节点的标准分配，然后是缓冲区的分配，如下所示.

MyNodeStruct *pNewNode = malloc(sizeof(MyNodeStruct));如果(pNewNode)pNewNode->pMegaBuffer = malloc(15000);

但是另一种方法是执行以下操作，它使用带有指针算法的单个内存分配，以便单个 malloc() 提供两个内存区域.

MyNodeStruct *pNewNode = malloc(sizeof(myNodeStruct) + 15000);如果(pNewNode)pNewNode->pMegaBuffer = ((unsigned char *)pNewNode) + sizeof(myNodeStruct);

但是，如果您使用这种单一分配方法，则需要确保您在使用指针 pMegaBuffer 时保持一致，以免意外执行 free() 就可以了.如果您必须用更大的缓冲区更改缓冲区，则需要释放节点并重新分配缓冲区和节点.所以程序员有更多的工作.

I'm porting a C project from Linux to Windows. On Linux it is completely stable. On Windows, it's working well most times, but sometimes I got a segmentation fault.

I'm using Microsoft Visual Studio 2010 to compile and debug and looks like sometimes my malloc calls simply doesn't allocate memory, returning NULL. The machine has free memory; it already passed through that code a thousand times, but it still happens in different locations.

Like I said, it doesn't happen all the time or in the same location; it looks like a random error.

Is there something I have to be more careful on Windows than on Linux? What can I be doing wrong?

解决方案

malloc() returns an invalid pointer of NULL when it is unable to service a memory request. In most cases the C memory allocation routines manage a list or heap of memory available memory with calls to the operating system to allocate additional chunks of memory when a malloc() call is made and there is not a block on the list or heap to satisfy the request.

So the first case of malloc() failing is when a memory request can not be satisfied because (1) there is not a usable block of memory on the list or heap of the C runtime and (2) when the C runtime memory management requested more memory from the operating system, the request was refused.

Here is an article about Pointer Allocation Strategies.

This forum article gives an example of malloc failure due to memory fragmentation.

Another reason why malloc() might fail is because the memory management data structures have become corrupted probably due to a buffer overflow in which a memory area that was allocated was used for an object larger than the size of the memory allocated. Different versions of malloc() can use different strategies for memory management and determining how much memory to provide when malloc() is called. For instance a malloc() may give you exactly the number of bytes requested or it may give you more than you asked for in order to fit the block allocated within memory boundaries or to make the memory management easier.

With modern operating systems and virtual memory, it is pretty difficult to run out of memory unless you are doing some really large memory resident storage. However as user Yeow_Meng mentioned in a comment below, if you are doing arithmetic to determine the size to allocate and the result is a negative number you could end up requesting a huge amount of memory because the argument to malloc() for the amount of memory to allocation is unsigned.

You can run into the problem of negative sizes when doing pointer arithmetic to determine how much space is needed for some data. This kind of error is common for text parsing that is done on text that is unexpected. For example the following code would result in a very large malloc() request.

char pathText[64] = "./dir/prefix";  // a buffer of text with path using dot (.) for current dir
char *pFile = strrchr (pathText, '/');  // find last slash where the file name begins
char *pExt = strrchr (pathText, '.');    // looking for file extension 

// at this point the programmer expected that
//   - pFile points to the last slash in the path name
//   - pExt point to the dot (.) in the file extension or NULL
// however with this data we instead have the following pointers because rather than
// an absolute path, it is a relative path
//   - pFile points to the last slash in the path name
//   - pExt point to the first dot (.) in the path name as there is no file extension
// the result is that rather than a non-NULL pExt value being larger than pFile,
// it is instead smaller for this specific data.
char *pNameNoExt;
if (pExt) {  // this really should be if (pExt && pFile < pExt) {
    // extension specified so allocate space just for the name, no extension
    // allocate space for just the file name without the extension
    // since pExt is less than pFile, we get a negative value which then becomes
    // a really huge unsigned value.
    pNameNoExt = malloc ((pExt - pFile + 1) * sizeof(char));
} else {
    pNameNoExt = malloc ((strlen(pFile) + 1) * sizeof(char));
}

A good run time memory management will try to coalesce freed chunks of memory so that many smaller blocks will be combined into larger blocks as they are freed. This combining of chunks of memory reduces the chances of being unable to service a memory request using what is already available on the list or heap of memory being managed by the C memory management run time.

The more that you can just reuse already allocated memory and the less you depend on malloc() and free() the better. If you are not doing a malloc() then it is difficult for it to fail.

The more that you can change many small size calls to malloc() to fewer large calls to malloc() the less chance you have for fragmenting the memory and expanding the size of the memory list or heap with lots of small blocks that can not be combined because they are not next to each other.

The more that you can malloc() and free() contiguous blocks at the same time, the more likely that the memory management run time can coalesce blocks.

There is no rule that says you must do a malloc() with the specific size of an object, the size argument provided to malloc() can be larger than the size needed for the object for which you are allocating memory. So you may want to use some kind of a rule for calls to malloc () so that standard sized blocks are allocated by rounding up to some standard amount of memory. So you may allocate in blocks of 16 bytes using a formula like ((size / 16) + 1) * 16 or more likely ((size >> 4) + 1) << 4. Many script languages use something similar so as to increase the chance of repeated calls to malloc() and free() being able to match up a request with a free block on the list or heap of memory.

Here is a somewhat simple example of trying to reduce the number of blocks allocated and deallocated. Lets say that we have a linked list of variable sized blocks of memory. So the struct for the nodes in the linked list look something like:

typedef struct __MyNodeStruct {
    struct __MyNodeStruct *pNext;
    unsigned char *pMegaBuffer;
} MyNodeStruct;

There could be two ways of allocating this memory for a particular buffer and its node. The first is a standard allocation of the node followed by an allocation of the buffer as in the following.

MyNodeStruct *pNewNode = malloc(sizeof(MyNodeStruct));
if (pNewNode)
    pNewNode->pMegaBuffer = malloc(15000);

However another way would be to do something like the following which uses a single memory allocation with pointer arithmetic so that a single malloc() provides both memory areas.

MyNodeStruct *pNewNode = malloc(sizeof(myNodeStruct) + 15000);
if (pNewNode)
    pNewNode->pMegaBuffer = ((unsigned char *)pNewNode) + sizeof(myNodeStruct);

However if you are using this single allocation method, you will need to make sure that you are consistent in the use of the pointer pMegaBuffer that you do not accidently do a free() on it. And if you are having to change out the buffer with a larger buffer, you will need to free the node and reallocate buffer and node. So there is more work for the programmer.

这篇关于为什么 malloc 有时不起作用?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！