在Windows Vista和Windows 7上使用HEAP_NO_SERIALIZE使用堆内存功能导致〜100倍速度下降的原因 [英] Reason for ~100x slowdown with heap memory functions using HEAP_NO_SERIALIZE on Windows Vista and Windows 7

问题描述

我试图追踪Windows Vista和Windows 7中堆内存功能的巨大下降(我没有在任何服务器版本上进行测试).它根本不会在Windows XP上发生，而只会在Microsoft较新的操作系统上发生.

I'm trying to tracedown a huge slowdown in the heap memory functions in Windows Vista and Windows 7 (I didn't test on any server editions). It doesn't happen on Windows XP at all, only on Microsoft's newer operating systems.

最初，我在Windows上使用PHP时遇到了此问题.脚本本身似乎以预期的速度运行，但是在脚本执行后，我在内部PHP关闭功能中遇到了1-2秒的延迟.启动调试后，我发现它与PHP内存管理器对HeapAlloc/HeapFree/HeapReAlloc的使用有关.

I originally ran into this problem with PHP complied on Windows. The scripts themselves seemed to run at the expected speed, but after script execution I was experiencing 1-2 seconds of delay in the internal PHP shutdown functions. After firing up the debugging I saw that it had to do with the PHP memory manager's use of HeapAlloc/HeapFree/HeapReAlloc.

我追溯到在堆函数上使用标志HEAP_NO_SERIALIZE:

I traced it down to the use of the flag HEAP_NO_SERIALIZE on the heap functions:

#ifdef ZEND_WIN32
#define ZEND_DO_MALLOC(size) (AG(memory_heap) ? HeapAlloc(AG(memory_heap), HEAP_NO_SERIALIZE, size) : malloc(size))
#define ZEND_DO_FREE(ptr) (AG(memory_heap) ? HeapFree(AG(memory_heap), HEAP_NO_SERIALIZE, ptr) : free(ptr))
#define ZEND_DO_REALLOC(ptr, size) (AG(memory_heap) ? HeapReAlloc(AG(memory_heap), HEAP_NO_SERIALIZE, ptr, size) : realloc(ptr, size))
#else
#define ZEND_DO_MALLOC(size) malloc(size)
#define ZEND_DO_FREE(ptr) free(ptr)
#define ZEND_DO_REALLOC(ptr, size) realloc(ptr, size)
#endif

和(实际上设置了HeapAlloc/HeapFree/HeapReAlloc的默认值)在函数start_memory_manager中:

and (which actually sets the default for HeapAlloc/HeapFree/HeapReAlloc) in the function start_memory_manager:

#ifdef ZEND_WIN32
    AG(memory_heap) = HeapCreate(HEAP_NO_SERIALIZE, 256*1024, 0);
#endif

我删除了HEAP_NO_SERIALIZE参数(替换为0)，它解决了该问题.现在，可以在CLI和SAPI Apache 2版本中快速清除脚本.这是针对PHP 4.4.9的，但是PHP 5和6源代码(正在开发中)在调用中包含相同的标志.

I removed the HEAP_NO_SERIALIZE parameter (replaced with 0), and it fixed the problem. Scripts now cleanup quickly in both the CLI and the SAPI Apache 2 version. This was for PHP 4.4.9, but the PHP 5 and 6 source code (in development) contains the same flag on the calls.

我不确定我所做的事情是否危险.这都是PHP内存管理器的一部分，因此我将不得不进行一些挖掘和研究，但这提出了一个问题:

I'm not sure if what I did was dangerous or not. It's all a part of the PHP memory manager, so I'm going to have to do some digging and research, but this brings up the question:

为什么在具有HEAP_NO_SERIALIZE的Windows Vista和Windows 7上堆内存功能这么慢?

Why are the heap memory function so slow on Windows Vista and Windows 7 with HEAP_NO_SERIALIZE?

在研究此问题时，我想出了一个不错的选择.请阅读博客文章 http://www.brainfarter.net/?p=69 张贴者在其中解释了问题，并提供了一个测试案例(包括可用的源代码和二进制文件)来突出显示问题.

While researching this problem I came up with exactly one good hit. Please read the blog post http://www.brainfarter.net/?p=69 where the poster explains the issue and offers a test case (both source and binaries available) to highlight the issue.

我在Windows 7 x64四核8 GB计算机上的测试给出了 43,836 .哎哟!没有HEAP_NO_SERIALIZE标志的相同结果是 655 ，在我的情况下要快70倍.

My tests on a Windows 7 x64 quad core 8 GB machine gives 43,836. Ouch! The same results without the HEAP_NO_SERIALIZE flag is 655, ~70x faster in my case.

最后，似乎使用Visual C ++ 6使用malloc/free或new/delete创建的任何程序似乎都在这些较新的平台上受到影响.默认情况下，Visual C ++ 2008编译器不会为那些函数/运算符设置此标志，因此它们不会受到影响-但这仍然使许多程序受到影响！

Lastly, it seems that any program created with Visual C++ 6 using malloc/free or new/delete seems to be affected on these newer platforms. The Visual C++ 2008 compiler doesn't set this flag by default for those functions/operators so they aren't affected -- but that still leaves a lot of programs affected!

我鼓励您下载概念证明并尝试一下.这个问题解释了为什么我在Windows安装上正常运行的PHP正在爬网，并且可能解释了为什么Windows Vista和Windows 7有时看起来变慢的原因.

I encourage you to download the proof of concept and give this a try. This problem explained why my normal PHP on Windows installation was crawling and may explain why Windows Vista and Windows 7 seems slower at times.

更新2010-01-26::我收到Microsoft的回复，指出

UPDATE 2010-01-26: I received a response from Microsoft stating that the low-fragmentation heap (LFH) is the de facto default policy for heaps that hold any appreciable number of allocations. In Windows Vista, they reorganized a lot of code to remove extra data structures and code paths that were no longer part of the common case for handling heap API calls. With the HEAP_NO_SERIALIZE flag and in certain debugging situations, they do not allow the use of the LFH and we get stuck on the slow and less optimized path through the heap manager. So... it's highly recommended to not use HEAP_NO_SERIALIZE since you'll miss out on all the work to the LFH and any future work in the Windows heap API.

推荐答案

我注意到的第一个区别是Windows Vista 始终使用 Low Fragmentation Heap (LFH). Windows XP似乎没有.结果是Windows Vista中的RtlFreeHeap要短得多-所有工作都委派给RtlpLowFragHeapFree. 有关LFH及其在各种场合中的存在的更多信息操作系统.请注意顶部的红色警告.

The first difference I noticed is that Windows Vista always uses the Low Fragmentation Heap (LFH). Windows XP does not seem to. RtlFreeHeap in Windows Vista is a lot shorter as a result -- all the work is delegated to RtlpLowFragHeapFree. More information regarding LFH and its presence in various OSs. Note the red warning at the top.

Windows XP，Windows Server 2003和 带有修补程序KB 816542的Windows 2000:

Windows XP, Windows Server 2003, and Windows 2000 with hotfix KB 816542:

后备列表是快速记忆 包含的分配机制 仅固定大小的块.旁观 默认情况下为堆启用列表 支持他们. 从 Windows Vista，后备列表是 未使用，并且通过以下方式启用了LFH: 默认.

A look-aside list is a fast memory allocation mechanism that contains only fixed-sized blocks. Look-aside lists are enabled by default for heaps that support them. Starting with Windows Vista, look-aside lists are not used and the LFH is enabled by default.

另一个重要信息:LFH和NO_SERIALIZE是互斥的(不能同时处于活动状态).与

Another important piece of information: LFH and NO_SERIALIZE are mutually-exclusive (both cannot be active simultaneously). Combined with

开始 Windows Vista中的后备列表是 未使用

Starting with Windows Vista, look-aside lists are not used

这表示，在Windows Vista中设置NO_SERIALIZE会禁用LFH，但是不会(并且不能)退回到标准后备列表(作为快速替代方法)，根据上述说明引用.我不清楚在指定NO_SERIALIZE时Windows Vista使用哪种堆分配策略.根据其性能，它似乎正在使用一些天真的东西.

This implies that setting NO_SERIALIZE in Windows Vista disables LFH, but it does not (and cannot) fall back to standard look-aside lists (as a fast replacement), according to the above quote. I'm unclear as to what heap allocation strategy Windows Vista uses when NO_SERIALIZE is specified. It looks like it's using something horribly naïve, based on its performance.

查看allocspeed.exe的一些堆栈快照，它似乎始终处于就绪"状态(而不是运行"或等待")，并且处于HeapFree的TryEnterCriticalSection中，并以接近100％的负载将CPU固定40秒. (在Windows Vista上.)

Looking at a few stack snapshots of allocspeed.exe, it seems to always be in a Ready state (not Running or Wait), and in TryEnterCriticalSection from HeapFree, and pegging the CPU at nearly 100% load for 40 seconds. (On Windows Vista.)

示例快照:

ntdll.dll!RtlInterlockedPushEntrySList+0xe8
ntdll.dll!RtlTryEnterCriticalSection+0x33b
kernel32.dll!HeapFree+0x14
allocspeed.EXE+0x11ad
allocspeed.EXE+0x1e15
kernel32.dll!BaseThreadInitThunk+0x12
ntdll.dll!LdrInitializeThunk+0x4d

这很奇怪，因为NO_SERIALIZE精确地告诉它 skip 锁获取.东西没加起来.

Which is strange, because NO_SERIALIZE precisely tells it to skip lock acquisition. Something doesn't add up.

这只是一个问题， Raymond Chen 或

This is a question only Raymond Chen or Mark Russinovich could answer :)

这篇关于在Windows Vista和Windows 7上使用HEAP_NO_SERIALIZE使用堆内存功能导致〜100倍速度下降的原因的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！