为什么 LuaJIT 的内存在 64 位平台上被限制在 1-2 GB? [英] Why is LuaJIT's memory limited to 1-2 GB on 64-bit platforms?
问题描述
在 64 位平台上，LuaJIT 最多只允许 1-2GB 的数据(不包括使用 malloc 分配的对象).这种限制从何而来，为什么比 32 位平台还要少?
On 64-bit platforms, LuaJIT allows only up to 1-2GB of data (not counting objects allocated with malloc). Where does this limitation come from, and why is this even less than on 32-bit platforms?
推荐答案
LuaJIT 旨在使用 32 位指针.在 x64 平台上，限制来自于 mmap 的使用和 MAP_32BIT 标志.
LuaJIT is designed to use 32-bit pointers. On x64 platforms the limit comes from the use of mmap and the MAP_32BIT flag.
MAP_32BIT(自 Linux 2.4.20、2.6 起):
MAP_32BIT (since Linux 2.4.20, 2.6):
将映射放入进程地址空间的前 2 GB.对于 64 位程序，此标志仅在 x86-64 上受支持.添加它是为了允许在前 2GB 内存中的某处分配线程堆栈，从而提高一些早期 64 位处理器上的上下文切换性能.
Put the mapping into the first 2 Gigabytes of the process address space. This flag is supported only on x86-64, for 64-bit programs. It was added to allow thread stacks to be allocated somewhere in the first 2GB of memory, so as to improve context-switch performance on some early 64-bit processors.
本质上使用此标志限制为前 31 位，而不是顾名思义的前 32 位.看看这里 在 Linux 内核中使用 MAP_32BIT 对 1GB 限制的一个很好的概述.
Essentially using this flag limits to the first 31-bits, not the first 32-bits as the name suggests. Have a look here for a nice overview of the 1GB limit using MAP_32BIT in the Linux kernel.
即使你可以拥有超过 1GB，LuaJIT 作者解释了为什么这会影响性能:
Even if you could have more than 1GB, the LuaJIT author explains why this would be bad for performance:
- 完整的 GC 比分配本身多花费 50% 的时间.
- 如果启用了 GC，它会将分配时间加倍.
- 为了模拟真实的应用程序，对象之间的链接在第三次运行中随机化.这会使 GC 时间加倍！
那只是 1GB！现在想象一下使用 8GB —— 一个完整的 GC 周期将使 CPU 忙碌高达 24 秒！好的，所以正常模式是使用增量GC.但这只是意味着开销要高出约 30%，它在分配之间混合，每次都会驱逐 CPU 缓存.基本上你的应用程序将被 GC 开销所支配，你会开始想知道为什么它很慢......
And that was just for 1GB! Now imagine using 8GB -- a full GC cycle would keep the CPU busy for a whopping 24 seconds! Ok, so the normal mode is to use the incremental GC. But this just means the overhead is ~30% higher, it's mixed in between the allocations and it will evict the CPU cache every time. Basically your application will be dominated by the GC overhead and you'll begin to wonder why it's slow ....
tl;dr 版本:不要在家里尝试这个.并且 GC 需要重写(推迟到 LuaJIT 2.1).
tl;dr version: Don't try this at home. And the GC needs a rewrite (postponed to LuaJIT 2.1).
总而言之，1GB 的限制是 Linux 内核和 LuaJIT 垃圾收集器的限制.这仅适用于 LuaJIT 状态内的对象，可以通过使用 malloc 来克服，这将在低 32 位地址空间之外分配.此外，可以在 32 位模式下使用基于 x64 的 x86 构建并访问完整的 4GB.
To summarize, the 1GB limit is a limitation of the Linux kernel and the LuaJIT garbage collector. This only applies to objects within the LuaJIT state and can be overcome by using malloc, which will allocate outside the lower 32-bit address space. Also, it's possible to use the x86 build on x64 in 32-bit mode and have access the full 4GB.
查看这些链接了解更多信息:
Check out these links for more information:
- 如何获得Linux 上的 64 位 LuaJIT 是否超过了 1GB 的内存限制?
- LuaJIT x64 限制为 31 位地址空间，即使没有 MAP_32BIT 限制?
- LuaJIT 奇怪的内存限制
- 挖掘2008 年你从未听说过的最疯狂的错误:Linux 线程回归
这篇关于为什么 LuaJIT 的内存在 64 位平台上被限制在 1-2 GB?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
