无法使用 mmap/malloc/open 等在 64 位 linux 系统上创建大于 2GB 的文件 [英] can't create a file larger than 2GB on 64 bit linux system with mmap/malloc/open etc

问题描述

好的，我知道以前曾以各种形式提出过这样的问题，我已经阅读了所有这些问题并尝试了所有建议的内容，但我仍然无法在 64 位系统上使用 malloc、open、lseek，等等，在阳光下的每一个把戏.

OK I know questions like this have been asked in various forms before and I have read them all and tried everything that has been suggested but I still cannot create a file that is more than 2GB on a 64bit system using malloc, open, lseek, blah blah every trick under the sun.

显然我在这里写的是 c.我正在运行 Fedora 20，我实际上是在尝试映射文件，但这不是它失败的地方，我原来的方法是使用 open()，然后查找到文件应该结束的位置，在这种情况下是3GB，edit: 然后在文件结束位置写一个字节实际创建那个大小的文件，然后mmap这个文件.我无法找到超过 2GB 的空间.我也不能 malloc 超过 2GB.ulimit -a 等都显示无限制，/etc/security/limits.conf 没有显示，....

Clearly I'm writing c here. I'm running Fedora 20, I'm actually trying to mmap the file but that is not where it fails, my original method was to use open(), then lseek to the position where the file should end which in this case is at 3GB, edit: and then write a byte at the file end position to actually create the file of that size, and then mmap the file. I cannot lseek to past 2GB. I cannot malloc more than 2GB either. ulimit -a etc all show unlimited, /etc/security/limits.conf shows nothing, ....

当我尝试 lseek 超过 2GB 时，我得到 EINVAL for errno 并且 lseek 的 ret val 是 -1.edit:lseek 的 size 参数是 off_t 类型，它被定义为 long int(64 位有符号)，而不是 size_t正如我之前所说.

when I try to lseek past 2GB I get EINVAL for errno and the ret val of lseek is -1.edit: The size parameter to lseek is of type off_t which is defined as a long int (64bit signed), not size_t as I said previously.

我已经尝试定义 _LARGEFILE64_SOURCE &_FILE_OFFSET_BITS 64 并没有什么区别.我也在专门为 64 位编译，即 -m64

edit: I've already tried defining _LARGEFILE64_SOURCE & _FILE_OFFSET_BITS 64 and it made no difference. I'm also compiling specifically for 64bit i.e. -m64

我迷路了.我不知道为什么我不能这样做.

I'm lost. I have no idea why I cant do this.

任何帮助将不胜感激.

谢谢.

我已经删除了很多完全不正确的胡言乱语，以及稍后处理的其他一些不重要的胡言乱语.

edit: I've removed a lot of completely incorrect babbling on my part and some other unimportant ramblings that have been dealt with later on.

我的 2GB 问题在于多种不同类型的可怕草率互换.混合签名和未签名是问题所在.基本上，我传递给 lseek 的 3GB 位置被解释/转换为 -1GB 的位置，显然 lseek 不喜欢那样.所以我的不好.完全愚蠢.

My 2GB problem was in the horribly sloppy interchanging of multiple different types. Mixing of signed and unsigned being the problem. Essentially the 3GB position I was passing to lseek was being interpreted/turned into a position of -1GB and clearly lseek didnt like that. So my bad. Totally stupid.

我将按照 p_l 的建议改为使用 posix_fallocate().虽然它确实删除了一个函数调用，即只需要 posix_fallocate 而不是 lseek 然后写，对我来说这并不重要，事实上 posix_fallocate 正在做我想要的事情，而 lseek 方法没有.因此，特别感谢 p_l 提出这一建议，并特别感谢 NominalAnimal，他知道他更清楚的坚持，间接使我意识到我无法计数，这反过来又使我接受了 posix_fallocate 会起作用，因此改用它.

I am going to change to using posix_fallocate() as p_l suggested. While it does remove one function call i.e. only need posix_fallocate instead of an lseek and then a write, for me that isn't significant, it is the fact that posix_fallocate is doing exactly what I want directly which the lseek method doesn't. So thanks in particular to p_l for suggesting that, and a special thanks to NominalAnimal whose persistence that he knew better indirectly lead me to the realisation that I cant count which in turn led me to accept that posix_fallocate would work and so change to using it.

无论我使用的结束方法如何.2GB 的问题完全是我自己的垃圾编码，再次感谢 EOF、chux、p_l 和 Jonathon Leffler，他们都提供了信息和建议，让我解决了我为自己创建的问题.

Regardless of the end method I used. The problem of 2GB was entirely my own crap coding and thanks again to EOF, chux, p_l and Jonathon Leffler who all contributed information and suggestions that lead me to the problem I had created for myself.

我在答案中包含了这个的较短版本.

I've included a shorter version of this in an answer.

推荐答案

我的 2GB 问题在于多种不同类型的可怕草率互换.混合签名和未签名是问题所在.基本上，我传递给 lseek 的 3GB 位置被解释/转换为 -1GB 的位置，显然 lseek 不喜欢那样.所以我的不好.完全愚蠢的垃圾编码.

My 2GB problem was in the horribly sloppy interchanging of multiple different types. Mixing of signed and unsigned being the problem. Essentially the 3GB position I was passing to lseek was being interpreted/turned into a position of -1GB and clearly lseek didnt like that. So my bad. Totally stupid crap coding.

再次感谢 EOF、chux、p_l 和 Jonathon Leffler，他们都提供了信息和建议，引导我解决我所创建的问题及其解决方案.

Thanks again to EOF, chux, p_l and Jonathon Leffler who all contributed information and suggestions that lead me to the problem I'd created and its solution.

再次感谢 p_l 建议使用 posix_fallocate()，并特别感谢 NominalAnimal，他知道他更清楚的坚持，间接让我意识到我无法数数，这反过来又让我接受 posix_fallocate 会起作用，因此更改为使用它.

Thanks again to p_l for suggesting posix_fallocate(), and a special thanks to NominalAnimal whose persistence that he knew better indirectly lead me to the realisation that I cant count which in turn led me to accept that posix_fallocate would work and so change to using it.

@p_l 虽然我的实际问题的解决方案不在您的答案中，但我仍然会投票支持您建议使用 posix_fallocate 的答案，但我没有足够的分数来做到这一点.

@p_l although the solution to my actual problem wasn't in your answer, I'd still up vote your answer that suggested using posix_fallocate but I dont have enough points to do that.

这篇关于无法使用 mmap/malloc/open 等在 64 位 linux 系统上创建大于 2GB 的文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！