无法使用mmap/malloc/open等在64位linux系统上创建大于2GB的文件 [英] can't create a file larger than 2GB on 64 bit linux system with mmap/malloc/open etc

问题描述

好吧，我知道以前已经以各种形式提出过类似的问题，我已经阅读了所有问题并尝试了所建议的一切，但是在使用malloc的64位系统上，我仍然无法创建大于2GB的文件，别这样，等等.

OK I know questions like this have been asked in various forms before and I have read them all and tried everything that has been suggested but I still cannot create a file that is more than 2GB on a 64bit system using malloc, open, lseek, blah blah every trick under the sun.

很明显，我在这里写c.我正在运行Fedora 20，实际上是在尝试映射文件，但这并不是失败的地方，我的原始方法是使用open()，然后将文件定位到文件应结束的位置，在这种情况下为3GB，然后在文件结束位置写入一个字节，以实际创建该大小的文件，然后映射该文件.我不能超过2GB.我也不能分配超过2GB的内存. ulimit -a等都显示无限制，/etc/security/limits.conf不显示任何内容，....

Clearly I'm writing c here. I'm running Fedora 20, I'm actually trying to mmap the file but that is not where it fails, my original method was to use open(), then lseek to the position where the file should end which in this case is at 3GB, edit: and then write a byte at the file end position to actually create the file of that size, and then mmap the file. I cannot lseek to past 2GB. I cannot malloc more than 2GB either. ulimit -a etc all show unlimited, /etc/security/limits.conf shows nothing, ....

当我尝试超过2GB时，我得到errno的EINVAL，而lseek的ret val为-1.edit:lseek的大小参数为off_t类型，该类型定义为long int(64位带符号)，而不是size_t如我之前所说.

when I try to lseek past 2GB I get EINVAL for errno and the ret val of lseek is -1.edit: The size parameter to lseek is of type off_t which is defined as a long int (64bit signed), not size_t as I said previously.

我已经尝试定义_LARGEFILE64_SOURCE& _FILE_OFFSET_BITS 64，并且没有区别. 我还专门针对64位(即-m64

edit: I've already tried defining _LARGEFILE64_SOURCE & _FILE_OFFSET_BITS 64 and it made no difference. I'm also compiling specifically for 64bit i.e. -m64

我迷路了.我不知道为什么我不能这样做.

I'm lost. I have no idea why I cant do this.

任何帮助将不胜感激.

谢谢.

edit:我已经删除了很多完全不正确的胡言乱语，以及稍后处理的其他一些不重要的杂语.

edit: I've removed a lot of completely incorrect babbling on my part and some other unimportant ramblings that have been dealt with later on.

我的2GB问题是多种不同类型的草率互换.有符号和无符号的混合是问题.从本质上讲，我传递给lseek的3GB位置被解释/变成了-1GB的位置，显然lseek不喜欢那样.所以我不好.完全愚蠢.

My 2GB problem was in the horribly sloppy interchanging of multiple different types. Mixing of signed and unsigned being the problem. Essentially the 3GB position I was passing to lseek was being interpreted/turned into a position of -1GB and clearly lseek didnt like that. So my bad. Totally stupid.

我将更改为按照p_l建议使用posix_fallocate().虽然它确实删除了一个函数调用，即只需要posix_fallocate而不是lseek然后进行写入，但对我来说并不重要，事实是posix_fallocate确实在直接执行我想要的东西，而lseek方法却没有.因此，特别要感谢p_l的建议，尤其要感谢NominalAnimal的坚持，他更了解他的坚持，这间接地使我意识到我无法计数，这反过来又使我接受posix_fallocate可以工作，因此改变了使用它.

I am going to change to using posix_fallocate() as p_l suggested. While it does remove one function call i.e. only need posix_fallocate instead of an lseek and then a write, for me that isn't significant, it is the fact that posix_fallocate is doing exactly what I want directly which the lseek method doesn't. So thanks in particular to p_l for suggesting that, and a special thanks to NominalAnimal whose persistence that he knew better indirectly lead me to the realisation that I cant count which in turn led me to accept that posix_fallocate would work and so change to using it.

无论我使用哪种最终方法. 2GB的问题完全是我自己的废话编码，再次感谢EOF，chux，p_l和Jonathon Leffler，他们全都提供了信息和建议，使我得以解决自己为自己创建的问题.

Regardless of the end method I used. The problem of 2GB was entirely my own crap coding and thanks again to EOF, chux, p_l and Jonathon Leffler who all contributed information and suggestions that lead me to the problem I had created for myself.

我在回答中包括了一个简短的版本.

I've included a shorter version of this in an answer.

推荐答案

我的2GB问题是多种不同类型的草率互换.有符号和无符号的混合是问题.从本质上讲，我传递给lseek的3GB位置被解释/变成了-1GB的位置，显然lseek不喜欢那样.所以我不好.完全是愚蠢的废话编码.

My 2GB problem was in the horribly sloppy interchanging of multiple different types. Mixing of signed and unsigned being the problem. Essentially the 3GB position I was passing to lseek was being interpreted/turned into a position of -1GB and clearly lseek didnt like that. So my bad. Totally stupid crap coding.

再次感谢EOF，chux，p_l和Jonathon Leffler，他们全都提供了信息和建议，使我得以解决所创建的问题及其解决方案.

Thanks again to EOF, chux, p_l and Jonathon Leffler who all contributed information and suggestions that lead me to the problem I'd created and its solution.

再次感谢p_l建议posix_fallocate()，尤其要感谢NominalAnimal，他的持久性使他更加了解，这间接地使我认识到我无法计数，这反过来又使我接受posix_fallocate可以工作，因此更改为使用它.

Thanks again to p_l for suggesting posix_fallocate(), and a special thanks to NominalAnimal whose persistence that he knew better indirectly lead me to the realisation that I cant count which in turn led me to accept that posix_fallocate would work and so change to using it.

@p_l尽管我的实际问题的解决方案不在您的答案中，但我仍然会投票赞成使用posix_fallocate提出的答案，但是我没有足够的观点来做到这一点.

@p_l although the solution to my actual problem wasn't in your answer, I'd still up vote your answer that suggested using posix_fallocate but I dont have enough points to do that.

这篇关于无法使用mmap/malloc/open等在64位linux系统上创建大于2GB的文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！