fallocate vs posix_fallocate [英] fallocate vs posix_fallocate

问题描述

我正在讨论在posix_fallocate和fallocate之间使用哪个函数. posix_fallocate立即写入文件(将字符初始化为NULL).但是，fallocate不会更改文件大小(使用FALLOC_FL_KEEP_SIZE标志时).根据我的实验，看来fallocate不会向文件写入NULL或零个字符.

I am debating which function to use between posix_fallocate and fallocate. posix_fallocate writes a file right away (initializes the characters to NULL). However, fallocate does not change the file size (when using FALLOC_FL_KEEP_SIZE flag). Based on my experimentation, it seems that fallocate does not write NULL or zero characters to the file.

有人可以根据您的经验发表评论吗?谢谢您的宝贵时间.

Can someone please comment based on your experience? Thanks for your time.

推荐答案

通常情况下，文件占用的存储空间大于显示的长度是不常见的，因此，除非您有充分的理由这样做(例如，您想使用文件长度以跟踪下载的长度(以恢复下载为目的)，最好使用默认的fallocate(2)行为. (没有FALLOC_FL_KEEP_SIZE).这与posix_fallocate(3)的语义相同.

Having files that take up more storage space than their displayed length is not usual, so unless you have a good reason for doing that (e.g. you want to use the file length to keep track of how far a download got, for the purpose of resuming it), best to use the default fallocate(2) behaviour. (without FALLOC_FL_KEEP_SIZE). This is the same semantics as posix_fallocate(3).

fallocate(2)的手册页甚至说，它的默认行为(无标志)旨在作为实现posix_fallocate(3)的最佳方式，并指出这是一种可移植的分配空间的方式.

The man page for fallocate(2) even says that its default behaviour (no flags) is intended as an optimal way of implementing posix_fallocate(3), and points to that as a portable way to allocate space.

最初的问题是关于将零写入文件的一些说明.除元数据外，这些调用均不写入任何内容.如果您从已预分配但尚未写入的空间中读取数据，则会得到零(以前没有该磁盘空间中的任何内容，那将是一个很大的安全漏洞).您只能读取文件的末尾(长度，由fallocate，ftruncate或其他各种方式设置)，因此，如果您的文件长度为零，并且使用FALLOC_FL_KEEP_SIZE进行fallocate，那么您将无法读取任何内容.与预分配无关，只是文件大小的语义.

The original question says something about writing zeros to the file. None of these calls write anything but metadata. If you read from space that's been preallocated but not yet written, you'll get zeros (not whatever was in that disk space previously, that would be a big security hole). You can only read up to the end of a file (the length, set by fallocate, ftruncate, or various other ways), so if you have a zero-length file and fallocate with FALLOC_FL_KEEP_SIZE, then you can't read anything. Nothing to do with preallocation, just file size semantics.

因此，如果您对POSIX语义满意，请使用它，因为它更易于移植.每个GNU/Linux系统都将支持posix_fallocate(3)，但其他一些系统也将支持.

So if you're fine with the POSIX semantics, use it, because it's more portable. Every GNU/Linux system will support posix_fallocate(3), but so will some other systems.

但是，由于POSIX语义的原因，它并不是那么简单.如果在不支持预分配的文件系统上使用它，它仍然会成功，但是可以通过回退到在文件的每个块中实际写入零来实现.

However, thanks to POSIX semantics, it's not that simple. If you use it on a filesystem that doesn't support preallocation, it will still succeed, but do so by falling back to actually writing a zero in every block of the file.

测试程序:

#include <fcntl.h>
int main() {
    int fd = open("foo", O_RDWR|O_CREAT, 0666);
    if (fd < 0) return 1;
    return posix_fallocate(fd, 0, 400000);
}

在XFS上

$ strace ~/src/c/falloc
...
open("foo", O_RDWR|O_CREAT, 0666) = 3
fallocate(3, 0, 0, 400000)              = 0
exit_group(0)                           = ?

在fat32闪存驱动器上:

on a fat32 flash drive:

open("foo", O_RDWR|O_CREAT, 0666) = 3
fallocate(3, 0, 0, 400000)              = -1 EOPNOTSUPP (Operation not supported)
fstat(3, {st_mode=S_IFREG|0755, st_size=400000, ...}) = 0
fstatfs(3, {f_type="MSDOS_SUPER_MAGIC", f_bsize=65536, f_blocks=122113, f_bfree=38274, f_bavail=38274, f_files=0, f_ffree=0, f_fsid={2145, 0}, f_namelen=1530, f_frsize=65536}) = 0
pread(3, "\0", 1, 6783)                 = 1
pwrite(3, "\0", 1, 6783)                = 1
pread(3, "\0", 1, 72319)                = 1
pwrite(3, "\0", 1, 72319)               = 1
pread(3, "\0", 1, 137855)               = 1
pwrite(3, "\0", 1, 137855)              = 1
pread(3, "\0", 1, 203391)               = 1
pwrite(3, "\0", 1, 203391)              = 1
pread(3, "\0", 1, 268927)               = 1
pwrite(3, "\0", 1, 268927)              = 1
pread(3, "\0", 1, 334463)               = 1
pwrite(3, "\0", 1, 334463)              = 1
pread(3, "\0", 1, 399999)               = 1
pwrite(3, "\0", 1, 399999)              = 1
exit_group(0)                           = ?

如果文件还没有那么长，它确实避免了读取，但是写每个块仍然很糟糕.

It does avoid the reads if the file wasn't yet that long, but writing every block is still horrible.

如果您想要简单的东西，我仍然只用posix_fallocate.有一个FreeBSD手册页，它由POSIX指定，因此每个POSIX兼容的系统都提供它.一个缺点是在不支持预分配的文件系统上使用glibc会使它感到可怕.例如，请参见 https://plus.google.com/+AaronSeigo/posts/FGtXM13QuhQ.对于使用大型文件(例如torrent)的程序，这可能真的很糟糕.

If you want something simple, I'd still just go with posix_fallocate. There's a FreeBSD man page for it, and it's specified by POSIX, so every POSIX-compliant system provides it. The one drawback is that it will be horrible with glibc on a filesystem that doesn't support preallocation. See for example https://plus.google.com/+AaronSeigo/posts/FGtXM13QuhQ. For a program that works with large files, (e.g. torrents), this could be really bad.

您可以感谢POSIX语义要求glibc这样做，因为它没有为文件系统不支持预分配"定义错误代码. http://pubs.opengroup.org/onlinepubs/009695399/functions/posix_fallocate. html .它还保证，如果调用成功，则由于磁盘空间不足，随后对分配的区域的写操作不会失败.因此，posix设计没有提供一种方法来处理调用者关心效率/性能/碎片而不是磁盘空间保证的情况.这迫使POSIX实现执行读写循环，而不是将其作为需要磁盘空间保证的调用者的选择.谢谢POSIX ...

You can thank POSIX semantics for requiring glibc to do this, as it doesn't define an error code for "the filesystem doesn't support preallocation". http://pubs.opengroup.org/onlinepubs/009695399/functions/posix_fallocate.html. It also guarantees that if the call succeeds, subsequent writes into the allocated region won't fail due to lack of disk space. So the posix design doesn't provide a way to handle the case where the caller cares about efficiency / performance / fragmentation, rather than disk space guarantees. This forces the POSIX implementation to do the read-write loop, rather than leaving that as an option for callers that need a disk-space guarantee. Thanks POSIX...

我不知道当文件系统不支持预分配时，posix_fallocate的非GNU实现是否同样会回落到极慢的读写行为. (FreeBSD，Solaris?).显然，OS X(Darwin)不会实现posix_fallocate，除非它是最新的.

I don't know whether non-GNU implementations of posix_fallocate similarly fall back to extremely slow read-write behaviour when the filesystem doesn't support preallocation. (FreeBSD, Solaris?). Apparently OS X (Darwin) doesn't implement posix_fallocate, unless it's very recent.

如果您希望在许多平台上支持预分配，但是如果OS可以尝试进行预分配，而又不至于先读后写，那么您必须使用可用的特定于平台的方法.例如退房 https://github.com/arvidn/libtorrent/blob/master/src/file.cpp

If you're looking to support preallocation across a lot of platforms, but without falling back to read-then-write if the OS has a way to just attempt preallocation, you have to use whatever platform-specific method is available. e.g. check out https://github.com/arvidn/libtorrent/blob/master/src/file.cpp

搜索file :: set_size.它有几个ifdeffed块，具体取决于编译目标支持的对象，从Windows代码开始加载DLL并在其中进行填充，然后依次是fcntl F_PREALLOCATE或fcntl F_ALLOCSP64，然后是Linux fallocate(2)，然后回退到使用posix_fallocate.另外，在OS X Darwin上找到了以下2007年列表帖子: http: //lists.apple.com/archives/darwin-dev/2007/Dec/msg00040.html

search for file::set_size. It has several ifdeffed blocks depending on what the compile target supports, starting with windows code to load DLLs and do stuff there, then fcntl F_PREALLOCATE, or fcntl F_ALLOCSP64, then Linux fallocate(2), then falls back to using posix_fallocate. Also, found this 2007 list post for OS X Darwin: http://lists.apple.com/archives/darwin-dev/2007/Dec/msg00040.html

这篇关于fallocate vs posix_fallocate的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！