如果Linux中单个目录下的文件太多，会发生什么情况？ [英] What happens if there are too many files under a single directory in Linux?

问题描述

如果单个目录中有100万个单独的文件（大多数是10万个），平坦地（其中没有其他目录和文件），在任何其他可能的方式中是否会有任何妥协的效率或缺点？ / p>

解决方案

ARG_MAX将会带来问题...例如，rm -rf *（在目录中）说太多的论据。如果这个目录对公众是可用的（可以通过ftp，或者通过ftp来访问，也可以通过ftp来访问）。 Web服务器），您可能会遇到其他问题。

对任何给定的文件系统的影响完全取决于该文件系统。这些文件的访问频率如何，文件系统是什么？请记住，根据您的设置，Linux（默认情况下）优先将最近访问的文件保存在内存中，同时将进程置于交换状态。这个目录是通过http服务的吗？ Google会看到并抓取它吗？如果是这样，您可能需要调整VFS缓存压力和swappiness。
$ b

编辑：

ARG_MAX是一个系统范围的限制，可以向程序的入口点显示多少个参数。因此，让我们拿rm，例如rm -rf * - shell将把'*'转换成空格分隔的文件列表，然后转换成rm的参数。

ls和其他几个工具也会发生同样的情况。例如，如果太多文件以'foo'开头，ls foo *可能会中断。

我建议（不管使用哪个fs）将其分解进入更小的目录块，仅仅因为这个原因。

If there are like 1,000,000 individual files (mostly 100k in size) in a single directory, flatly (no other directories and files in them), is there going to be any compromises in efficiency or disadvantages in any other possible ways?

解决方案

ARG_MAX is going to take issue with that... for instance, rm -rf * (while in the directory) is going to say "too many arguments". Utilities that want to do some kind of globbing (or a shell) will have some functionality break.

If that directory is available to the public (lets say via ftp, or web server) you may encounter additional problems.

The effect on any given file system depends entirely on that file system. How frequently are these files accessed, what is the file system? Remember, Linux (by default) prefers keeping recently accessed files in memory while putting processes into swap, depending on your settings. Is this directory served via http? Is Google going to see and crawl it? If so, you might need to adjust VFS cache pressure and swappiness.

Edit:

ARG_MAX is a system wide limit to how many arguments can be presented to a program's entry point. So, lets take 'rm', and the example "rm -rf *" - the shell is going to turn '*' into a space delimited list of files which in turn becomes the arguments to 'rm'.

The same thing is going to happen with ls, and several other tools. For instance, ls foo* might break if too many files start with 'foo'.

I'd advise (no matter what fs is in use) to break it up into smaller directory chunks, just for that reason alone.

这篇关于如果Linux中单个目录下的文件太多，会发生什么情况？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！