NTFS 性能和大量文件和目录 [英] NTFS performance and large volumes of files and directories

查看:28
本文介绍了NTFS 性能和大量文件和目录的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

使用 NTFS 的 Windows 如何处理大量文件和目录?

How does Windows with NTFS perform with large volumes of files and directories?

在遇到性能问题或其他问题之前,是否有关于文件或目录限制的指导?

Is there any guidance around limits of files or directories you can place in a single directory before you run into performance problems or other issues?

例如有一个里面有 100,000 个文件夹的文件夹是一件好事吗?

E.g. is having a folder with 100,000 folders inside of it an OK thing to do?

推荐答案

以下是来自我们的文件夹包含数千万个文件的环境的一些建议.

Here's some advice from someone with an environment where we have folders containing tens of millions of files.

  1. 文件夹将索引信息(链接到子文件和子文件夹)存储在索引文件中.当你有很多孩子时,这个文件会变得非常大.请注意,它不区分作为文件夹的子项和作为文件的子项.唯一的区别实际上是该子项的内容是子项的文件夹索引或子项的文件数据.注意:我对此进行了一些简化,但这说明了要点.
  2. 索引文件将变得碎片化.当它变得过于分散时,您将无法将文件添加到该文件夹​​中.这是因为允许的片段数量是有限制的.这是设计使然.我已经在支持事件电话中与 Microsoft 确认了这一点.因此,尽管一个文件夹中可以包含的文件数量的理论限制是数十亿,但当您开始处理数千万个文件时,祝您好运,因为您将首先达到碎片限制.
  3. 不过,也并非全是坏事.您可以使用该工具:contig.exe 对该索引进行碎片整理.它不会减少索引的大小(对于数千万个文件最多可以达到几个 Gig),但您可以减少片段的数量.注意:磁盘碎片整理工具不会对文件夹的索引进行碎片整理.它将对文件数据进行碎片整理.只有 contig.exe 工具会对索引进行碎片整理.仅供参考:您还可以使用它来对单个文件的数据进行碎片整理.
  4. 如果您进行碎片整理,请不要等到达到最大碎片数量限制.我有一个文件夹无法进行碎片整理,因为我一直等到为时已晚.我的下一个测试是尝试将一些文件从该文件夹中移到另一个文件夹中,看看我是否可以对其进行碎片整理.如果失败,那么我要做的是 1) 创建一个新文件夹.2) 将一批文件移动到新文件夹中.3) 对新文件夹进行碎片整理.重复#2 &#3 直到完成,然后 4) 删除旧文件夹并重命名新文件夹以匹配旧文件夹.
  1. A folder stores the index information (links to child files & child folder) in an index file. This file will get very large when you have a lot of children. Note that it doesn't distinguish between a child that's a folder and a child that's a file. The only difference really is the content of that child is either the child's folder index or the child's file data. Note: I am simplifying this somewhat but this gets the point across.
  2. The index file will get fragmented. When it gets too fragmented, you will be unable to add files to that folder. This is because there is a limit on the # of fragments that's allowed. It's by design. I've confirmed it with Microsoft in a support incident call. So although the theoretical limit to the number of files that you can have in a folder is several billions, good luck when you start hitting tens of million of files as you will hit the fragmentation limitation first.
  3. It's not all bad however. You can use the tool: contig.exe to defragment this index. It will not reduce the size of the index (which can reach up to several Gigs for tens of million of files) but you can reduce the # of fragments. Note: The Disk Defragment tool will NOT defrag the folder's index. It will defrag file data. Only the contig.exe tool will defrag the index. FYI: You can also use that to defrag an individual file's data.
  4. If you DO defrag, don't wait until you hit the max # of fragment limit. I have a folder where I cannot defrag because I've waited until it's too late. My next test is to try to move some files out of that folder into another folder to see if I could defrag it then. If this fails, then what I would have to do is 1) create a new folder. 2) move a batch of files to the new folder. 3) defrag the new folder. repeat #2 & #3 until this is done and then 4) remove the old folder and rename the new folder to match the old.

更直接地回答您的问题:如果您查看 10 万个条目,不用担心.去自首吧.如果您要查看数以千万计的条目,则可以:

To answer your question more directly: If you're looking at 100K entries, no worries. Go knock yourself out. If you're looking at tens of millions of entries, then either:

a) 计划将它们细分为子文件夹(例如,假设您有 1 亿个文件.最好将它们存储在 1000 个文件夹中,这样每个文件夹中只有 100,000 个文件,而不是将它们存储在一个大的文件夹中)文件夹.这将创建 1000 个文件夹索引,而不是一个更可能达到最大片段数限制或

a) Make plans to sub-divide them into sub-folders (e.g., lets say you have 100M files. It's better to store them in 1000 folders so that you only have 100,000 files per folder than to store them into 1 big folder. This will create 1000 folder indices instead of a single big one that's more likely to hit the max # of fragments limit or

b) 计划定期运行 contig.exe 以保持大文件夹的索引碎片整理.

b) Make plans to run contig.exe on a regular basis to keep your big folder's index defragmented.

仅当您感到无聊时才阅读以下内容.

实际的限制不在于片段的数量,而在于存储指向片段的指针的数据段的记录数.

The actual limit isn't on the # of fragment, but on the number of records of the data segment that stores the pointers to the fragment.

所以您拥有的是一个数据段,用于存储指向目录数据片段的指针.目录数据存储关于子目录的信息目录应该存储的子文件.实际上,目录并不存储"任何东西.由于存储介质本身是线性的,因此它只是一个跟踪和呈现功能,向用户呈现层次结构的错觉.

So what you have is a data segment that stores pointers to the fragments of the directory data. The directory data stores information about the sub-directories & sub-files that the directory supposedly stored. Actually, a directory doesn't "store" anything. It's just a tracking and presentation feature that presents the illusion of hierarchy to the user since the storage medium itself is linear.

这篇关于NTFS 性能和大量文件和目录的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆