许多图像在基于Linux的应用程序的同一文件夹中 [英] Many images in the same folder in a Linux based application

问题描述

我有一个基于Linux的应用程序（Drupal - PHP + Apache + MySQL），可以快速扩展，将数千张图片保存在同一个文件夹中。我现在可能接近2,000。在单个文件夹中有这么多图像有什么缺点吗？可能会对性能造成影响？

我不打算浏览该文件夹中的文件，而服务器只是在请求URL时提供图像，而是我想知道我将来是否会遇到问题（应用程序正在扩展，按照图片的尺寸进行缩放，最多可以达到20,000,300个图像...）。也许我应该制定一个将这个怪异的图像文件夹分解为子文件夹的策略，例如使用子文件夹的用户名，或图片上传的年份。

简而言之，问题是：在Linux服务器的同一个文件夹中有数千张图像是不好的（不知道我的应用程序使用了哪种风格，它实际上是在共享托管环境中）？我应该做些什么来避免这种做法，将这些项目分解成子文件夹？有没有关于每个文件夹的文件数量或每个文件夹的最大文件数量的建议？

感谢您分享您的想法。

解决方案

它很大程度上取决于文件系统（还有mount选项）。

最近的文件系统（ ext4 ， btrfs ....）能够处理包含大量文件的巨大目录（所以可能会处理一个有五百万个文件的目录）。 p>

^{但是，在罕见的情况下，您将不会高兴 fsck 一个多TB的文件系统！}

然而，shell（和globbing函数 - 读取 glob（7）和 glob（3）等...）可能不满意一千个条目的目录。 （认为​​交互式shell中的自动完成可能需要扫描目录）。如果 ls 需要一分钟来回答数千行...，那么人类用户（例如某些系统管理员）可能会生气。

我建议每个目录不得超过几千个文件（包括子目录）。考虑组织您的图像，如 a0 / 001.png ... a0 / 999.png a1 / 001.png ... a1 / 999.png .... b9 / etc ...

如果你真的有很多文件，你可能想为他们使用不同的文件系统（例如 a0 / ... a9 / 在一个磁盘上， b0 / ... b9 / 另一个）....或使用 LVM ，或者 OpenStack Swift 对象存储或 Cinder 块存储等

另请参见 Nosql 数据库...

BTW，如果您的目标是扩展到PB级存储，事情将变得不那么容易....

您可能希望至少存储一些数据库中的文件名。

I have a Linux based application (Drupal - PHP + Apache + MySQL) that is quickly scaling to hold thousands of pictures in the same folder. I am probably close to 2,000 right now. Are there any drawbacks of having so many images in a single folder? Could it represent an impact in terms of performance?

I am not planning to browse files in that folder and the server is simply serving the images when their URL is requested, but I wonder if I will have problems in the future (the application is scaling, scaling in terms of number of pictures and it could go as far as 20,000, 30,000 images...). Maybe I should plan a strategy for splitting this monstrous images folder into subfolders, like using usernames for subfolders, or the year-month the picture was uploaded.

In a nutshell, the questions are: is it bad to have thousands of images in the same folder for a Linux server (not sure which flavor is being used for my app, it is actually in a shared hosting environment)? Should I do something to avoid this approach and split those items into subfolders? Are there any recommendations in terms of number of files per folder, or maximum number of files per folder?

Thank you for sharing your thoughts about this.

解决方案

It strongly depends on the file system (and also the mount options).

Recent file-systems (ext4, btrfs ....) are able to deal with huge directories containing a lot of files (so probably could deal with a directory with half a million files).

^{However, you won't be happy in the rare cases you need to fsck a multi-terabyte file system!}

However, the shell (and globbing functions - read glob(7) and glob(3) etc...) may be unhappy with a directory of many thousand entries. (consider that autocomplete in an interactive shell may need to scan the directory). And the human user (e.g. some sysadmin) might be mad if ls needs a minute to answer many thousand lines....

I would recommend having no more than a few thousand files (including sub-directories) per directory. Consider organizing your images like a0/001.png ... a0/999.png a1/001.png ... a1/999.png .... b9/ etc...

And if you have really a huge lot of files, you might want to use different file systems for them (e.g. a0/ ... a9/ on one disk and b0/ ... b9/ on another one) .... or use LVM, or perhaps OpenStack Swift object storage or Cinder block storage, etc.

Look also into Nosql databases...

BTW, if you aim to scale to petabyte storage, things will become less easy....

You probably want to store in some database at least the name of the files.

这篇关于许多图像在基于Linux的应用程序的同一文件夹中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！