更有效的查找方式tar 数百万个文件 [英] More efficient way to find & tar millions of files

问题描述

我已经在我的服务器上通过命令行提示符运行了两天了:

I've got a job running on my server at the command line prompt for a two days now:

find data/ -name filepattern-*2009* -exec tar uf 2009.tar {} ;

它需要永远，然后是一些.是的，目标目录中有数百万个文件.(每个文件在一个很好的哈希目录结构中只有区区 8 个字节.)但是只是运行......

It is taking forever, and then some. Yes, there are millions of files in the target directory. (Each file is a measly 8 bytes in a well hashed directory structure.) But just running...

find data/ -name filepattern-*2009* -print > filesOfInterest.txt

...只需要两个小时左右.按照我的工作运行速度，它不会在几个 周 内完成.这似乎不合理.有没有更有效的方法?也许使用更复杂的 bash 脚本?

...takes only two hours or so. At the rate my job is running, it won't be finished for a couple of weeks.. That seems unreasonable. Is there a more efficient to do this? Maybe with a more complicated bash script?

次要问题是为什么我目前的方法这么慢?"

推荐答案

如果您已经执行了创建文件列表的第二条命令，只需使用 -T 选项告诉 tar 读取文件保存的文件列表中的名称.运行 1 个 tar 命令与 N 个 tar 命令会好很多.

If you already did the second command that created the file list, just use the -T option to tell tar to read the files names from that saved file list. Running 1 tar command vs N tar commands will be a lot better.

这篇关于更有效的查找方式tar 数百万个文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！