在bash中进行并行处理? [英] Doing parallel processing in bash?

问题描述

我有成千上万个png文件，我想使用pngcrush将其缩小.我有一个简单的find .. -exec作业，但是它是连续的.我的机器上有很多资源，我会并行进行.

I've thousands of png files which I like to make smaller with pngcrush. I've a simple find .. -exec job, but it's sequential. My machine has quite some resources and I'd make this in parallel.

每个png上要执行的操作是:

The operation to be performed on every png is:

pngcrush input output && mv output input

理想情况下，我可以指定并行操作的最大数量.

Ideally I can specify the maximum number of parallel operations.

有没有一种方法可以使用bash和/或其他shell助手来做到这一点?我是Ubuntu或Debian.

Is there a way to do this with bash and/or other shell helpers? I'm Ubuntu or Debian.

推荐答案

您可以使用xargs并行运行多个进程:

You can use xargs to run multiple processes in parallel:

find /path -print0 | xargs -0 -n 1 -P <nr_procs> sh -c 'pngcrush $1 temp.$$ && mv temp.$$ $1' sh

xargs将读取find生成的文件列表(以0个字符(-0分隔))，并一次使用一个参数(-n 1)运行提供的命令(sh -c '...' sh). xargs将并行运行<nr_procs>(-P <nr_procs>).

xargs will read the list of files produced by find (separated by 0 characters (-0)) and run the provided command (sh -c '...' sh) with one parameter at a time (-n 1). xargs will run <nr_procs> (-P <nr_procs>) in parallel.

这篇关于在bash中进行并行处理?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！