如何使用xargs在命令匹配中为每个匹配项运行一个函数? [英] How can I use xargs to run a function in a command substitution for each match?

问题描述

在编写用于替换字符串的Bash函数时，我在使用xargs时遇到了奇怪的行为.目前，这实际上使我发疯，因为我无法使它正常工作.幸运的是，我能够将其固定为以下简单示例:

While writing Bash functions for string replacements I have encountered a strange behaviour when using xargs. This is actually driving me mad currently as I cannot get it to work. Fortunately I have been able to nail it down to the following simple example:

定义一个简单函数，将给定参数的每个字符加倍:

Define a simple function which doubles every character of the given parameter:

function subs { echo $1 | sed -E "s/(.)/\1\1/g"; }

调用函数:

echo $(subs "ABC")

如预期的那样，输出为:

As expected the output is:

AABBCC

现在使用xargs调用函数:

Now call the function using xargs:

echo "ABC" | xargs -I % echo $(subs "%")

令人惊讶的是，现在的结果是:

Surprisingly the result now is:

ABCABC

函数内部的sed命令似乎现在将整个字符串视为一个字符.为什么会发生这种情况，如何预防呢?

It seems as if the sed command inside the function treats the whole string now as a single character. Why does this happen and how can it be prevented?

您可能会问，为什么我完全使用xargs.当然，这是一个简化的示例，实际用例要复杂得多.

You might ask, why I use xargs at all. Of course, this is a simplified example and the actual use case is much more complex.

在原始用例中，我有一个程序会产生大量输出.我通过几次抓取来传递输出，以得到感兴趣的行.然后，我通过管道将sed提取出来，以便从这些行中提取所需的数据.由于我需要对数据进行的某些转换过于复杂，无法单独使用正则表达式进行处理，因此我想对它们使用一个函数.因此，我最初的想法是简单地将其放入函数中，但是我无法使其正常工作并最终得到xargs解决方案.我最初的想法是这样的:

In the original use case, I have a program which produces lots of output. I pipe the output through several greps to get the lines of interest. Afterwards, I pipe the lines to sed to extract the data I need from the lines. Because some transformations I need to do on the data are too complex to do with regular expressions alone, I'd like to use a function for these. So, my original idea was to simply pipe into the function but I couldn't get that to work and end up with the xargs solution. My original idea was something like this:

command | grep ... | grep ... | grep ... | sed ... | subs

顺便说一句:我不是从命令行而是从脚本中执行此操作.该功能是在使用该脚本的相同脚本中定义的.

BTW: I do not do this from the command line but from within a script. The function is defined in the very same script in which it is used.

我使用的是Bash 3.2(默认为Mac OS X)，所以花哨的Bash 4.x内容对我没有帮助，对不起.

I'm using Bash 3.2 (Mac OS X default), so fancy Bash 4.x stuff won't help me, sorry.

我会对所有可能使您对该主题有所了解的事情感到高兴.

I'll be happy about everything which might shed some light on this topic.

最诚挚的问候

坦率

推荐答案

如果您确实需要这样做(您可能不需要，但是如果没有更具代表性的示例，我们将无济于事)，更好的做法可能是这样的:

If you really need to do this (and you probably don't, but we can't help without a more representative sample), a better-practice approach might look like:

subs() { sed -E "s/(.)/\1\1/g" <<<"$1"; }
export -f subs

echo "ABC" | xargs bash -c 'for arg; do subs "$arg"; done' _

• 使用 sub"$(subs" $ arg)" 而不是仅仅 subs"$ arg" 只会增加错误(请考虑以下情况之一会发生什么情况:您的参数为 -n -并假设相对温和的 echo ；即使没有 -e 参数，它们也可以使用反斜杠，并且做所有其他令人惊讶的事情).您可以 在上面执行此操作，但是它会使您的程序变慢，并使它更容易出现令人惊讶的行为.没有意义.
• 运行 export -f subs 会将函数导出到环境，因此它可以由作为子进程调用的bash的其他实例运行( xargs 调用的所有程序都是在您的外壳程序之外，因此他们看不到外壳程序本地变量或函数.

• 没有 -I 的情况-也就是说，在其默认操作模式下- xargs 将参数附加到给出的命令的末尾.这样可以提供一种更有效的使用模式，该模式无需将输入的尽可能多的参数传递给尽可能短的子进程，而不必每输入一行调用一个命令.

  • The use of echo "$(subs "$arg")" instead of just subs "$arg" adds nothing but bugs (consider what happens if one of your arguments is -n -- and that's assuming a relatively tame echo; they're allowed to consume backslashes even without a -e argument and to do all manner of other surprising things). You could do it above, but it slows your program down and makes it more prone to surprising behaviors; there's no point.
  • Running export -f subs export your function to the environment, so it can be run by other instances of bash invoked as child processes (all programs invoked by xargs are outside your shell, so they can't see shell-local variables or functions).

  • Without -I -- which is to say, in its default mode of operation -- xargs appends arguments to the end of the command it's given. This permits a much more efficient usage mode, where instead of invoking one command per line of input, it passes as many arguments as possible to the shortest possible number of subprocesses.

    这还避免了将 xargs -I 与 bash -c'...'或 sh -c'结合使用时可能发生的主要安全错误....'.(如果您曾经使用 -I％sh -c'...％...'，则文件名将成为代码的一部分，并且可以在系统的注入攻击中使用)

    This also avoids major security bugs that can happen when using xargs -I in conjunction with bash -c '...' or sh -c '...'. (If you ever use -I% sh -c '...%...', then your filenames become part of your code, and are able to be used in injection attacks on your system).

    这篇关于如何使用xargs在命令匹配中为每个匹配项运行一个函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！