awk命令在snakemake --use-singularity中失败 [英] awk command fails in snakemake --use-singularity

问题描述

我正在尝试将Snakemake与奇点性结合使用，并且我注意到，使用奇点性时，简单的 awk 命令不再起作用.最后一行中的 $ 1 被bash代替，而不是被 awk 用作第一个字段.

I am trying to combine Snakemake with Singularity, and I noticed that a simple awk command no longer works when using singularity. The $1 in the last line gets replaced by bash instead of being used as the first field by awk.

这是一个最小的工作示例( Snakefile ):

Here is a minimal working example (Snakefile):

singularity: "docker://debian:stretch"
rule all:
    input: "test.txt"
rule test:
    output: 
        "test.txt"
    shell:
        "cat /etc/passwd | awk -F':' '{{print $1}}' > {output}"

当我运行 snakemake 而不使用奇异性时，输出 test.txt 看起来像预期的那样(仅包含用户名).当我运行 snakemake --use-singularity 时，文件包含整行，例如 root:x:0:0:root:/root:/bin/bash .

When I run snakemake without singularity, the output test.txt looks as expected (containing only user names). When I run snakemake --use-singularity, the file contains whole lines, e.g. root:x:0:0:root:/root:/bin/bash.

这是Snakemake的日志:

This is the log from Snakemake:

$ snakemake --use-singularity --printshellcmd                                                                                                               
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1
Rules claiming more threads will be scaled down.
Job counts:
        count   jobs
        1       all
        1       test
        2

rule test:
    output: test.txt
    jobid: 1

cat /etc/passwd | awk -F':' '{print $1}' > test.txt
Activating singularity image /scratch/test/.snakemake/singularity/fa9c8c7220ff16e314142a5d78ad6cff.simg
Finished job 1.
1 of 2 steps (50%) done

localrule all:
    input: test.txt
    jobid: 0

Finished job 0.
2 of 2 steps (100%) done

推荐答案

我遇到了类似的问题，经过大量的试验和错误终于解决了它.当前(2018年11月，对于Snakemake 5.3)，此文件尚未记录，因此我认为将其放在此处以供将来参考以帮助其他人是很好的...

I had a similar issue and after lots of trial and error finally solved it. Currently (November 2018, for Snakemake 5.3), this is somewhat undocumented, so I thought it is good to put it here for future reference to help others...

以上所有示例在bash -c上均错误地使用了双引号，而Snakemake并不是这样构造双引号的.相反，Snakemake使用 bash -c'modified_command'调用奇点，因此使用单引号.首先，这改变了命令中特殊字符的处理方式.其次，到目前为止，Snakemake会将实际命令中的所有单引号替换为转义版本\'.但是，仅当与奇点一起使用时才适用.

All examples above incorrectly use the double quotation mark with bash -c, which is NOT how Snakemake constructs it. Instead, Snakemake calls Singularity with bash -c ' modified_command ', so single quotes. First, this changes how special characters are treated within the command. Second, as of now, Snakemake replaces all single quotation marks within the actual command with the escaped version \'. However, this applies ONLY when used with Singularity.

因此，如果您的命令包含单引号，则使用--use-singularity提交时或在正常模式下运行时，东西都会中断.我知道，在两种情况下都可行的唯一可行的解​​决方案是:

Therefore, if your command contains single quotes, things break either when submitting with --use-singularity or when running in normal mode. The only working solution that I am aware of that works in both cases is the following:

shell: """awk "{{OFS="\\t"}};{{print \$2}}" {input}"""

因此，以下规则适用:

1. 请勿在命令中使用单引号，否则会将其替换，否则会导致错误.
2. 转义某些字符，例如\ t减\\ t，$减\ $和{减{{..
3. 使用三引号引起来的命令行调用.

我希望这会有所帮助，一旦实现更新，我将更新此帖子.

I hope this helps, I will update this post once there are implementation updates.

这篇关于awk命令在snakemake --use-singularity中失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！