Snakemake Temp() [英] Snakemake temp()

问题描述

我已将Snakemake连接到S3帐户，并且我希望在处理我们的管道后删除某些temp()文件。

我有一条规则将某些文件指定为temp()。下面是一个示例：

 #Split rep element mapped bam file into subfiles
rule split_rep_bam:
  input:
    'rep_element_pipeline/{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam'
  output:
    temp('rep_element_pipeline/AA.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/AC.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/AG.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/AN.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/AT.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/CA.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/CC.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/CG.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/CN.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/CT.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/GA.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/GC.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/GG.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/GN.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/GT.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/NA.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/NC.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/NG.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/NN.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/NT.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/TA.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/TC.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/TG.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/TN.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/TT.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp')
  conda:
    '../envs/rep_element.yaml'
  params:
    fp=full_path
  shell:
    'perl ../scripts/split_bam_to_subfiles.pl {input[0]} '
    '{params.fp}/rep_element_pipeline/'

我一直在这样运行Snakemake：

snakemake --default-remote-provider S3 --default-remote-prefix '$s3' --use-conda --cores 32 --rerun-incomplete --printshellcmds --delete-temp-output

其中我指定了--delete-temp-output选项。

运行此规则时，Snakemake似乎正在删除这些文件。但是，这些文件仍然保留在S3中。有人知道为什么S3中没有删除这些文件吗？

Building DAG of jobs...
Deleting 20211222-rp030/rep_element_pipeline/AA.APP-01_rep_1_Ribo_R1.fastq.gz.mapped_vs_bt2_hg38.sam.tmp
Deleting 20211222-rp030/rep_element_pipeline/AC.APP-01_rep_1_Ribo_R1.fastq.gz.mapped_vs_bt2_hg38.sam.tmp

推荐答案

这个问题与another question非常相似，核心问题是：

Remote()包装与temp()和Protected()包装互斥。

来源：docs。

一种解决方案是使用local函数显式标记文件，因此如下所示：

rule some_rule:
     output: temp(local('test.txt'))
     run: # ...

这篇关于Snakemake Temp()的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！