在集群中运行作业时强制加载R软件包 [英] Force load R packages while running the job in cluster

问题描述

当我以交互模式在HPC集群中运行作业时，我可以加载软件包，如果加载失败（不确定为什么某些软件包一开始无法加载），可以通过运行<$ c来加载它$ c>库（失败的程序包）多次，但当我执行 qsub my_rscript_job.pbs 时，程序包无法加载。

When I run a job in HPC cluster in interactive mode, I can load the packages and if it fails (not sure why some packages fail to load at first instance) to load, I can load it by running the library (failed package) multiple times, but when I do qsub my_rscript_job.pbs, the packages fail to load.

我的my_rscript_job.pbs脚本是：

my my_rscript_job.pbs script is:

#!/bin/bash 
#PBS -l walltime=100:00:00
#PBS -l ncpus=1,mem=100g

source ~/.bashrc

Rscript /dmf/mypath/map.r -t 100

我需要加载的软件包在map.r脚本中是

The packages I need to load in the map.r script are

library(biomaRt)
library(dplyr)
library(stringi)
library(GenomicFeatures)
library(Rsamtools)
library(foreach)
library(doMC)
library(doMC)

如果我以交互方式提交作业并将rscript直接提交到终端，我可以加载它，但是当我执行qsub时，我得到以下信息错误：

which I can load if I submit the job in interactive mode and submit the rscript directly to the terminal, but when I do qsub I get the following error:

Loading required package: methods
Warning messages:
1: package ‘biomaRt’ was built under R version 3.2.2 
2: In eval(quote({ : bytecode version mismatch; using eval
3: In .recacheSubclasses(def@className, def, doSubclasses, env) :
  undefined subclass "externalRefMethod" of class "expressionORfunction"; definition not updated
4: In .recacheSubclasses(def@className, def, doSubclasses, env) :
  undefined subclass "externalRefMethod" of class "functionORNULL"; definition not updated
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/dmf/bin/R/x86_64-redhat-linux-gnu-library/3.2/dplyr/libs/dplyr.so':
  /dmf/bin/R/x86_64-redhat-linux-gnu-library/3.2/dplyr/libs/dplyr.so: undefined symbol: Rf_installChar
In addition: Warning message:
package ‘dplyr’ was built under R version 3.2.2 
Error: package or namespace load failed for ‘dplyr’
Execution halted

有没有一种方法可以在将r作为qsub运行时强制加载软件包？

Is there a way to force load the packages while running r as qsub?

推荐答案

设置计时器以重新加载每个程序包，直到成功加载列表中的每个程序包为止。运行 qsub 选项时，有5秒的计时器强制加载程序包。

Setting timer to reload each package until each package in the list is successfully loaded. There is a timer of 5 second to force load the package when running the qsub option.

 myPackages <- c("biomaRt", "dplyr", "stringi","GenomicFeatures","Rsamtools","foreach","doMC")
    tryCount <- 0    

    while( !all(myPackages %in% (.packages())) ){

      try(require(biomaRt))
      try(require(dplyr))
      try(require(stringi))
      try(require(GenomicFeatures))
      try(require(Rsamtools))
      try(require(foreach))
      try(require(doMC))

      tryCount <- tryCount + 1

      if( !all(myPackages %in% (.packages()))  ){
        cat(paste0("Failure: ", tryCount, "\n"))
        cat("Failed to load: ")
        cat(myPackages[ !myPackages %in% (.packages()) ])
        cat("\n")
      } else {
        print(paste0("Success!"))
      }

      Sys.sleep(5)

    }

这篇关于在集群中运行作业时强制加载R软件包的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！