如何让子进程调用呼叫/ POPEN继承的环境变量 [英] how to make subprocess called with call/Popen inherit environment variables

问题描述

首先，道歉，我肯定会很明显的是我的bash和壳和子流程的基本理解。

First off, apologies for what I'm sure will be obvious is my rudimentary understanding of bash and shells and subprocesses.

我试图使用Python自动到一个名为Freesurfer程序调用（实际上，我打电话调用子程序侦察 - 所有。）

I am trying to use Python to automate calls to a program called Freesurfer (actually, the subprogram I'm calling is called recon-all.)

如果我是在命令行直接这样做，我会的源的脚本称为mySetUpFreeSurfer.sh，什么也不做，但设置三个环境变量，然后在源另一个脚本，FreeSurferEnv.sh。 FreesurferEnv.sh似乎没有给我做任何事情，但设置了很多环境变量和echo一些东西到终端，但它比其他的bash脚本比较复杂，所以我不知道这一点。

If I were doing this directly at the command line, I'd "source" a script called mySetUpFreeSurfer.sh that does nothing but set three environment variables, and then "source" another script, FreeSurferEnv.sh. FreesurferEnv.sh doesn't seem to me to do anything but set a lot of environment variables and echo some stuff to the terminal, but it's more complicated than the other bash script, so I'm not sure of that.

下面是我现在所拥有的：

Here is what I have right now:

from subprocess import Popen, PIPE, call, check_output
import os

root = "/media/foo/"

#I got this function from another Stack Overflow question.

def source(script, update=1):
    pipe = Popen(". %s; env" % script, stdout=PIPE, shell=True)
    data = pipe.communicate()[0]
    env = dict((line.split("=", 1) for line in data.splitlines()))
    if update:
        os.environ.update(env)
    return env

source('~/scripts/mySetUpFreeSurfer.sh')
source('/usr/local/freesurfer/FreeSurferEnv.sh')

for sub_dir in os.listdir(root):
    sub = "s" + sub_dir[0:4]
    anat_dir = os.path.join(root, sub_dir, "anatomical")
    for directory in os.listdir(anat_dir):
        time_dir = os.path.join(anat_dir, directory)
        for d in os.listdir(time_dir):
            dicoms_dir = os.path.join(time_dir, d, 'dicoms')
            dicom_list = os.listdir(dicoms_dir)
            dicom = dicom_list[0]
            path = os.path.join(dicoms_dir, dicom)
            cmd1 = "recon-all -i " + path + " -subjid " + sub
            check_output(cmd1, shell=True)
            call(cmd1, shell=True)
            cmd2 = "recon-all -all -subjid " + sub,
            call(cmd2, shell=True)

这是失败的：

Traceback (most recent call last):
     File "/home/katie/scripts/autoReconSO.py", line 28, in <module>
        check_output(cmd1, shell=True)
      File "/usr/lib/python2.7/subprocess.py", line 544, in check_output
        raise CalledProcessError(retcode, cmd, output=output)
    CalledProcessError: Command 'recon-all -i /media/foo/bar -subjid s1001' returned non-zero exit status 127

我也许明白这是为什么。我的电话，后来在脚本中养不从由源（）函数调用的募集进程继承的环境变量新的子进程。我做了一些事情，试图确认我的理解。其中一个例子 - 我把这些行：

I maybe understand why this is. My "calls" later in the script are raising new subprocesses that do not inherit environment variables from the processes that are raised by invocation of the source() function. I have done a number of things to try to confirm my understanding. One example -- I put these lines:

mkdir ~/testFreeSurferEnv
export TEST_ENV_VAR=~/testFreeSurferEnv

在FreeSurferEnv.sh脚本。该目录被制作得很好，但在Python脚本这样的：

in the FreeSurferEnv.sh script. The directory gets made just fine, but in the Python script this:

cmd = 'mkdir $TEST_ENV_VAR/test'
check_output(cmd, shell=True)

失败是这样的：

File "/usr/lib/python2.7/subprocess.py", line 544, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command 'mkdir $TEST_ENV_VAR/test' returned non-zero exit status 1

问：

我怎样才能运行侦察所有继承它需要的环境变量的子进程？或者，我该怎么做的一切，我需要做的 - 运行脚本来设置环境变量，并调用侦察一切，在同一个进程？或者我应该解决这个问题的另一种方式？或者我可能误解了问题？

How can I make the subprocess that runs "recon-all" inherit the environment variables it needs? Or how can I do everything I need to do -- run the scripts to set the environment variables, and call recon-all, in the same process? Or should I approach the problem another way? Or do I likely misunderstand the problem?

推荐答案

关于

的 如果我直接在命令行这样做，我会的源的脚本称为mySetUpFreeSurfer.sh，什么也不做，但设置三个环境变量，然后在源另一脚本，FreeSurferEnv.sh。 的

我认为你会使用Python自动写作过程中会更好
一个shell脚本 newscript.sh ，然后调用此脚本的有一个的通话
subprocess.check_output 的（而不是多次调用 POPEN ， check_output
呼叫等）：

I think you would be better off using Python to automate the process of writing a shell script newscript.sh, and then calling this script with one call subprocess.check_output (instead of many calls to Popen, check_output, call, etc.):

newscript.sh：

#!/bin/bash
source ~/scripts/mySetUpFreeSurfer.sh
source /usr/local/freesurfer/FreeSurferEnv.sh
recon-all -i /media/foo/bar -subjid s1001
...

，然后调用

subprocess.check_output(['newscript.sh'])

---

import subprocess
import tempfile
import os
import stat


with tempfile.NamedTemporaryFile(mode='w', delete=False) as f:
    f.write('''\
#!/bin/bash
source ~/scripts/mySetUpFreeSurfer.sh
source /usr/local/freesurfer/FreeSurferEnv.sh
''')
    root = "/media/foo/"
    for sub_dir in os.listdir(root):
        sub = "s" + sub_dir[0:4]
        anat_dir = os.path.join(root, sub_dir, "anatomical")
        for directory in os.listdir(anat_dir):
            time_dir = os.path.join(anat_dir, directory)
            for d in os.listdir(time_dir):
                dicoms_dir = os.path.join(time_dir, d, 'dicoms')
                dicom_list = os.listdir(dicoms_dir)
                dicom = dicom_list[0]
                path = os.path.join(dicoms_dir, dicom)
                cmd1 = "recon-all -i {}  -subjid {}\n".format(path, sub)
                f.write(cmd1)
                cmd2 = "recon-all -all -subjid {}\n".format(sub)
                f.write(cmd2)

filename = f.name
os.chmod(filename, stat.S_IRUSR | stat.S_IXUSR)
subprocess.call([filename])
os.unlink(filename)

---

顺便说一句，

---

By the way,

def source(script, update=1):
    pipe = Popen(". %s; env" % script, stdout=PIPE, shell=True)
    data = pipe.communicate()[0]
    env = dict((line.split("=", 1) for line in data.splitlines()))
    if update:
        os.environ.update(env)
    return env

坏。例如，如果剧本包含像

VAR=`ls -1`
export VAR

然后

. script; env

可以返回输出像

VAR=file1
file2
file3

这将导致源（脚本）养 ValueError错误：

env = dict((line.split("=", 1) for line in data.splitlines()))
ValueError: dictionary update sequence element #21 has length 1; 2 is required

---

有是修复源的方式：有用零字节，而不是模棱两可的ENV 单独的环境变量新行：

---

There is a way to fix source: have env separate environment variables with a zero byte instead of the ambiguous newline:

def source(script, update=True):
    """
    http://pythonwise.blogspot.fr/2010/04/sourcing-shell-script.html (Miki Tebeka)
    http://stackoverflow.com/questions/3503719/#comment28061110_3505826 (ahal)
    """
    import subprocess
    import os
    proc = subprocess.Popen(
        ['bash', '-c', 'set -a && source {} && env -0'.format(script)], 
        stdout=subprocess.PIPE, shell=False)
    output, err = proc.communicate()
    output = output.decode('utf8')
    env = dict((line.split("=", 1) for line in output.split('\x00') if line))
    if update:
        os.environ.update(env)
    return env

可固定或没有，但是，你仍然可能会更好过构建
集团shell脚本（如上图所示），比你会解析 ENV 和
通过 ENV 类型的字典到子通话。

这篇关于如何让子进程调用呼叫/ POPEN继承的环境变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！