如何从python脚本成功调用gsutil rsync? [英] How do you successfully invoke gsutil rsync from a python script?

问题描述

我正在尝试执行以下行

gsutil -m rsync s3://input gs://output

在python中.在shell终端中运行此行时，它可以正常工作.但是，我试图通过使用以下行在python脚本中运行此脚本.

in python. When running this line in the shell terminal it works fine. However, I am trying to run this in a python script by using the following line.

subprocess.Popen(["gsutil", "-m", "rsync", "s3://input", "gs://output"])

但是它永远挂着.它输出以下内容:

However it just hangs forever. It outputs the following:

Building synchronization state...
Starting synchronization...

bash命令成功打印:

The bash command successfully prints:

Building synchronization state...
Starting synchronization...
Copying s3://input/0000
[0/1 files][  1.0 MiB/ 5.1 MiB]   (number here)% Done

文件显示在我的gs存储桶中

and the file shows in my gs bucket

推荐答案

我猜这是因为最后两行可能是写到stderr而不是stdout的.您可以尝试使用对 Popen 的调用作为上下文管理器，然后调用 communicate()从输出流中读取吗?

I'm guessing this is because the last two lines are probably written to stderr instead of stdout. Can you try using the call to Popen as a context manager and then calling communicate() to read from the output streams?

proc = subprocess.Popen(["gsutil", "-m", "rsync", "s3://input", "gs://output"])
try:
    outs, errs = proc.communicate(timeout=15)
    # now you can do something with the text in outs and errs
except TimeoutExpired:
    proc.kill()
    outs, errs = proc.communicate()

这篇关于如何从python脚本成功调用gsutil rsync?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！