Python-使用Subprocess的Hive命令-空结果 [英] Python - Hive commands using Subprocess - empty results

问题描述

我正在使用子进程在python中运行hive命令，但结果却空了.如果我从Hive CLI运行相同的命令，将会获得结果.

I'm using subprocess to run hive commands in python, but am getting empty results. If i run the same commands from hive CLI, am getting results.

 query = "set hive.cli.print.header=true;use mydb;describe table1;"  
 process = subprocess.Popen( ["ssh", "hadoop" , "hive", "-e", "%r" % query], stdout = subprocess.PIPE, stderr = subprocess.PIPE )  
 data = [line.split('\t') for line in process.stdout]  
 cols = list(itertools.chain.from_iterable(data[:1]))  
 df = pd.DataFrame(data[1:], columns = cols)  
 print "==>"+df+"<----"  

它返回空的数据框.

请帮助我

推荐答案

myfile=open("query_result.tsv", 'w')
p=subprocess.Popen("your query",
        shell=True,
        stdout=myfile,stderr=subprocess.PIPE)
stdout,stderr = p.communicate()
if p.returncode != 0:
    print stderr
    sys.exit(1)

myfile是一个tsv文件，您可以使用pandas.read_csv(sep ='\ t')并设置sep ='\ t'，您可能需要查看pandas api才能找到有关read_csv()的更多用法.

myfile is a tsv file,you can use pandas.read_csv(sep='\t') and set sep='\t' ,you may need to look up pandas api to find more usage about read_csv().

您应该在17.1.2中查找有关Popen Object的子进程api.它会向您发出有关stdout = PIPE的警告. https://docs.python.org/2/library/subprocess.html#frequently-used-arguments

you should look up subprocess api in 17.1.2 about Popen Object.it gives you a warning about stdout=PIPE. https://docs.python.org/2/library/subprocess.html#frequently-used-arguments

这篇关于Python-使用Subprocess的Hive命令-空结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！