使用python从配置单元查询输出中删除空行 [英] remove empty line printed from hive query output using python

查看:494
本文介绍了使用python从配置单元查询输出中删除空行的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我正在执行配置单元查询并将输出存储在本地FS中的tsv文件中。我为配置单元查询运行for循环并传递不同的参数。如果配置单元查询在for循环中不返回任何输出,它将在tsv文件中输出一个空行。这导致NULL值被推送到我的数据库在后端。因此,在for循环运行并创建文件后 - 我有下面的代码来删除所有打印的空行,但它不起作用。



我从这个文件中删除空行?

 `395.9 429.61 PT 
`

code:

  with open('file.tsv','r + ('):
file.write(line)

谢谢

解决方案

通常你会打开输入文件并写入非空行到第二个文件:

  with open('file.tsv')as infile,open('filtered_file ()):
outfile.write(line)
for line in infile:
如果line.strip c>

如果要过滤inplace文件,可以使用 FileInput 如果line.strip():
打印行

然而,这使用中间文件,并且可能无法在磁盘空间不足的情况下使用。



要过滤未放置的文件你可以尝试这样的额外磁盘空间:

  with open('file.tsv','r +')as infile: 
read_pos = write_pos = 0
line = infile.readline()
while line:
read_pos + = len(line)
如果line.strip():
infile.seek(write_pos)
infile.write(line)
write_pos + = len(line)
infile.seek(read_pos)
line = infile.readline( )
#将文件大小更新为可能缩小的新大小
infile.truncate(write_pos)


i am performing a hive query and storing the output in a tsv file in the local FS. I am running a for loop for the hive query and passing different parameters. If the hive query returns no output once in the for loop it prints an empty line in the tsv file. This causes NULL values to be pushed to my DB in the backend. Hence, after the for loop runs and the file is created - i have the below code to remove all the empty lines printed, but it doesn't work.

How do i remove the empty line from this file?

` 395.9   429.61  PT  
                       `

code:

with open('file.tsv','r+w') as file:
        for line in file:
          if line.strip():
            file.write(line)

thanks

解决方案

Usually you would open the input file and write the non-empty lines to a second file:

with open('file.tsv') as infile, open('filtered_file.tsv', 'w') as outfile:
    for line in infile:
        if line.strip():
            outfile.write(line)

If you want to filter the file inplace you can use FileInput with the inplace option:

import fileinput
for line in fileinput.FileInput("infile", inplace=1):
    if line.strip():
        print line

however, this uses an intermediate file and may not work in low disk space situations.

To filter the file inplace without allocating any additional disk space you could try something like this:

with open('file.tsv', 'r+') as infile:
    read_pos = write_pos = 0
    line = infile.readline()
    while line:
        read_pos += len(line)
        if line.strip():
            infile.seek(write_pos)
            infile.write(line)
            write_pos += len(line)
        infile.seek(read_pos)
        line = infile.readline()
    # update file size to the new, possibly reduced, size
    infile.truncate(write_pos)

这篇关于使用python从配置单元查询输出中删除空行的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆