在Python中修改大文本文件的最后一行的最有效方法 [英] Most efficient way to modify the last line of a large text file in Python
本文介绍了在Python中修改大文本文件的最后一行的最有效方法的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我需要更新一些超过2GB文件的最后一行,这些文件由 readlines()
无法读取的文本行组成。目前,它通过逐行循环工作正常。但是,我想知道是否有任何编译库可以更有效地实现这一点?谢谢!
I need to update the last line from a few more than 2GB files made up of lines of text that can not be read with readlines()
. Currently, it work fine by looping through line by line. However, I am wondering if there is any compiled library can achieve this more efficiently? Thanks!
myfile = open("large.XML")
for line in myfile:
do_something()
推荐答案
更新:使用 ShadowRanger的答案。它更短更健壮。
Update: Use ShadowRanger's answer. It's much shorter and robust.
对于后代:
读取文件的最后N个字节,然后向后搜索换行符。
Read the last N bytes of the file and search backwards for the newline.
#!/usr/bin/env python
with open("test.txt", "wb") as testfile:
testfile.write('\n'.join(["one", "two", "three"]) + '\n')
with open("test.txt", "r+b") as myfile:
# Read the last 1kiB of the file
# we could make this be dynamic, but chances are there's
# a number like 1kiB that'll work 100% of the time for you
myfile.seek(0,2)
filesize = myfile.tell()
blocksize = min(1024, filesize)
myfile.seek(-blocksize, 2)
# search backwards for a newline (excluding very last byte
# in case the file ends with a newline)
index = myfile.read().rindex('\n', 0, blocksize - 1)
# seek to the character just after the newline
myfile.seek(index + 1 - blocksize, 2)
# read in the last line of the file
lastline = myfile.read()
# modify last_line
lastline = "Brand New Line!\n"
# seek back to the start of the last line
myfile.seek(index + 1 - blocksize, 2)
# write out new version of the last line
myfile.write(lastline)
myfile.truncate()
这篇关于在Python中修改大文本文件的最后一行的最有效方法的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文