在 Python 中修改大文本文件最后一行的最有效方法 [英] Most efficient way to modify the last line of a large text file in Python

问题描述

我需要从一些超过 2GB 的文件更新最后一行，这些文件由无法使用 readlines() 读取的文本行组成.目前，它可以通过逐行循环来正常工作.但是，我想知道是否有任何编译库可以更有效地实现这一点?谢谢！

I need to update the last line from a few more than 2GB files made up of lines of text that can not be read with readlines(). Currently, it work fine by looping through line by line. However, I am wondering if there is any compiled library can achieve this more efficiently? Thanks!

    myfile = open("large.XML")
    for line in myfile:
        do_something()

推荐答案

更新:使用 ShadowRanger 的回答.它更短更健壮.

Update: Use ShadowRanger's answer. It's much shorter and robust.

为了后代:

读取文件的最后 N 个字节并向后搜索换行符.

Read the last N bytes of the file and search backwards for the newline.

#!/usr/bin/env python

with open("test.txt", "wb") as testfile:
    testfile.write('
'.join(["one", "two", "three"]) + '
')

with open("test.txt", "r+b") as myfile:
    # Read the last 1kiB of the file
    # we could make this be dynamic, but chances are there's
    # a number like 1kiB that'll work 100% of the time for you
    myfile.seek(0,2)
    filesize = myfile.tell()
    blocksize = min(1024, filesize)
    myfile.seek(-blocksize, 2)
    # search backwards for a newline (excluding very last byte
    # in case the file ends with a newline)
    index = myfile.read().rindex('
', 0, blocksize - 1)
    # seek to the character just after the newline
    myfile.seek(index + 1 - blocksize, 2)
    # read in the last line of the file
    lastline = myfile.read()
    # modify last_line
    lastline = "Brand New Line!
"
    # seek back to the start of the last line
    myfile.seek(index + 1 - blocksize, 2)
    # write out new version of the last line
    myfile.write(lastline)
    myfile.truncate()

这篇关于在 Python 中修改大文本文件最后一行的最有效方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！