python实时写入文件 [英] python write to file in real time

问题描述

我有这段python代码，它循环遍历文本文件(urls.txt)中的URL列表，然后跟随所有URL的重定向，如果该URL包含特定字符串，则将其写入名为redirects的文件中.txt

I have this piece of python code, that loops thru a list of urls in a text file(urls.txt) then follows redirects of all urls and if the url contains a specific string, it writes it to a file called redirects.txt

import urllib.request
import ssl
redf = open('redirect.txt', 'w')
with open('urls.txt') as f:
   for row in f:
    #try:

      context = ssl._create_unverified_context()
      finalurl = ''
      try:
        res      = urllib.request.urlopen(row, context=context, timeout=10)
        finalurl = res.geturl().strip()
      except:
          #remove from list
          print("error:"+finalurl)

      # filedata = file.read()
      if finalurl.strip():
        if "/admin/" in finalurl:
            redf.write(finalurl+"\n");

问题在于，在创建redirect.txt文件之前，我必须等待整个URS处理完毕.

The problem is that I have to wait for the entire URS to be processed before the redirect.txt file is created.

我该如何实时写作?

推荐答案

文件已创建，但是由于您的输出很小，因此很有可能所有文件都卡在了写缓冲区中，直到关闭文件为止.如果您需要更迅速地填写文件，请在行缓冲中打开文件传递 buffering = 1 :

The file is created, but since your output is small, it's likely that it's all stuck in the write buffer until the file is closed. If you need the file to be filled in more promptly, either open it in line buffered mode by passing buffering=1:

open('redirect.txt', 'w', buffering=1)

每次 write 后

或 flush ，通过

or flush after each write, either by explicitly calling flush:

redf.write(finalurl+"\n")
redf.flush()

或者，由于您仍然要添加换行符，因此最好使用带有 flush = True 的 print :

or, since you're adding newlines anyway so you may as well let it work for you, by using print with flush=True:

print(finalurl, file=redf, flush=True)

旁注:您真的想使用 with 语句尤其是打开了要写入的文件，但您只将其用于正在读取的文件(在此情况下，它不太重要，因为最坏的情况只是延迟关闭了句柄，而不会丢失)写道).否则，异常可能导致刷新/关闭文件中的任意延迟.只需将两个打开组合成一个 with ，例如:

Side-note: You really want to use with statements with files opened for write in particular, but you only used it for the file being read (where it's less critical, since the worst case is just a delayed handle close, not lost writes). Otherwise exceptions can lead to arbitrary delaying in the file being flushed/closed. Just combine the two opens into one with, e.g.:

with open('urls.txt') as f, open('redirect.txt', 'w', buffering=1) as redf:

这篇关于python实时写入文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！