每次打开/关闭Python文件与保持打开状态直到过程完成 [英] Python file open/close every time vs keeping it open until the process is finished

问题描述

我有大约50 GB的文本文件，我正在检查每行的前几个字符，并将这些字符写入指定的其他文件。

。
我的输入包含：

  cow_ilovecow 
 dog_whreismydog 
 cat_thatcatshouldgotoreddit 
 dog_gotitfromshelter 
 ............... 
  

所以，我想如果写入标记== 1：在牛，狗和猫（约200个）类别中处理它们
所以，

  if writeflag == 1： 
 writefile1 = open（writefile，a）#writefile是somedir / dog.txt .... 
 writefile1.write（remline +\\\
）
＃writefile1.close ）
  

那么，最好的方法是什么？否则，如果我保持打开状态，是 writefile1 = open（writefile，a）做正确的事情？

解决方案

您应该尽可能少地打开/关闭文件。因为即使与文件读/写比较，文件

考虑两个代码块：

  f.ope（'test1.txt'，'w'）
 f.write（'\\\
'）
 f.close（） 
  

和

  for i in range（1000）：
f = open（'test2.txt'，'a'）
 f.write（'\\\
'）
 f.close （）
  

第一个采用 0.025s ，第二个采用<强> 0.309s

I have about 50 GB of text file and I am checking the first few characters each line and writing those to other files specified for that beginning text.

For example. my input contains:

cow_ilovecow
dog_whreismydog
cat_thatcatshouldgotoreddit
dog_gotitfromshelter
...............

So, I want to process them in cow, dog and cat (about 200) categories so,

if writeflag==1:
    writefile1=open(writefile,"a") #writefile is somedir/dog.txt....
    writefile1.write(remline+"\n")
    #writefile1.close()

so, what is the best way, should I close? Otherwise if I keep it open, is writefile1=open(writefile,"a") doing the right thing?

解决方案

You should definitely try to open/close the file as less as possible

Because even comparing with file read/write, file open/close is far more expensive

Consider two code blocks:

f=open('test1.txt', 'w')
for i in range(1000):
    f.write('\n')
f.close()

and

for i in range(1000):
    f=open('test2.txt', 'a')
    f.write('\n')
    f.close()

The first one takes 0.025s while the second one takes 0.309s

这篇关于每次打开/关闭Python文件与保持打开状态直到过程完成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！