使用python在txt文件中查找字符串的最佳方法是什么? [英] What is the best way to find string in txt file by using python?

问题描述

使用python在txt文件中查找字符串肯定有多种方法，但最好的方法是什么?(为了速度，为了资源..)

there must be various ways to find string in txt file by using python, but what is the best way? ( for speed, for resources .. )

我的第一个想法如下.

file = open('/home/socfw/src/edl/outbound_monthly.txt')

inputIP = '127.0.0.1'

while (1):
    line = file.readline()
    if inputIP in line:
        print("ok")
        break

但是，正确使用web服务太慢了(它实际上是我的web服务的后端逻辑)txt 文件如下所示

But, it's too slow to use web service properly (it is actually backend logic of my web service) txt file looks like as below

test.txt(这里有 IPV4 地址，几乎有 6 万个)

test.txt ( IPV4 addresses are in here, and they counts almost 60k)

x.x.x.x
x.x.x.x
.
.
.
.

我的源代码导致 100% CPU 持续几分钟，所以我想找到另一种方法.有什么好的解决办法吗?提前致谢.

My source code causes 100 percent CPU for several mins, so I want to find another way. Is there any good solution for me? thanks in advance.

谢谢你回答我.我改变了我的来源，如下所示.

Thank you for answering me. I changed my sources as below.

with open('/home/socfw/src/edl/outbound_monthly.txt') as outMonIPs:
    ip = set(line.strip() for line in outMonIPs)

inputIP = '111.90.150.249'
#while True:
if inputIP in ip:
    print("ok")
#        break
else:
    print("no")
#        break

我还有一个问题，我应该使用 loop 来完成这项工作吗?当我将整个文件保存在内存中时，我认为不再需要循环.

I have one more question, shoud I use loop for this work? I think loop is no more need when I save whole file in memory.

推荐答案

如果您必须使用文本文件，您可以尝试将整个文件读入内存而不是逐行搜索以加快速度.(如果您将所有文件读入内存，则不再需要循环)

If you have to use text files, you could try reading the entire file into memory instead of searching it line by line to speed things up. (If you read all of the file into memory, you don't need the loop any more)

您可以尝试使用 grep 或 find，而不是编写 Python 脚本来进行搜索.

Instead of writing a python script to do the search, you could try using grep or find.

您应该考虑将数据放入数据库并查询以查找匹配项.这种方法应该更有效地利用资源并且应该更快，因为数据库可以使用索引，并且它们不必将整个数据集读入内存来查找匹配项.如果您的应用程序足够简单，您或许可以使用 sqlite.

You should consider putting your data into a database and querying it to find matches. This approach should be a lot more resource efficient and should be faster since databases can make use of indexes and they don't necessarily have to read the entire dataset into memory to find matches. If your application is simple enough, you might be able to use sqlite.

这篇关于使用python在txt文件中查找字符串的最佳方法是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！