在不使用 nltk 语料库的情况下删除停用词 [英] removing stop words without using nltk corpus

问题描述

我试图在不使用 nltk 的情况下删除文本文件中的停用词.我有 f1,f2,f3 三个文本文件.f1 有逐行文本，f2 有停用词列表，f3 是空文件.我想逐行阅读 f1，然后逐字阅读，需要检查它是否在 f2(停用词)中.如果该词不在停用词中，则将该词写入 f3.因此，最后 f3 应该有像 f1 一样的文本，但在每一行中，应该删除 f2(停用词)中的单词.

I am trying to remove stop words in a text file without using nltk. I have f1,f2,f3 three text files. f1 has text line by line and f2 has stop words list and f3 is empty file. I want to read f1 line by line and in turn word by word and need to check whether it is in f2(stop words). If the word is not in the stop word then write the word in f3. Thus at the end f3 should have text as in f1 but in each line, words in f2(stop words) should be removed.

f1 = open("file1.txt","r")
f2 = open("stop.txt","r")
f3 = open("file2.txt","w")

for line in f1:
    words = line.split()
    for word in words:
        t=word

for line in f2:
    w = line.split()
    for word in w:
        t1=w
        if t!=t1:
            f3.write(word)

f1.close()
f2.close()
f3.close()

这段代码是错误的.但是任何人都可以通过更改代码来完成这项任务.

this code is wrong. but can any one do this task by changing the code.

提前致谢.

推荐答案

你可以使用 Linux Sed 方法去除停用词

YOu can use Linux Sed method for removing the stopwords

sed -f <(sed 's/.*/s|\\\<&\\\>||g/' stopwords.txt) all_lo.txt > all_remove1.txt

这篇关于在不使用 nltk 语料库的情况下删除停用词的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！