如何在 Pandas 数据框中的多行中搜索多个搜索词? [英] How to search for multiple search terms across multiple rows in a Pandas dataframe?
问题描述
所以我以前的更简单的问题在这里 - 如何在熊猫数据框中跨多行搜索文本?
So my previous, more simplified question is here - How to search for text across multiple rows in a pandas dataframe?
我想要做的基本上是能够将包含多个短语的文本文档提供给搜索，而不仅仅是单数词，即新泽西州"等，然后搜索多行中的术语并输出表中的一个新列，如果存在，则为True"，如果不存在，则为False".例如，这是我表格的一小部分，我想搜索新泽西州"和长大"，并使用不同行中的词.
What I want to do is basically to be able to feed a text document containing multiple phrases, not just singular words, i.e. 'new jersey,' etc, into a search and then to search for the terms across multiple rows and output a new column in the table with 'True,' if the terms and present and 'False,' if not. For instance, this is a very small section of my table, and I would like to search 'new jersey' and 'grew up,' with words that are in separate rows.
subtitle start end duration
14 new 71.986000 72.096000 0.110000
15 jersey 72.106000 72.616000 0.510000
16 grew 72.696000 73.006000 0.310000
17 up 73.007000 73.147000 0.140000
18 believing 73.156000 73.716000 0.560000
到目前为止，感谢旧线程的帮助，这就是我所拥有的，terms.txt 是搜索词列表:
So far, thanks to kind help on the old thread, this is what I have, with terms.txt being the list of search terms:
import re
search = [term.strip() for term in open("terms.txt").readlines()]
search = fr"({'|'.join(search)})"
text = " ".join(df["subtitle"])
end = df["subtitle"].apply(len).cumsum() + pd.RangeIndex(len(df))
start = end.shift(fill_value=-1) + 1
df["start"] = start.tolist()
df["end"] = end.tolist()
df["match"] = False
到目前为止一切正常:
for match in re.finditer(search, text, re.IGNORECASE):
idx1 = df[df["start"] == match.start()].index[0]
idx2 = df[df["end"] == match.end()].index[0]
df.loc[idx1:idx2, "match"] = True
我收到错误消息:
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-14-9f347152f616> in <module>
1 for match in re.finditer(search, text, re.IGNORECASE):
----> 2 idx1 = df[df["start"] == match.start()].index[0]
3 idx2 = df[df["end"] == match.end()].index[0]
4 df.loc[idx1:idx2, "match"] = True
~/opt/anaconda3/lib/python3.8/site-packages/pandas/core/indexes/base.py in __getitem__(self, key)
4099 if is_scalar(key):
4100 key = com.cast_scalar_indexer(key, warn_float=True)
-> 4101 return getitem(key)
4102
4103 if isinstance(key, slice):
IndexError: index 0 is out of bounds for axis 0 with size 0
有谁知道我如何解决这个问题，或者是否还有其他方法可以用来达到预期的结果?感谢所有帮助，对于任何格式问题，我深表歉意，因为我是新来的.
Does anyone know how I could fix this or if there are other methods I could use to acheive the desired result? All help is appreciated, and I apologise for any formatting issues since I am very new here.
推荐答案
有 2 列开始"和结束".
There are 2 columns 'start' and 'end'.
import re
terms = [term.strip() for term in open("terms.txt").readlines()]
word = df["subtitle"].str.strip()
end = word.apply(len).cumsum() + pd.RangeIndex(len(df))
start = end.shift(fill_value=-1) + 1
text = " ".join(word)
df["match"] = False
for term in terms:
for match in re.finditer(fr"\b{term}\b", text, re.IGNORECASE):
idx1 = start[start == match.start()].index[0]
idx2 = end[end == match.end()].index[0]
df[idx1:idx2] = True
输出:
$ cat terms.txt
new jersey
hello
>>> df
id subtitle start end duration match
0 14 new 71.986 72.096 0.11 True
1 15 jersey 72.106 72.616 0.51 True
2 16 grew 72.696 73.006 0.31 False
3 17 up 73.007 73.147 0.14 False
4 18 believing 73.156 73.716 0.56 False
这篇关于如何在 Pandas 数据框中的多行中搜索多个搜索词?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
