尝试在Python中执行模糊匹配 [英] Trying to Perform Fuzzy Matching in Python

问题描述

我正在尝试执行Fuzzywuzzy命令，比较数据帧中的两列.我想知道一列(关系")中的字符串是否存在于另一列("CUST_NAME")中，甚至部分存在.然后，在与之前的同一列("CUST_NAME")相同的第二列("Dealer_Name")上重复该过程.我目前正在尝试运行以下代码:

I am trying to perform a fuzzywuzzy command comparing two columns in a dataframe. I want to know if a character string from one column ('Relationship') exists in another ('CUST_NAME'), even partially. Then repeat the process for a second column ('Dealer_Name'), on the same column as prior ('CUST_NAME'). I am currently trying to run the following code:

这是我的数据框:

RapDF1 = RapDF[['APP_KEY','Relationship','Dealer_Name','CUST_NAME']]

这是模糊匹配:

from fuzzywuzzy import process, fuzz

RapDF1.assign(dealer_compare=[process.extract(i, RapDF1['Dealer_Name'], limit=3) for i in RapDF1['CUST_NAME']])
RapDF1.assign(broker_compare=[process.extract(i, RapDF1['Relationship'], limit=3) for i in RapDF1['CUST_NAME']])

但是，我收到以下python错误:

However, I receive the following python error:

TypeError                                 Traceback (most recent call last)
<ipython-input-76-2faf28514c26> in <module>()
     52 # Attempt 7
     53 
---> 54 RapDF1.assign(dealer_compare=[process.extract(i, RapDF1['Dealer_Name'], limit=3) for i in RapDF1['CUST_NAME']])
     55 RapDF1.assign(broker_compare=[process.extract(i, RapDF1['Relationship'], limit=3) for i in RapDF1['CUST_NAME']])
     56 

<ipython-input-76-2faf28514c26> in <listcomp>(.0)
     52 # Attempt 7
     53 
---> 54 RapDF1.assign(dealer_compare=[process.extract(i, RapDF1['Dealer_Name'], limit=3) for i in RapDF1['CUST_NAME']])
     55 RapDF1.assign(broker_compare=[process.extract(i, RapDF1['Relationship'], limit=3) for i in RapDF1['CUST_NAME']])
     56 

C:\ProgramData\Anaconda3\lib\site-packages\fuzzywuzzy\process.py in extract(query, choices, processor, scorer, limit)
    166     """
    167     sl = extractWithoutOrder(query, choices, processor, scorer)
--> 168     return heapq.nlargest(limit, sl, key=lambda i: i[1]) if limit is not None else \
    169         sorted(sl, key=lambda i: i[1], reverse=True)
    170 

C:\ProgramData\Anaconda3\lib\heapq.py in nlargest(n, iterable, key)
    567     # General case, slowest method
    568     it = iter(iterable)
--> 569     result = [(key(elem), i, elem) for i, elem in zip(range(0, -n, -1), it)]
    570     if not result:
    571         return result

C:\ProgramData\Anaconda3\lib\heapq.py in <listcomp>(.0)
    567     # General case, slowest method
    568     it = iter(iterable)
--> 569     result = [(key(elem), i, elem) for i, elem in zip(range(0, -n, -1), it)]
    570     if not result:
    571         return result

C:\ProgramData\Anaconda3\lib\site-packages\fuzzywuzzy\process.py in extractWithoutOrder(query, choices, processor, scorer, score_cutoff)
     76 
     77     # Run the processor on the input query.
---> 78     processed_query = processor(query)
     79 
     80     if len(processed_query) == 0:

C:\ProgramData\Anaconda3\lib\site-packages\fuzzywuzzy\utils.py in full_process(s, force_ascii)
     93         s = asciidammit(s)
     94     # Keep only Letters and Numbers (see Unicode docs).
---> 95     string_out = StringProcessor.replace_non_letters_non_numbers_with_whitespace(s)
     96     # Force into lowercase.
     97     string_out = StringProcessor.to_lower_case(string_out)

C:\ProgramData\Anaconda3\lib\site-packages\fuzzywuzzy\string_processing.py in replace_non_letters_non_numbers_with_whitespace(cls, a_string)
     24         numbers with a single white space.
     25         """
---> 26         return cls.regex.sub(" ", a_string)
     27 
     28     strip = staticmethod(string.strip)

TypeError: expected string or bytes-like object

推荐答案

数据框中可能存在nan值，nan具有浮点类型并导致错误:

Propably there are nan values in the dataframe, nan has a type float and causes an error:

from fuzzywuzzy import process, fuzz
import pandas as pd
import numpy as np

df_nan = pd.DataFrame({'text1': ["quick", "brown", "fox"], "text2": ["hello", np.NaN, "world"]})
df_nan
Out:
   text1  text2
0  quick  hello
1  brown    NaN
2    fox  world

仅是引起相同错误的代码示例:

Just an example of code which causes the same error:

[process.extract(i, df_nan['text1'], limit=3) for i in df_nan['text2']]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
...
/usr/local/lib/python3.6/dist-packages/fuzzywuzzy/string_processing.py in replace_non_letters_non_numbers_with_whitespace(cls, a_string)
     24         numbers with a single white space.
     25         """
---> 26         return cls.regex.sub(" ", a_string)
     27 
     28     strip = staticmethod(string.strip)

TypeError: expected string or bytes-like object

带有某些令牌的副本nan(选择正确的令牌将是困难且与数据相关的任务，可能是空字符串是一个不好的选择):

Replcace nan's with some token (choose correct token will be hard and data-dependent task, probably empty string is a bad choice):

df = df_nan.fillna('##SOME_TOKEN##') 
[process.extract(i, df['text1'], limit=3) for i in df['text2']]
Out:
[[('fox', 36, 2), ('brown', 20, 1), ('quick', 0, 0)],
 [('brown', 36, 1), ('fox', 30, 2), ('quick', 18, 0)],
 [('fox', 30, 2), ('brown', 20, 1), ('quick', 0, 0)]]

我猜想替换或删除所有非字符串值会有所帮助.

I guess replace or drop all non-string values will help.

这篇关于尝试在Python中执行模糊匹配的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！