将相似词分组的好策略是什么? [英] What is a good strategy to group similar words?

问题描述

假设我有一个电影名称列表，其中包含拼写错误和类似这样的细微变化 -

加勒比海盗:黑珍珠号的诅咒"《加勒比海盗》《加勒比海盗:死人的箱子》加勒比海盗三部曲"加勒比海盗"《加勒比海盗》

我如何分组或查找这些词组，最好使用 python 和/或 redis?

解决方案

看看模糊匹配".下面线程中的一些很棒的工具可以计算字符串之间的相似度.

我特别喜欢 difflib 模块

<预><代码>>>>get_close_matches('appel', ['ape', 'apple', 'peach', 'puppy'])['苹果'，'猿']>>>导入关键字>>>get_close_matches('wheel',keyword.kwlist)['尽管']>>>get_close_matches('apple',keyword.kwlist)[]>>>get_close_matches('accept',keyword.kwlist)['除了']

https://stackoverflow.com/questions/682367/good-python-modules-for-fuzzy-string-comparison

Say I have a list of movie names with misspellings and small variations like this -

 "Pirates of the Caribbean: The Curse of the Black Pearl"
 "Pirates of the carribean"
 "Pirates of the Caribbean: Dead Man's Chest"
 "Pirates of the Caribbean trilogy"
 "Pirates of the Caribbean"
 "Pirates Of The Carribean"

How do I group or find such sets of words, preferably using python and/or redis?

解决方案

Have a look at "fuzzy matching". Some great tools in the thread below that calculates similarities between strings.

I'm especially fond of the difflib module

>>> get_close_matches('appel', ['ape', 'apple', 'peach', 'puppy'])
['apple', 'ape']
>>> import keyword
>>> get_close_matches('wheel', keyword.kwlist)
['while']
>>> get_close_matches('apple', keyword.kwlist)
[]
>>> get_close_matches('accept', keyword.kwlist)
['except']

https://stackoverflow.com/questions/682367/good-python-modules-for-fuzzy-string-comparison

这篇关于将相似词分组的好策略是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！