确定两个名称是否彼此接近 [英] Determine if two names are close to each other

问题描述

我正在为我的学校制作一个系统，我们可以在其中检查学生是否在聚会和其他活动中被列入黑名单.我可以很容易地检查学生是否被列入黑名单，因为我可以在我的数据库中查找该学生并查看他/她是否被列入黑名单.

这就是困难的地方.

在我们的聚会上，每个学生可以邀请一个人.理论上，一个被列入黑名单的学生可以被另一个学生邀请并绕过系统.我无法检查列入黑名单的学生的来宾表，因为当您邀请您的客人时，只提供了一个名字.

所以我需要检查列入黑名单的姓名是否接近客人姓名，如果接近则显示警告，不幸的是，有一些事情需要考虑.

名称可以完全不同.在丹麦，标准名称包含三个名称"，例如Niels Faurskov Andersen"但学生可能只输入Niels Faurskov"或Niels Andersen"，甚至删除一些字符.

所以像 Niels Faurskov Andersen 这样的全名可能是

• 尼尔斯·安徒生
• 尼尔斯·法乌斯科夫
• 尼尔斯·福尔斯科夫·安徒生
• 尼尔斯·福尔斯科夫·安徒生
• 尼尔斯·安徒生
• 尼尔斯·福尔斯科夫
• 尼尔斯·法尔斯科夫

等等……

另一件事是丹麦字母表除了通常的 a-z 之外还包含æøå".话虽如此，整个站点和数据库都是 UTF-8 编码的.

我研究了各种方法来检查两个字符串之间的差异，但 Levenshtein 距离并不能完全做到这一点.

我在 StackOverflow 上找到了这个帖子:具有 O(N³) 复杂度，因此对于大型比较集，它会非常慢.

但是您仍然可以通过动态检查来改进当前的解决方案.现在这段代码将首先生成所有字符串子序列，然后开始一一检查.通常情况下您不需要这样做，因此您可能希望将其替换为行为，在生成下一个序列后将立即对其进行检查.然后，您将提高具有肯定答案的字符串的性能(但不是那些不匹配的字符串).

I'm making a system for my school where we can check if a student is black-listed, at parties and other events. It's easy for me to check if a student is black-listed, since I can just look the student up in my database and see if he/she is black-listed.

Here is where it gets difficult though.

At our parties, each student can invite one person. In theory a student who is black-listed, can be invited by another student and bypass the system. I cannot check the guest table for students black-listed, because only a name is provided when you invite your guest.

So I need to check if a black-listed name is close to a guest name, and display a warning if they are close, unfortunately there are some stuff to take into account.

Names can be quite different. In Denmark, the standard name, contains three "names", like "Niels Faurskov Andersen" But a student may just type "Niels Faurskov" or "Niels Andersen", or even some characters removed.

So a fullname such as Niels Faurskov Andersen could be

• Niels Andersen
• Niels Faurskov
• Niels Faurskov Andersen
• Nils Faurskov Andersen
• Nils Andersen
• niels faurskov
• niels Faurskov

And so on...

Another thing is that the Danish alphabet contains "æøå" apart from the usual a-z. With that said the whole site and database is UTF-8 encoded.

I've looked into various methods to check the difference between two strings, and the Levenshtein distance doesn't quite do it.

I found this thread on StackOverflow: Getting the closest string match

Which seemed to provided the right data, however I wasn't quite sure what method too choose

I'm coding this part in php, does anybody have an idea how to do this? maybe with MySQL? or a modified version of the Levenshtein distance? Could regex be possible?

解决方案

Introduction

Quite now your matching conditions may be too broad. However, you can use levenshtein distance to check your words. It may be not too easy to fulfill all desired goals with it, like sound similarity. Thus, I'm suggesting to split your issue into some other issues.

For example, you can create some custom checker which will use passed callable input which takes two strings and then answering question about are they same (for levenshtein that will be distance lesser than some value, for similar_text - some percent of similarity e t.c. - it's up to you to define rules).

Similarity, based on words

Well, all of built-in functions will fail if we are talking about case when you're looking for partial match - especially if it's about non-ordered match. Thus, you'll need to create more complex comparison tool. You have:

• Data string (that will be in DB, for example). It looks like D = D₀ D₁ D₂ ... D_n
• Search string (that will be user input). It looks like S = S₀ S₁ ... S_m

Here space symbols means just any space (I assume that space symbols will not affect similarity). Also n > m. With this definition your issue is about - to find set of m words in D which will be similar to S. By set I mean any unordered sequence. Hence, if we'll found any such sequence in D, then S is similar to D.

Obviously, if n < m then input contains more words than data string. In this case you may either think that they are not similar or act like above, but switch data and input (that, however, looks a little bit odd, but is applicable in some sense)

Implementation

To do the stuff, you'll need to be able to create set of string which are parts from m words from D. Based on my this question you can do this with:

protected function nextAssoc($assoc)
{
   if(false !== ($pos = strrpos($assoc, '01')))
   {
      $assoc[$pos]   = '1';
      $assoc[$pos+1] = '0';
      return substr($assoc, 0, $pos+2).
             str_repeat('0', substr_count(substr($assoc, $pos+2), '0')).
             str_repeat('1', substr_count(substr($assoc, $pos+2), '1'));
   }
   return false;
}

protected function getAssoc(array $data, $count=2)
{
   if(count($data)<$count)
   {
      return null;
   }
   $assoc   = str_repeat('0', count($data)-$count).str_repeat('1', $count);
   $result = [];
   do
   {
      $result[]=array_intersect_key($data, array_filter(str_split($assoc)));
   }
   while($assoc=$this->nextAssoc($assoc));
   return $result;
}

-so for any array, getAssoc() will return array of unordered selections consisting from m items each.

Next step is about order in produced selection. We should search both Niels Andersen and Andersen Niels in our D string. Therefore, you'll need to be able to create permutations for array. It's very common issue, but I'll put my version here too:

protected function getPermutations(array $input)
{
   if(count($input)==1)
   {
      return [$input];
   }
   $result = [];
   foreach($input as $key=>$element)
   {
      foreach($this->getPermutations(array_diff_key($input, [$key=>0])) as $subarray)
      {
         $result[] = array_merge([$element], $subarray);
      }
   }
   return $result;
}

After this you'll be able to create selections of m words and then, permutating each of them, get all variants for compare with search string S. That comparison each time will be done via some callback, such as levenshtein. Here's sample:

public function checkMatch($search, callable $checker=null, array $args=[], $return=false)
{
   $data   = preg_split('/s+/', strtolower($this->data), -1, PREG_SPLIT_NO_EMPTY);
   $search = trim(preg_replace('/s+/', ' ', strtolower($search)));
   foreach($this->getAssoc($data, substr_count($search, ' ')+1) as $assoc)
   {
       foreach($this->getPermutations($assoc) as $ordered)
       {
           $ordered = join(' ', $ordered);
           $result  = call_user_func_array($checker, array_merge([$ordered, $search], $args));
           if($result<=$this->distance)
           {
               return $return?$ordered:true;
           }
       }
   }
   
   return $return?null:false;
}

This will check on similarity, based on user callback, which must accept at least two parameters (i.e. compared strings). Also you may wish to return string which triggered callback positive return. Please, note, that this code will not differ upper and lower case - but may be you do not want such behavior (then just replace strtolower()).

Sample of full code is available in this listing (I didn't used sandbox since I'm not sure about how long code listing will be available there). With this sample of usage:

$data   = 'Niels Faurskov Andersen';
$search = [
    'Niels Andersen',
    'Niels Faurskov',
    'Niels Faurskov Andersen',
    'Nils Faurskov Andersen',
    'Nils Andersen',
    'niels faurskov',
    'niels Faurskov',
    'niffddels Faurskovffre'//I've added this crap
];

$checker = new Similarity($data, 2);

echo(sprintf('Testing "%s"'.PHP_EOL.PHP_EOL, $data));
foreach($search as $name)
{
   echo(sprintf(
      'Name "%s" has %s'.PHP_EOL, 
      $name, 
      ($result=$checker->checkMatch($name, 'levenshtein', [], 1))
         ?sprintf('matched with "%s"', $result)
         :'mismatched'
      )
   );

}

you'll get result like:

Testing "Niels Faurskov Andersen"

Name "Niels Andersen" has matched with "niels andersen"
Name "Niels Faurskov" has matched with "niels faurskov"
Name "Niels Faurskov Andersen" has matched with "niels faurskov andersen"
Name "Nils Faurskov Andersen" has matched with "niels faurskov andersen"
Name "Nils Andersen" has matched with "niels andersen"
Name "niels faurskov" has matched with "niels faurskov"
Name "niels Faurskov" has matched with "niels faurskov"
Name "niffddels Faurskovffre" has mismatched

-here is demo for this code, just in case.

Complexity

Since you're caring about not just any methods, but also about - how good is it, you may notice, that such code will produce quite excessive operations. I mean, at least, generation of string parts. Complexity here consists of two parts:

• Strings parts generation part. If you want to generate all string parts - you'll have to do this like I've described above. Possible point to improve - generation of unordered string sets (that comes before permutation). But still I doubt it can be done because method in provided code will generate them not with "brute-force", but as they are mathematically calculated (with cardinality of )
• Similarity checking part. Here your complexity depends of given similarity checker. For example, similar_text() has O(N³) complexity, thus with large comparison sets it will be extremely slow.

But you still may improve current solution with checking on the fly. Now this code will first generate all string sub-sequences and then start checking them one by one. In common case you don't need to do that, so you may want to replace that with behavior, when after generating next sequence it will be checked immediately. Then you'll increase performance for strings which have positive answer (but not for those which have no match).

这篇关于确定两个名称是否彼此接近的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！