如何聚类搜索引擎关键字？ [英] How to cluster search engine keywords?

问题描述

从Google Analytics我有一个（长）关键字列表，人们在搜索引擎中用它来查找我的网站。我想找到'核心关键词'，假设的例子：

From Google Analytics I have a (long) list of keywords that people used in search engines to find my website. I want to find the 'core keywords', hypothetical example:

java online training
learning java
scala training
training for java
online training java
learn scala programming

<理想的结果是：'java'，'在线培训'，'培训'，'scala'和'学习'。

The ideal result would be: 'java', 'online training', 'training', 'scala' and 'learn'.

难度似乎是检测到完整的短语，忽略常用词（for）和处理变体（学习 - 学习）。

The difficulty seems to be detecting complete phrases, ignoring common words (for) and handling variations (learn-learning).

是否有可以做到这一点的库（最好是JVM）？或者我可以自己实现一个合适的算法吗？

推荐答案

这是术语或关键字提取问题。我做了一个搜索，它出现了 Kea ，这看起来非常符合您的需求。

This is a term or keyword extraction problem. I did a search and it turned up Kea, which looks to be very much what you want.

您可以通过以下算法实现一个天真的解决方案：

You can implement a naive solution by the following algorithm:

• 生成一个ngrams列表具有您想要的短语长度的文档（选择任意短语长度限制，如3或4）


• 将ngram放入 Multiset


• 遍历条目multiset按照他们的学位或计数的顺序，也许是任意截止

就像你说的那样，这将有一个停用词的问题。你可以做一些简单的事情，比如有一个停用词典，或者你可以做一些事情，比如 Term频率反向文档频率，可以帮助您自动识别非常频繁的术语。 KEA会为你做这件事，最好先调查一下。

Like you said, this will have a problem with stopwords. You can do something simple like have a dictionary of stopwords, or you can do something like Term Frequency-Inverse Document Frequency which can help you automatically recognize very frequent terms. KEA will do this for you, it might be best to look into that first.

希望有所帮助！

这篇关于如何聚类搜索引擎关键字？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！