Apache Nutch 只索引页面内容的一部分 [英] Apache Nutch to index only part of page content

问题描述

将使用 Apache Nutch v1.3 仅从网页中提取某些特定内容.检查 parse-html 插件.似乎它使用 tagsoup 或 nekohtml 规范化每个 html 页面.这很好.我只需要提取网页上 和 elemetns 中的文本.如果将提取的文本保存到不同的字段(例如 content_xxx、content_yyy)，那就太好了.我的问题是:我应该编写自己的插件还是可以使用某种标准方式来完成?

Going to use Apache Nutch v1.3 to extract only some specific content from the webpages. Checked parse-html plugin. Seems it normalizes each html page using tagsoup or nekohtml. This is good. I need to extract only text inside <span class='xxx'> and <span class='yyy'> elemetns on the web-page. Would be great if extracted texts are saved into different fields (e.g. content_xxx, content_yyy). My question is: should I write my own plugin or this could be done using some standard way?

最好的方法是在规范化的网页上应用 XSLT 并获得结果.这可能吗?

The best way would be apply XSLT on normalized web-page and get the result. Is that possible?

推荐答案

构建您自己的 ParsingFilter 和 IndexingFilter 很容易.Nutch 为你提供了 DOM 文档，你只需要遍历和搜索你的 div.然后，您只需将新字段添加到索引和架构中即可.

Building your own ParsingFilter and IndexingFilter is easy. Nutch provides you with the DOM document, which you only need to traverse and search for your div. Then you simply add the new fields to your index and schema and your done.

有一些关于如何执行此操作的示例:

There are some examples on how to do this:

http://wiki.apache.org/nutch/HowToMakeCustomSearch

http://sujitpal.blogspot.com/2009/07/nutch-custom-plugin-to-parse-and-add.html

祝你好运

这篇关于Apache Nutch 只索引页面内容的一部分的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！