使用Nutch 2.3，Cassandra 2.0和solr 4.10.3进行爬网将返回0个结果 [英] crawling with Nutch 2.3, Cassandra 2.0, and solr 4.10.3 returns 0 results

问题描述

我主要按照本页上的指南进行操作。我安装了Nutch 2.3，Cassandra 2.0和solr 4.10.3。设置进行得很好。但是当我执行以下命令时。

I mainly followed the guide on this page. I installed Nutch 2.3, Cassandra 2.0, and solr 4.10.3. Set up went well. But when I executed the following command. No urls were fetched.

./bin/crawl urls/seed.txt TestCrawl http://localhost:8983/solr/ 2

以下是我的设置。

nutch-site.xml

nutch-site.xml

http://ideone.com/H8MPcl

regex-urlfilter.txt

regex-urlfilter.txt

+^http://([a-z0-9]*\.)*nutch.apache.org/

hadoop.log

hadoop.log

http://ideone.com/LnpAw4

我在日志文件中没有看到任何错误。我真的迷路了。任何帮助，将不胜感激。谢谢！

I don't see any errors in the log file. I am really lost. Any help would be appreciated. Thanks!

推荐答案

最近我遇到了类似的问题。我认为您可以尝试以下步骤找出问题所在。

I got a similar problem recently. I think you can try the following steps to find out the problem.

1做一些测试以确保数据库运行良好。

1 Do some tests to make sure the DB works well.

2不必分批运行爬网，而是可以逐步调用nutch并观察日志更改以及数据库内容（尤其是新的URL）的更改。

2 Instead of running the crawl in batch, you can call nutch step by step and watch the log change as well as the change of DB content, in particular, the new urls.

3关闭solr并专注于小节和DB。

3 Turn off solr and focus on nutch and the DB.

这篇关于使用Nutch 2.3，Cassandra 2.0和solr 4.10.3进行爬网将返回0个结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！