Grails中的Web爬网程序以计算页面深度 [英] Web Crawler in Grails to calculate page depth

问题描述

我正在制作搜寻器应用程序.我希望抓取网站并找到所检索网页的深度.我了解了不同的爬网和解析工具，但无济于事.他们似乎都没有提供支持来计算深度.我也不确定使用哪种爬网程序工具可以最接近所需的功能.任何帮助表示赞赏.

I am making a crawler application. I wish to crawl websites and find the depth of the webpages retrieved. I read about different crawling and parsing tools but to no avail. None of them seem to provide support to calculate the depth. I am also unsure about which crawler tool to use which can get closest to desired functionality. Any help is appreciated.

推荐答案

最重要的事情可能是您的域的映射(而不是解析器).

The most important thing is probably the mapping of your Domain (and not the parser).

因为，如果您使用的是树(有关维基百科的更多信息) ，很容易计算出网址的深度(最小深度).

Because, if you are using a tree (More information on wikipedia), it is easy to calculate the depth (the min depth) of your URL.

希望这会有所帮助.

这篇关于Grails中的Web爬网程序以计算页面深度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！