如何通过Scrubyt获取“下一页"链接 [英] How to get 'Next Page' link with Scrubyt

问题描述

我正在尝试使用Scrubyt从此页面获取详细信息

I'm trying to use Scrubyt to get the details from this page http://www.nuffieldtheatre.co.uk/cn/events/event_listings.php?section=events. I've managed to get the titles and detail URLs from the list, but I can't use next_page to get the scraper to go to the next page. I assume that's cause I'm not using the correct pattern for the next page link. I tried the string "Next Page", and I've also tried the XPath. Any other ideas?

代码如下:

require 'rubygems'
require 'scrubyt'

nuffield_data = Scrubyt::Extractor.define do
  fetch 'http://www.nuffieldtheatre.co.uk/cn/events/event_listings.php?section=events'

  event do
    title 'The Coast of Mayo'
    #url "href", :type => :attribute
    link_url
  end

  next_page "Next Page", :limit => 2


end

  nuffield_data.to_xml.write($stdout,1)

推荐答案

使用稍微不同的URL尝试一下:

Try this with a slightly different URL:

fetch 'http://www.nuffieldtheatre.co.uk/cn/events/event_listings.php'

scrubyt似乎在URL末尾出现?section = events"查询问题.

scrubyt seems to be having issues with "?section=events" query on the end of the URL.

当它寻找下一页时，它正在尝试返回此URL:

When it looks for the next page it is trying to return this URL:

http://www. nuffieldtheatre.co.uk/cn/events/?pageNum_rsSearch=1&totalRows_rsSearch=39&section=events

代替:

http://www.nuffieldtheatre.co.uk/cn/events/event_listings.php?pageNum_rsSearch=1&totalRows_rsSearch=39&section=events

删除URL末尾的查询字符串似乎可以解决此问题-您可能希望将其归档为错误.

Removing the query string on the end of the URL seems to fix this - you might want to file this as a bug.

这篇关于如何通过Scrubyt获取“下一页"链接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！