最好的方法来移动到带有请求承诺的新页面? [英] Best method of moving to a new page with request-promise?

问题描述

我正在修改请求诺言以抓取朋友的网页.我在他们的GitHub上使用更好地抓取网页示例.到目前为止，我的情况是这样:

I am tinkering around with request-promise to crawl a friends webpage. I am using the crawl a webpage better example on their GitHub. What I have so far is this:

var rp = require('request-promise');
var cheerio = require('cheerio'); // Basically jQuery for node.js

var options = {
  uri: 'https://friendspage.org',
  transform: function(body) {
    return cheerio.load(body);
  }
};

rp(options)
  .then(function($) {
    // Process html like you would with jQuery...
    var nxtPage = $("a[data-url$='nxtPageId']").attr('data');

    // How do I use nxtPage here to go to that site

  })
  .catch(function(err) {
    // Crawling failed or Cheerio choked...
  });

转到nxtPage中的链接的正确方法是什么?我仍然希望能够在其上使用cheerio/jQuery.我需要在当前then函数中重复整个var option = ...事情吗?

What is the proper way to go to the link I have in nxtPage? I still want to be able to use cheerio/jQuery on it. Do I need to repeat the whole var option = ... thing inside the current then function?

推荐答案

您可以创建自己的实用程序函数来创建您的选项，然后像这样调用rp():

You can just create your own utility function that creates your options and then calls rp() like this:

const rp = require('request-promise');
const cheerio = require('cheerio'); // Basically jQuery for node.js

// shared function
function getPage(url) {
    const options = {
        uri: url,
        transform: function(body) {
          return cheerio.load(body);
        }
    };
    return rp(options);
}

getPage('https://friendspage.org').then($ => {
    // Process html like you would with jQuery...
    const nxtPage = $("a[data-url$='nxtPageId']").attr('data');
    return getPage(nxtPage).then($ => {
        // more processing here
    });
}).catch(err => {
    console.log(err);
    // error handling here
});

这只是要在共享功能的多个位置使用的分解代码.与rp()或cheerio无关，仅是Java语言(或任何语言)中的常规代码分解.

This is just factoring code that you want to use in multiple places into a shared function. Nothing in particular to do with rp() or cheerio, just regular code factoring in Javascript (or any language).

这篇关于最好的方法来移动到带有请求承诺的新页面?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！