从URL集合中进行节点X射线爬网数据 [英] Node x-ray crawling data from collection of url

问题描述

我正在尝试在网站中刮取一个列表，该列表会导致其他具有相同格式的页面.

I'm trying to scrape a list in a site that leads to other pages that has the same formatting.

我能够创建所有a标签的集合，但是当我尝试访问页面集合时，我尝试使用它创建的键并没有添加到返回的对象中.

I was able to create a collection of all the a tags, but when I try to visit a collection of pages, the key I try to create with it doesn't get added in my returned object.

这是我要尝试处理堆栈溢出的一个示例:

Here's an example of what I'm trying to do with stack overflow:

var Xray = require('x-ray');
var x = Xray();
x('http://stackoverflow.com/', {
    title: x(['a@href'], 'title'),
}) (function(err, obj) {
    console.log(obj);
});

我希望我的obj.title是所有a href页面的标题列表，而不是一个空对象.

I'm expecting my obj.title to be a list of titles of all the a href pages, instead I just get an empty object.

但是，如果我只尝试使用第一个href，那么标题就没问题了.

However if I were to try just using the first a href then I get the title no problem.

var Xray = require('x-ray');
var x = Xray();
x('http://stackoverflow.com/', {
    title: x('a@href', 'title'),
}) (function(err, obj) {
    console.log(obj);
});

以前有人遇到过这个问题吗?

Has anyone run into this problem before?

推荐答案

我之前遇到过这个问题，而我的解决方案是这样的:

I ran into that problem before and my solution goes like this:

var Xray = require('x-ray');
var x = Xray();
x('http://stackoverflow.com/', {
    title: x('a', [{links:'@href'}])
}) (function(err, obj) {
    obj.forEach(function(links.link) {
        x(links.link, "title")(function(err, data){
                console.log(data) // should print the title
        });
});

让我知道您是否遇到任何问题.

Let me know if you run into any problems.

这篇关于从URL集合中进行节点X射线爬网数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！