节点无法抓取某些页面 [英] Node can't scrape certain pages

问题描述

我不知道这是否与coldfusion页面有关还是什么，但我无法抓取这些.cfm页面

I don't know if this is something to do with coldfusion pages or what but I can't scrape these .cfm pages

在带有 request 的目录中的命令行中运行:

In the command line in a directory with request run:

node> var request = require('request');
node> var url = 'http://linguistlist.org/callconf/browse-conf-action.cfm?confid=173395';
node> request(url, function (err, res, body) { if (err) { console.log(err) } else { console.log('body:', body) }; });

我尝试过其他一些 .cfm 网站，但它们可以工作，但结果却是空白，所以我不知道可能是什么

I've tried with some other .cfm sites but they work, and am only getting blank results so I don't know what it could be

注意:我也尝试过使用准系统require('http').get(url,...) 路由，但我得到相同的空白结果

Note: I've also tried doing it the barebones require('http').get(url,…) route but I get the same blank result

推荐答案

我明白了！(终于)

这个网络服务器真的需要知道如何回答你.试试这个(它对我有用)

This web server really need to know how to answer to you. Try this (it work for me)

var request = require('request');
var options = {
  url: 'http://linguistlist.org/callconf/browse-conf-action.cfm?confid=173395',
  headers: {
   'Accept-Encoding':'none'
  }
};
request(options, function (err, res, body) { if (err) { console.log(err) } else { console.log('body:', body) }; });

这篇关于节点无法抓取某些页面的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！