NodeJS + 请求 - 请求网站时拒绝访问 [英] NodeJS + Request - Access denied when requesting website
问题描述
我正在尝试使用 request 请求网站的 html，但我一直收到拒绝访问错误.我该如何度过这一关?下面是函数的代码:
I'm trying to request the html of a website using request but I keep getting an access denied error. How do I get past this? Here is the code for the function below:
const request = require('request');
function firstShoe() {
request('https://www.jdsports.co.uk/product/green-nike-vapormax/281735/', function (error, response, body) {
console.log('body:', body);
});
}
错误:
</BODY>
</HTML>
body: <HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>
You don't have permission to access "http://www.jdsports.co.uk/product/green-nike-vapormax/281735/" on this server.<P>
Reference #18.609d3e17.1500116386.15f0cb85
</BODY>
</HTML>
通过将用户代理传递到标头中找到了解决方案.
Found a solution by passing the user-agent into the headers.
function firstShoe() {
var options = {
headers: {'user-agent': 'node.js'}
}
request('https://www.jdsports.co.uk/product/green-nike-vapormax/281735/', options, function (error, response, body) {
console.log(body);
message.channel.send(body);
});
}
推荐答案
您收到 403 Forbidden 因为该网站正在阻止使用非通用用户代理发送的所有请求(基本上他们检查 User-Agent 标头).这是一个非常简单的保护来避免刮板.
You are getting a 403 Forbidden because that website is blocking all requests sent using non common user agents (basically they check User-Agent header). It is a very simple protection to avoid scrappers.
例如，如果您使用其标准 User-Agent 发送以下 cURL，则可以完美接收响应:
For example, if you send the following cURL using its standard User-Agent, the response is received perfectly:
curl -v 'https://www.jdsports.co.uk/product/green-nike-vapormax/281735/'
然而，如果您重复该请求并指定一个不存在的用户代理，该请求将被阻止:
Nevertheless, if you repeat that request specifying a non existing User-Agent, the request is blocked:
curl -v 'https://www.jdsports.co.uk/product/green-nike-vapormax/281735/' -H 'User-Agent: StackOverflow'
这篇关于NodeJS + 请求 - 请求网站时拒绝访问的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
