HttpWebRequest返回"(403)禁止".错误 [英] HttpWebRequest returns " (403) Forbidden" Error

问题描述

我编写了一个xml采集器，用于从网站接收/解码xml文件.大多数情况下，它都能正常工作，但总是返回错误:

I wrote a xml grabber to receive/decode xml files from website. It works fine mostly but it always return error:

远程服务器返回错误:(403)禁止."

对于网站 http://w1.weather.gov/xml/current_obs/KSRQ.xml

我的代码是:

CookieContainer cookies = new CookieContainer();
HttpWebRequest webRequest = (HttpWebRequest)HttpWebRequest.Create(Path);
webRequest.Method = "GET";
webRequest.CookieContainer = cookies;
using (HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse())
{
    using (StreamReader streamReader = new StreamReader(webResponse.GetResponseStream()))
    {
        string xml = streamReader.ReadToEnd();
        xmldoc.LoadXml(xml);
    }
}

并且在GetResponse方法中引发了异常.我怎么知道发生了什么事?

And the exception is throw in GetResponse method. How can I find out what happened?

推荐答案

可能是您的请求缺少服务器所需的标头.我在浏览器中请求该页面，使用Fiddler记录了确切的请求，然后删除了 User-Agent 标头并重新发出了该请求.结果是403.

It could be that your request is missing a header that is required by the server. I requested the page in a browser, recorded the exact request using Fiddler and then removed the User-Agent header and reissued the request. This resulted in a 403 response.

服务器通常使用此方法来试图阻止脚本仿照您的网站； o)

This is often used by servers in an attempt to prevent scripting of their sites just like you are doing ;o)

在这种情况下，403响应中的服务器标头是"AkamaiGHost"，它表示来自Akamai的某些云安全解决方案的边缘节点.也许是WAF规则可以防止漫游器触发403.

In this case, the server header in the 403 response is "AkamaiGHost" which indicates an edge node from some cloud security solution from Akamai. Maybe a WAF rule to prevent bots is triggering the 403.

在 User-Agent 标头中添加任何值似乎都适用于该网站.例如，我将其设置为绝对不是屏幕抓取工具"，这似乎很好用.

It seems like adding any value to the User-Agent header will work for this site. For example I set it to "definitely-not-a-screen-scraper" and that seems to work fine.

通常，当您遇到此类问题时，通常可以使用浏览器工具或Fiddler之类的代理来查看实际的HTTP请求和响应.正如斯科特·汉塞尔曼(Scott Hanselman)所说

In general, when you have this kind of problem it very often helps to look at the actual HTTP requests and responses using browser tools or a proxy like Fiddler. As Scott Hanselman says

互联网不是黑匣子

The internet is not a black box

http://www.hanselman.com/blog/TheInternetIsNotABlackBoxLookInside.aspx

这篇关于HttpWebRequest返回"(403)禁止".错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！