HTTPREQUEST没有给出预期的结果 [英] HTTPREQUEST not giving expected result

问题描述

我正在爬行一个网站.
我正在抓取的网址是"http://www.dealsandyou.com/catalog/product/citysitedeal/productId/8060".
在浏览器中打开时，该URL显示ahmedab​​ad链接. IE.指向第一款产品BlackBerry Smartphones的链接是"http://www.dealsandyou.com/blackberry-smartphones-ahmedab​​ad-26092011.html".但是，当您在asp.net中读取此URL时，它总是将其他城市的数据显示为"http://www.dealsandyou.com/bluedio-stereo-i4-headset-othercities-29092011.html".那是其他城市的产品.我不明白你在说什么.请帮助我&给我一些解决方案.
我的源代码有点像:

I am crawling a website.
The url that i am crawling is "http://www.dealsandyou.com/catalog/product/citysitedeal/productId/8060".
This url shows ahmedabad links when opened in browser. ie. link to first product BlackBerry Smartphones is "http://www.dealsandyou.com/blackberry-smartphones-ahmedabad-26092011.html". But when u read this url in asp.net it always gives data of other cities as "http://www.dealsandyou.com/bluedio-stereo-i4-headset-othercities-29092011.html". That is the products for other cities. I cant understand y is it show. Please help me & give me some solution for this.
My source code is somewhat like :

 private static HttpWebRequest _request;
 private static HttpWebResponse response;
 StreamReader strreader = null;

 _request = (HttpWebRequest)WebRequest.Create("http://www.dealsandyou.com/catalog /product/citysitedeal/productId/8060");
            
 response = (HttpWebResponse)_request.GetResponse();

             
 //create a streamreader object from the response 
 strreader = new StreamReader(response.GetResponseStream());

//reading data in string. This string always give data of other cities
 string strdata = strreader.ReadToEnd();

您也可以检查"http://www.dealsandyou.com/catalog/product/citysitedeal/productId/8085".这就是Banglore产品的网址.抓取时还将显示其他城市的数据.

请帮我. 非常紧急.

You can also check for "http://www.dealsandyou.com/catalog/product/citysitedeal/productId/8085". That is the url for banglore products. It will also show other cities data when crawled.

Please help me. Its very urgent.

推荐答案

使用类似 Fiddler2 [ ^ ]，您必须分析通过浏览器完成后，Web请求中发送了哪些其他信息(标题).在程序中生成此Web请求时，您必须确保将此附加信息也发送到您要从中请求信息的服务器.

有关如何将HTTP headers添加到您的Web请求的信息，请参见此处: http://msdn.microsoft.com/en-us/library/system.net.webrequest.headers.aspx [

Using a tool like Fiddler2[^] you''ll have to analyze what additional information (headers) is sent in the web request when it''s done via browser. When generating this web request in your program you''ll have to make sure that this additional information is also sent to the server you''re requesting the information from.

See here on how to add HTTP headers to your web request: http://msdn.microsoft.com/en-us/library/system.net.webrequest.headers.aspx[^].

Regards,

—MRB

这篇关于HTTPREQUEST没有给出预期的结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！