WebRequest返回比浏览器更多的来源>查看源代码 [英] WebRequest returning more source than Browser > view source

问题描述

我正在用C＃编写一个Selenium实用程序来帮助进行自动化测试。我可能会在这里错过一些完全明显的东西，但为什么如果我发送一个HttpWebRequest到服务器并检索响应流，我最终会得到更多的源代码（比如Selenium的更多可映射的WebElements）点击 - 在我的浏览器中查看页面源代码？

这意味着我映射Selenium驱动程序无法找到的元素来运行。

我很迷惑：S

  HttpWebRequest req =（HttpWebRequest）WebRequest.Create（url）; 
 HttpWebResponse res =（HttpWebResponse）req.GetResponse（）; 
 StreamReader sr = new StreamReader（res.GetResponseStream（））; 
 
 var src = sr.ReadToEnd（）; 
  

解决方案

有很多可能性。一个可能的罪魁祸首是您发送到服务器的信息不同。浏览器通常会发送不同的标题，cookie等，除非您明确地添加它，否则web请求不会有。

例如，当我点击www.google.com通过我的网络浏览器，我可以从Google Plus获得东西，并且我可以获得最好的体验，因为我使用的是常青树浏览器。大约139000个字符出现在我的查看源页面中。

但是，当我对同一个URL执行web请求时（使用下面的代码），我只有45000个字符响应流：

  async void Main（）
 {
 var result = await GetTextAsync（https ：//www.google.com）; 
 Console.Write（result.Length）; 
} 
 
公共异步任务< string> GetTextAsync（string url）{
 var result = await WebRequest.Create（url）.GetResponseAsync（）; 
 using（var stream = result.GetResponseStream（））
 using（var reader = new StreamReader（stream））
 {
 return await reader.ReadToEndAsync（）; 
} 
} 
  

我怀疑如果我要将WebRequest设置为发送Chrome发送给Google的所有相同的Cookie和标头，我的结果会更加类似。

I am writing a Selenium utility in C# to help with automated testing. I may be missing something completely obvious here, but why is it that if I send an HttpWebRequest off to a server and retrieve the response stream, I end up with more source (i.e more mappable WebElements for Selenium) than I get when I right-click - view page source in my browser?

This means that I am mapping elements that the Selenium driver cant find come runtime.

Me so confuse :S

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
StreamReader sr = new StreamReader(res.GetResponseStream());

var src = sr.ReadToEnd();

解决方案

There are a number of possibilities. One likely culprit is the difference in the information you're sending to the server. Browsers typically send various headers, cookies, etc., that a web request does not have unless you explicitly add it.

For example, when I hit www.google.com with my web browser, I get stuff from Google Plus, and I get the fanciest experience possible because I'm on an evergreen browser. Roughly 139000 characters appear in my View Source page.

However, when I do a web request to the same URL (using the following code), I get only 45000 characters in the response stream:

async void Main()
{
    var result = await GetTextAsync("https://www.google.com");
    Console.Write(result.Length);
}

public async Task<string> GetTextAsync(string url){
    var result = await WebRequest.Create(url).GetResponseAsync();
    using (var stream = result.GetResponseStream())
    using (var reader = new StreamReader(stream))
    {
        return await reader.ReadToEndAsync();
    }
}

I suspect that if I were to set the WebRequest to send all the same cookies and headers that Chrome is sending to Google, my results would be much more similar.

这篇关于WebRequest返回比浏览器更多的来源>查看源代码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！