解析在文档加载中存在Ajax请求的网页的HTML [英] Parse HTML of a webpage in which Ajax requests are present on document load

问题描述

我试图通过我的控制台应用程序获取一个网站的HTML，这一直工作正常。



现在页面中有一个部分通过AJAX调用加载到文档就绪事件上。现在在我的对象中，我没有得到这个内容。



我如何获得通过ajax加载的内容。



在此先感谢。

Hi,

I am trying to fetch the HTML of one website through my console application and this is working fine till now.

Now there is one section in page which is being loaded on document ready event through an AJAX call. Now in my object, i am not getting this content.

How can i get such content which are loaded through ajax.

Thanks in advance.

推荐答案

Ajax调用需要运行javascript VM。您将无法使用.Net WebRequest对象获取Ajax结果，因为如果不将Javascript VM链接到DOM管理框架，您将无法执行文档的Javascript。 A.k.a a Browser。



您有三种选择。



尝试编写查看页面javascript的代码，并允许您模拟AJAX调用并集成内容。但这不是一个很好的方法，对页面的任何更改都可以完全抛弃你的代码。



Fiddler是一个实现man-in-的工具中间过程并记录所有请求和响应，包括AJAX。您可以使用此浏览网站并重新编码结果。我不确定Fiddler如何实现它的功能，但你可以设法编写类似的过程。



更好的选择是使用.Net WebBrowser控件。您可以使用此控件来访问当前页面文本。加载页面等待足够长的时间让AJAX完成，然后从浏览器控件中获取内容。

Ajax calls require a javascript VM to run. You will not be able, using .Net WebRequest objects, to get the Ajax result as you will not be able to execute the Javascript for the document without linking a Javascript VM to a DOM management framework. A.k.a a Browser.

You have three options.

Try and write code which looks at the pages javascript and allows you to emulate the AJAX call an integrate the content. But this is not a nice way of doing it and any change to a page could complete throw off your code.

Fiddler is a tool which implements a man-in-the-middle process and logs all requests and responses, including AJAX. You could browser the site with this and recode the results. I'm not sure how Fiddler achieves what it does but you could aim to write a similar process.

The better option is to use the .Net WebBrowser control. You can use this control to access the current page text. Load the page wait long enough for the AJAX to complete and then get the content from the browser control.

嗨Stephen

感谢您的回复，但我试过了网页浏览器的东西已经......（并且它是我在我的场景中唯一可以实现的东西）

现在我放置了一个像1000000这样的巨大延迟，但仍然是ajax内容未来，这就是为什么我不得不来到这里...... :(

Hi Stephen
Thanks for your reply but i tried that web browser stuff already...(nd its the only thing which i can implement in my scenario)
Now i placed a huge delay like 1000000 but still ajax content is not coming that is why i had to come here... :(

我已经尝试了一件事...

i猜这一个对我有用......



我接受了硒的帮助，它为我做了一件事...... :)

I have tried one more thing...
i guess this one is working for me...

I took help of selenium and it did the thing for me.. :)

这篇关于解析在文档加载中存在Ajax请求的网页的HTML的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！