在java中读取动态网页内容 [英] Reading dynamic web page content in java

问题描述

我需要阅读网页内容的帮助。目前我正在使用以下方法来阅读内容

I need help reading the contents of a webpage. Currently i am using the following method to read the contents

BufferedReader in = new BufferedReader(new InputStreamReader(page.openStream())); 
String inputLine;
while ((inputLine = in.readLine()) != null)
{Content = Content + inputLine;}

但是这种方法存在问题。 。一些jsp页面中有ajax，它随机更新一个网页的css类，就像这样
Javascript代码只是为了给出一个想法：

However with this method there is a problem. . some jsp pages have ajax in them which randomly updates a css class of a webpage like so Javascript code just to give an idea:

if (request.readyState === 4 && request.status === 200) 
{
var type = request.getResponseHeader("Content-Type");
$('.update').empty();
$('.update').append(request.responseText); //update the css class
}

因此读取此页面阅读器时的结果通过我上面提到的java方法我得到

So as a result when this page reader is read through my java method as mentioned above i just get

<div class="update"></div>

虽然在屏幕上这个类有一个值。
现在，如果我首先保存页面（通过单击Firefox中的save），那么jquery附加在CSS类中的值也是可见的。
是否有方法或方法可以通过保存页面来读取值或获取像firefox那样的值。我想用字符串中的Ajax值读取整个网页的内容。

although on the screen this class has a value. Now however if i save the page first (by clicking save as in Firefox) then the values appended in the CSS class by jquery are also visible. Is there a method or a way on how i could read the values or obtain the values like firefox does by saving the pages.. I want to read the contents of the entire webpage with the Ajax values present in the string.

一方面我读到这很难，因为浏览器渲染和执行JAvascript所以我想知道firefox有没有可能有帮助的api？任何建议将不胜感激。

On one side i read that this is difficult since the JAvascript in rendered and executed by the browser so i wanted to know does firefox have any apis that might help ? Any suggestions would be appreciated.

推荐答案

您可能会发现以下项目有用：

You may find the following project useful:

• HTMLUnit

• HTMLUnit

这里还有信息丰富的博客文章。

这篇关于在java中读取动态网页内容的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！