页面内容加载了 JavaScript 而 Jsoup 看不到它 [英] Page content is loaded with JavaScript and Jsoup doesn't see it

问题描述

页面上的一个块由 JavaScript 填充了内容，在使用 Jsoup 加载页面后，没有任何信息.使用 Jsoup 解析页面时，有没有办法获取 JavaScript 生成的内容?

无法在此处粘贴页面代码，因为它太长:http://pastebin.com/qw4Rfqgw

这是我需要的内容元素:<div id='tags_list'></div>

我需要用 Java 获取这些信息.最好使用 Jsoup.元素是在 JavaScript 的帮助下的字段:

<a href="/tagsc0t20099.html" style="font-size:14;">разведчик</a><a href="/tagsc0t1879.html" style="font-size:14;">Sr</a><a href="/tagsc0t3140.html" style="font-size:14;">стратегический</a>

Java 代码:

import org.jsoup.Jsoup;导入 org.jsoup.nodes.Document;导入 org.jsoup.nodes.Element;导入 org.jsoup.select.Elements;导入 java.io.IOException;公开课测试{公共静态无效主(字符串[] args){尝试{文档 Doc = Jsoup.connect( "http://www.bestreferat.ru/referat-32558.html" ).get();Elements Tags = Doc.select("#tags_list a");对于(元素标签:标签){System.out.println( Tag.text() );}}捕获 ( IOException e ){e.printStackTrace();}}}

解决方案

JSoup 是一个 HTML 解析器，而不是某种嵌入式浏览器引擎.这意味着它完全不知道在初始页面加载后 Javascript 添加到 DOM 的任何内容.

要访问那种类型的内容，您需要一个嵌入式浏览器组件，关于这种组件的 SO 上有很多讨论，例如 有没有办法在 Java 中嵌入浏览器?

One block on the page is filled with content by JavaScript and after loading page with Jsoup there is none of that inforamtion. Is there a way to get also JavaScript generated content when parsing page with Jsoup?

Can't paste page code here, since it is too long: http://pastebin.com/qw4Rfqgw

Here's element which content I need: <div id='tags_list'></div>

I need to get this information in Java. Preferably using Jsoup. Element is field with help of JavaScript:

<div id="tags_list">
    <a href="/tagsc0t20099.html" style="font-size:14;">разведчик</a>
    <a href="/tagsc0t1879.html" style="font-size:14;">Sr</a>
    <a href="/tagsc0t3140.html" style="font-size:14;">стратегический</a>
</div>

Java code:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

import java.io.IOException;

public class Test
{
    public static void main( String[] args )
    {
        try
        {
            Document Doc = Jsoup.connect( "http://www.bestreferat.ru/referat-32558.html" ).get();
            Elements Tags = Doc.select( "#tags_list a" );

            for ( Element Tag : Tags )
            {
                System.out.println( Tag.text() );
            }
        }
        catch ( IOException e )
        {
            e.printStackTrace();
        }
    }
}

解决方案

JSoup is an HTML parser, not some kind of embedded browser engine. This means that it's completely unaware of any content that is added to the DOM by Javascript after the initial page load.

To get access to that type of content you will need an embedded browser component, there are a number of discussions on SO regarding that kind of component, eg Is there a way to embed a browser in Java?

这篇关于页面内容加载了 JavaScript 而 Jsoup 看不到它的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！