Java的带有刮一JSoup动态网站 [英] Java-Scrape a dynamic website with JSoup

问题描述

我想用刮一JSoup网站。这个网站是动态的，每秒更新左右。我是pretty确保它使用jQuery，这在HTML更新一些标记。我使用JSoup，但动态领域的空白。是否JSoup有什么特别的解决这个问题？或者，这是一个限制？
编辑：我打算把这个Java小程序在我的网站

I would like to scrape a website with JSoup. This website is dynamic and updates every second or so. I'm pretty sure it uses JQuery, which updates some tags in the HTML. I'm using JSoup, but the dynamic areas that are blank. Does JSoup have anything special to fix this problem? Or is this a limitation? I'm planning to put this in a Java applet on my website.

推荐答案

听起来像是你想JSoup表现得像个支持JavaScript的浏览器。
这是行不通的，我害怕。 JSoup是可以执行一个HTTP请求，然后使用响应对身体有益的东西的工具。

Sounds like you want JSoup to behave like a browser with JavaScript support. That won't work, I'm afraid. JSoup is a tool that can execute a HTTP request, and then use the response body for something useful.

这一些有用的东西是从响应的（X）HTML文本中提取信息。
如果你想一个JavaScript，注入HTML页面（=动态网页）的负荷下后续的Ajax请求的内容，您需要模拟这些后续请求自己并指示JSoup手动执行那些适合你。

This 'something useful' is to extract information from the (X)HTML text in the response. If you want to the contents of subsequent ajax-request following the loading of a JavaScript-infused HTML page (=dynamic web page) you'll need to model those follow-up requests yourself and instruct JSoup to execute those for you manually.

这篇关于Java的带有刮一JSoup动态网站的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！