如何使用Java或Jsoup获取网页的移动响应 [英] How to get mobile response of webpage using java or jsoup
问题描述
我正在尝试将Java与JSoup结合使用来获得youtube.com的响应.
I am trying to get response of youtube.com using java with JSoup.
我可以使用JSoup获得youtube的响应，如下所示，它返回桌面网站的响应
I am able to get the response of youtube using JSoup as follows, it returns the desktop website's response
String str = "https://www.youtube.com/";
doc = Jsoup.connect(str)
.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36")
.get();
以同样的方式，我试图按以下方式将移动版本的响应发送到同一网站，
Same way, I am trying to get the response for mobile version to this same site as follows,
doc = Jsoup.connect("https://"+url2.getHost()+"/search?q="+q)
.userAgent("Mozilla/5.0(Linux; U; Android 2.2; en-gb; LG-P500 Build/FRF91) AppleWebKit/533.0 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1")
.get();
但这只会给出台式机/笔记本电脑版本的响应，而不能给出移动设备的响应.
But this gives only desktop/laptop version response and not the mobile response.
如何从jsoup获取移动响应.
How to get the mobile response from jsoup.
谢谢.
推荐答案
我认为您的User-Agent可能不太正确.
I think that maybe your User-Agent isn't quite correct.
我刚刚尝试了以下方法，并且似乎访问了移动YouTube网站:
I've just tried it with the following and appear to have hit the mobile YouTube site:
String mob = "https://m.youtube.com/";
mobile = Jsoup.connect(mob)
.userAgent("Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1")
.get();
更新 我已经详细了解了一下，我相信一旦DOM加载后，页面内容就会被某些Javascript修改.
Update I've had a look in more detail and I believe that the page content is being modified by some Javascript once the DOM has loaded.
看看上面的JSoup代码返回的HTML，我得到了这一点(请注意content div为空):
Looking at the HTML returned by the JSoup code above I get this (note the content div is empty):
<body id="body" class="atom fusion-tn">
<div id="player"></div>
<div id="guide-layout-container">
<div id="guide-container"></div>
<div id="content-container">
<div id="content"></div>
</div>
<div id="guide-overlay"></div>
<div id="lightbox"></div>
<div id="toast"></div>
<div id="content-overlay"></div>
</div>
<div id="_yt_orientation_detect"></div>
</body>
与在Chrome开发工具中查看的HTML相比，我看到了:
Comparing to the HTML viewed in Chrome's dev tools I see this:
JSoup只是HTML解析器，而不是Web浏览器.为了达到您的要求，我认为您可能需要查看以下内容
JSoup is just an HTML parser, not a web browser. In order to do achieve what you require, I think you might need to look at something like this Is there a way to embed a browser in Java?
这篇关于如何使用Java或Jsoup获取网页的移动响应的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
