如何使用Java获取外部网页的XMLHTTPRequest响应文本? [英] How to get XMLHTTPRequest response text of external web page using Java?

问题描述

我努力解析来自外部网站的数据，例如 stackoverflow.com，仅使用 Java.为了了解这个网页的内容，我访问了 Chrome Development Tools 并在 XMLHTTPRequest response 中找到了我需要的所有信息！如果有用，则响应具有 JSON 格式.问题是如何仅使用 Java 而不使用 Servlet 来获取这些数据.

I struggled to parse data from external website, for example, stackoverflow.com, using only Java. To find out what this webpage is about I went to Chrome Development Tools and found there XMLHTTPRequest response with all the information I need! If its useful, response has a JSON format. The question is how to get this data using only Java and without Servlets.

推荐答案

不要试图抓取和解析网页，只需使用 StackExchange API 和标准 Java 工具来发出请求并获取原始数据.

Don't try to grab and parse the web page, just use the StackExchange API and standard Java tools to make a request and get the raw data.

如果您的实际问题与 SO 没有直接关系，那么，请查找相关站点的 API.否则，您将需要通过手动解析网页或在浏览器引擎中运行页面并在无头引擎中使用标准 JS 来获取数据来逐字抓取网页.

If your actual question isn't related to SO directly then, well, look for an API to the site in question. Otherwise you are going to need to literally scrape the web page by parsing it manually or running the page in a browser engine and using standard JS in a headless engine to get the data out.

这篇关于如何使用Java获取外部网页的XMLHTTPRequest响应文本?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！