如何在Android中执行Web Scraping？ [英] How do I perform Web Scraping in Android?

问题描述

我想抓取我的网站，然后使用网站上的数据填充我的应用中的元素，我的网站有登录页面，某些页面只有在登录完成后才会打开。

I want to scrape my website and then use the data from the website to populate elements in my app, my website has login pages and certain pages only open after the login has been done.

我开始使用HtmlUnit，因为它是一个无头浏览器并在java IDE中完成了自定义api，后来我尝试使用从Java IDE生成的jar，发现HtmlUnit存在不兼容问题Android。

I started working with HtmlUnit as it is a headless browser and completed the custom api in a java IDE, later i tried to use the jar i generated from the java IDE and found that there are incompatibility issues with HtmlUnit and Android.

有人可以提出这个问题的解决方案吗？

Can anyone propose a solution to this problem?

编辑
因为没有人真正回答这个问题我目前正在使用android的本机WebView，将其Visibility设置为不可见，然后使用javascript连接到Java对象，我可以注入JS代码来抓取任何数据。 / p>

Edit : Since no one actually answered this question I am currently going with a work around using android's native WebView, settings its Visibility to invisible and then using javascript interfacing to a Java object, I can inject JS code to scrape any data.

推荐答案

使用 Jsoup 为此目的的图书馆。非常方便易用。
从此答案开始，并按照文档和其他示例进行操作。

Use Jsoup library for such purpose. Very handy and easy to use. Start with this answer and follow documents and other examples.

这篇关于如何在Android中执行Web Scraping？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！