如何在 Android 中执行 Web Scraping? [英] How do I perform Web Scraping in Android?

问题描述

我想抓取我的网站，然后使用该网站的数据填充我的应用程序中的元素，我的网站有登录页面，某些页面仅在登录完成后打开.

I want to scrape my website and then use the data from the website to populate elements in my app, my website has login pages and certain pages only open after the login has been done.

我开始使用HtmlUnit，因为它是一个无头浏览器，并在java IDE中完成了自定义api，后来我尝试使用我从java IDE生成的jar，发现HtmlUnit和Android存在不兼容问题.

I started working with HtmlUnit as it is a headless browser and completed the custom api in a java IDE, later i tried to use the jar i generated from the java IDE and found that there are incompatibility issues with HtmlUnit and Android.

有人能提出解决这个问题的方法吗?

Can anyone propose a solution to this problem?

由于没有人真正回答过这个问题，我目前正在尝试使用 android 的原生 WebView，将其 Visibility 设置为不可见，然后使用与 Java 对象接口的 javascript，我可以注入 JS 代码来抓取任何数据.

Edit : Since no one actually answered this question I am currently going with a work around using android's native WebView, settings its Visibility to invisible and then using javascript interfacing to a Java object, I can inject JS code to scrape any data.

推荐答案

为此目的使用 Jsoup 库.非常方便且易于使用.从这个答案开始，然后按照文档和其他示例进行操作.

Use Jsoup library for such purpose. Very handy and easy to use. Start with this answer and follow documents and other examples.

这篇关于如何在 Android 中执行 Web Scraping?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！