Looking for JavaScript-Enabled Web Scraping Solutions on Android

[Neo_Movies]

I need help finding a good library for web scraping on Android that can handle dynamic content. Here’s what I’m trying to do:

My Requirements:

• Load web pages completely in the background without showing them
• Handle JavaScript and AJAX calls that load content after page loads
• Extract data using CSS selectors or XPath
• Eventually simulate clicks and form submissions

What I’ve Tested:

• Jsoup - Great for static HTML but can’t run JavaScript
• Built-in HTTP clients - Same JavaScript limitation as Jsoup
• HtmlUnit - Perfect features but won’t work on Android (missing java.awt and other dependencies)
• Rhino - Too complicated and not sure if it fits my needs
• Selenium WebDriver - Might work but seems hard to run headless on mobile

I really hoped HtmlUnit would work since it has everything I need. Has anyone found a working alternative that supports JavaScript execution on Android? I’m using Android Studio but can switch to Eclipse if that helps.

Any suggestions would be greatly appreciated!

[ZoeStar42]

Had the same problem building a price monitoring app last year. WebView works but it’s pretty limited for background stuff - Android kills invisible WebViews constantly. What actually saved me was a hybrid approach using Chrome Custom Tabs Service API in headless mode. You bind to the service and run JavaScript without any UI rendering. Setup’s trickier than WebView but handles memory way better and doesn’t die in the background. I also tried Node.js with Puppeteer on Termux, but that means users have to install extra stuff. For pure Android solutions, check out WebKit bridge libraries - they give you direct V8 access without all the WebView bloat.

[Alice45]

have u considered using a WebView with js on? it’s invisble and lets you run js for scrapes. works well for dynamic stuff. just set visibility to GONE and use evaluateJavascript() method.