I want to build an iOS app that can visit websites automatically, enter login details, and grab specific data from pages. Someone suggested using a headless browser (basically a browser that works behind the scenes without showing anything to the user).
I know there are tools like HTMLUnit for other platforms, but I can’t seem to find anything similar that works with Objective-C or Swift for iOS apps. Has anyone worked with this kind of web automation on iOS before? What libraries or frameworks would you recommend for this type of task?
just dealt with this a few months back. ended up using WKWebView with a workaround - made a 1x1 pixel webview and presented it modally with a clear background. acts like headless but technically still has a view. way easier than fighting the system and apple won’t flag it during review.
WKWebView is your best bet for iOS automation. I’ve built similar stuff for scraping authenticated web pages. Just set it up with zero frame dimensions in a hidden container - no UI needed. Use evaluateJavaScript to inject login credentials and pull data once pages load. The trick is getting the navigation delegate methods right so you know when page transitions happen and content’s ready to scrape. I add delays between actions since some sites load dynamically - makes it way more reliable. Heads up though - Apple’s pretty strict about automated web stuff in their App Store guidelines. Double-check your use case complies before you submit.
Hit this same issue last year building an app to monitor our company dashboards. Tried a bunch of different approaches before landing on Safari View Controller with background implementation + JavaScript injection. Works way better than WKWebView because it actually handles auth flows properly - shares cookies with Safari. You can run it off-screen and control everything through delegate methods. For pulling data out, don’t use fixed delays. Wait for specific DOM elements instead - throw a MutationObserver into your injected JS to catch when content actually loads. Pro tip: some sites detect automation by checking viewport size, so set realistic frame dimensions even on hidden views. Been running this in production for months without issues.