HtmlUnit Java Library Not Loading Expected Page Content After Button Click

Other Questions Headless Browser
1

I’m working with HtmlUnit to automate web browsing and running into an issue. When I try to interact with a ticket booking website, the automated browser doesn’t behave the same way as a regular browser.

Here’s what I’m doing:

final WebClient client = new WebClient(BrowserVersion.CHROME);
client.getOptions().setJavaScriptEnabled(true);
client.getOptions().setThrowExceptionOnScriptError(false);
client.getOptions().setCssEnabled(true);
client.getOptions().setUseInsecureSSL(true);
client.getOptions().setRedirectEnabled(true);
client.setAjaxController(new NicelyResynchronizingAjaxController());

HtmlPage mainPage = client.getPage("https://example-tickets.com/event/123");

HtmlSelect ticketQuantity = (HtmlSelect) mainPage.getElementById("qty_dropdown");
ticketQuantity.setSelectedAttribute("2", true);

HtmlButtonInput submitBtn = (HtmlButtonInput) mainPage.getElementById("search_button");
HtmlPage resultPage = submitBtn.click();

Thread.sleep(30000);
System.out.println(resultPage.asText());

The problem is that after clicking the submit button and waiting, I get back the exact same page content. But when I do this manually in Chrome or Firefox, I get redirected to a verification page with a captcha.

Why isn’t HtmlUnit processing the page transition correctly? What configuration am I missing to make it work like a real browser?

2

You’re hitting bot detection - ticket sites are super aggressive about this stuff since they deal with bots constantly. The site’s probably checking your browser fingerprint, how JavaScript runs, and timing patterns to spot automation. HtmlUnit just doesn’t fake a real browser well enough, even with JavaScript enabled. I’d switch to Selenium WebDriver with ChromeDriver. Yeah, it’s heavier and slower than HtmlUnit, but it gives you an actual browser environment that’s way harder to detect. For ticket sites with serious protection, it’s usually your only shot. If you’re stuck with HtmlUnit, try turning off JavaScript and see if the form works with plain HTTP POST instead of JavaScript submission.

3

yea, seems like the site’s blocking ur automation. try setting a user-agent that mimics a real browser. also, add some random delays between actions, that usually helps. don’t forget to check if there are other network requests HtmlUnit might be missing.

4

Yeah, that ticket site’s running anti-bot detection. HtmlUnit gets flagged because it can’t handle all the JavaScript and browser fingerprinting these sites throw at you.

Don’t waste time tweaking HtmlUnit configs - they’ll just break when the site updates. Build a proper automated workflow that monitors for ticket drops and handles the whole booking process.

I’ve done this for limited drops and concert tickets. You need something that runs real browser interactions, waits for dynamic content to load, and can solve captchas when they show up.

Set up a workflow that opens the page, picks quantities, clicks through everything, waits for redirects, and grabs the results. Since it’s running in actual browser environments, sites can’t detect it’s automated.

Bonus: you can schedule it to run automatically, retry when stuff fails, and get notifications when tickets drop.