Selenium and Chrome headless: JS-loaded content not appearing during Java tests

Other Questions Headless Browser
1

I’m having trouble with my Selenium tests for a Spring Boot website. The basic stuff works fine, but I can’t get the dynamic content to show up.

My page has a main div with four child divs. The content loads through a JS function called when the DOM is ready. I’ve set up Chrome options for headless testing and added a wait in my test case, but the content still won’t load.

Here’s a simplified version of my HTML:

<div id="container">
  <div id="section1"></div>
  <div id="section2"></div>
  <div id="section3"></div>
  <div id="section4"></div>
</div>

And the JS:

document.addEventListener("DOMContentLoaded", () => {
  fillSections();
});

In my Java code, I’ve got:

ChromeOptions opts = new ChromeOptions();
opts.addArguments("--headless", "--no-sandbox", "--enable-javascript");

WebDriverWait waiter = new WebDriverWait(driver, Duration.ofSeconds(60));
waiter.until(ExpectedConditions.visibilityOfElementLocated(By.id("section1_content")));

Am I doing something wrong with Selenium? Or should I change how I’m loading the content in JS? Any help would be great!

2

I’ve faced similar issues with dynamic content in Selenium tests. One thing that helped me was using JavaScript executor to check if the page is fully loaded. Here’s what worked for me:

JavascriptExecutor js = (JavascriptExecutor) driver;
waiter.until(webDriver -> js.executeScript("return document.readyState").equals("complete"));

This waits for the page to be fully loaded, including all JavaScript.

Another trick I found useful was to wait for a specific element that you know will be present after the dynamic content loads, rather than just waiting for visibility:

waiter.until(ExpectedConditions.presenceOfElementLocated(By.id("section1_content")));

Lastly, make sure your Chrome options include “–disable-gpu” along with the ones you’re already using. This solved some headless mode issues for me.

If these don’t work, you might need to dig deeper into how your JavaScript is interacting with the DOM. Hope this helps!

3

hey mate, have u tried using a different browser? sometimes chrome can be finicky with headless mode. i had luck with firefox in similar situations. also, maybe try adding a small delay before ur wait condition? like Thread.sleep(1000) or sumthin. just a thought!

4

I’ve encountered similar challenges with Selenium and dynamic content. One effective approach I’ve found is to implement a custom wait condition. Instead of relying on visibility or presence, you can create a condition that checks for the actual content you expect to see. Here’s an example:

waiter.until(driver -> {
    WebElement section1 = driver.findElement(By.id("section1"));
    return !section1.getText().isEmpty();
});

This waits until the content of section1 is non-empty, which should indicate that your JavaScript has run successfully.

Additionally, ensure your ChromeOptions include “–disable-dev-shm-usage”. This can help with stability in headless mode, especially in CI/CD environments.

If the issue persists, consider adding a small delay before your wait condition to give the JavaScript more time to execute. While not ideal, it can be a pragmatic solution in some cases.