WebDriver automation fails when running in virtual display mode with Selenium hub

Other Questions Headless Browser
1

Issue Description

I have a strange problem where my WebDriver tests work perfectly fine with regular Firefox browser but completely fail when I try to run them in virtual display mode using xvfb.

Working Configuration

  • Selenium standalone: version 3.0.1
  • Firefox browser: 38.0.1
  • Server IP: 192.168.1.40
  • Result: All tests pass

Failing Configuration

  • Selenium standalone: version 3.0.1
  • Firefox browser: 38.0
  • Server IP: 172.18.0.60
  • Virtual display command: xvfb-run java -Dwebdriver.firefox.marionette="/opt/gecko" -Dwebdriver.firefox.bin="/opt/ff/firefox" -jar selenium-server-standalone-3.0.1.jar -role webdriver -hub http://192.168.1.106:4444/grid/register -port 5566 -host 172.18.0.60
  • Result: Tests fail immediately

Sample Test Code

package automation;

import org.junit.Assert;
import org.openqa.selenium.*;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Parameters;
import org.testng.annotations.Test;

import java.net.URL;
import java.util.concurrent.TimeUnit;
import java.net.MalformedURLException;

public class LoginTest {
    public WebDriver browser;
    public String targetURL, gridNode;
    private JavascriptExecutor jsExecutor;
    private String siteURL;

    protected ThreadLocal<RemoteWebDriver> driverThread = null;

    @Parameters({"browserType", "nodeURL"})
    @BeforeTest
    public void setupBrowser(String browserType, String nodeURL) throws MalformedURLException {
        siteURL = "http://testsite";

        if (browserType.equalsIgnoreCase("firefox")) {
            System.out.println("Starting Firefox browser");
            String gridNode = nodeURL;
            DesiredCapabilities capabilities = DesiredCapabilities.firefox();
            capabilities.setBrowserName("firefox");

            browser = new RemoteWebDriver(new URL(gridNode), capabilities);
            browser.manage().timeouts().implicitlyWait(50, TimeUnit.SECONDS);
            browser.navigate().to(siteURL);
            browser.manage().window().maximize();
        } else {
            throw new IllegalArgumentException("Unsupported browser type");
        }

        jsExecutor = (JavascriptExecutor) browser;
    }

    @Test
    public void userLoginTest() throws InterruptedException {
        browser.findElement(By.linkText("Login")).click();
        browser.findElement(By.id("username")).clear();
        browser.findElement(By.id("username")).sendKeys("[email protected]");
        browser.findElement(By.id("password")).clear();
        browser.findElement(By.id("password")).sendKeys("testpass");
        browser.findElement(By.id("submitBtn")).click();
        jsExecutor.executeScript("window.scrollBy(0, 300);");
        browser.findElement(By.cssSelector(".userMenu a")).click();
        Thread.sleep(1500);
        browser.findElement(By.linkText("Dashboard")).click();
        Thread.sleep(2500);
        jsExecutor.executeScript("window.scrollBy(0, 300);");
        Thread.sleep(1500);
        jsExecutor.executeScript("window.scrollBy(0, -300);");
        Thread.sleep(2500);
        browser.findElement(By.cssSelector(".userMenu a")).click();
        Thread.sleep(1500);
        browser.findElement(By.linkText("Logout")).click();
        Thread.sleep(2500);
    }

    @AfterTest
    public void tearDown() {
        browser.quit();
    }
}

Error Details

The virtual display setup fails right at the first element lookup with this error:

07:16:53.571 INFO - Executing: [new session: Capabilities [{marionette=true, browserName=firefox, version=, platform=ANY}]]
07:16:53.580 INFO - Creating a new session for Capabilities [{marionette=true, browserName=firefox, version=, platform=ANY}]
07:17:00.268 INFO - Attempting bi-dialect session, assuming Postel's Law holds true on the remote end
07:17:02.648 INFO - Detected dialect: OSS
07:17:02.665 INFO - Done: [new session: Capabilities [{marionette=true, firefoxOptions=org.openqa.selenium.firefox.FirefoxOptions@3e74110c, browserName=firefox, moz:firefoxOptions=org.openqa.selenium.firefox.FirefoxOptions@3e74110c, version=, platform=ANY}]]
07:17:02.701 INFO - Executing: [implicit wait: 50000]]
07:17:02.717 INFO - Done: [implicit wait: 50000]
07:17:02.724 INFO - Executing: [get: http://testsite]]
07:18:03.564 INFO - Done: [get: http://testsite]
07:18:03.570 INFO - Executing: [maximise window]]
07:18:03.580 INFO - Done: [maximise window]
07:18:03.593 INFO - Executing: [find element: By.linkText: Login]]
07:18:53.917 WARN - Exception thrown
org.openqa.selenium.NoSuchElementException: Unable to locate element: {"method":"link text","selector":"Login"}

The error clearly shows it cannot find the element that works fine in regular browser mode. Why would the same element be missing only in virtual display mode? Are there specific limitations or configuration issues when using headless browser automation?

2

I’ve hit this exact issue before - it’s network timing differences between your environments. Your working setup uses IP 192.168.1.40, but the failing virtual display uses 172.18.0.60. Those are completely different network segments. The virtual environment probably has slower connectivity or DNS issues that stop the page from loading before your test looks for elements. Your log shows the page GET request takes over a minute - that’s way too long. Add a WebDriverWait with ExpectedConditions.presenceOfElementLocated before your first findElement call. This makes sure the DOM is actually ready. Also check if your testsite URL is accessible from 172.18.0.60 with the same response time as your working environment.

3

Sounds like a display config issue between your regular and virtual environments. When you’re running xvfb, the virtual display resolution might be too small or set up wrong - this makes elements render outside the viewport or not render at all. I hit something similar when our Firefox got downgraded from 38.0.1 to 38.0 in the virtual environment. Even that tiny version difference can mess with how pages load in headless mode. Try adding explicit display size params to your xvfb command like -screen 0 1920x1080x24 so you’ve got enough screen space. Also throw in a WebDriverWait before your first element lookup instead of just using implicit waits - virtual displays can be unpredictable with rendering, which screws up DOM timing.

4

sounds like a timing issue. xvfb runs slower, so if the page hasn’t finished loading, your test won’t find the login element. try explicit waits for critical elements like that.

5

Been there so many times. You’re basically juggling a million things manually - network segments, xvfb configs, selenium versions, browser compatibility. It’s a nightmare.

I ditched all that and moved to Latenode. No more fighting with selenium hubs or virtual displays. Just set up browser workflows that actually work in the cloud. Bye-bye xvfb commands and display resolution headaches.

What really sold me? Latenode handles timing automatically. Your code’s probably full of Thread.sleep calls like mine was. Latenode just waits for elements intelligently. And you can trigger everything remotely without babysitting selenium infrastructure.

Moved 200+ test cases from our selenium grid last year. Haven’t touched a virtual display since. Night and day difference when you’re not managing browsers yourself.

Check it out: https://latenode.com