Setting up headless browser automation with XVFB and Grunt on CentOS 7

Other Questions Headless Browser
1

I’m trying to set up headless testing but running into issues. My current setup works fine when running tests with UI using the regular command, but when I try to run in headless mode, the browser window still appears.

Here’s my current grunt configuration:

module.exports = function (grunt) {

require('load-grunt-tasks')(grunt);

  grunt.initConfig({
      webdriver: {
      options: {
          keepAlive: true,
          configFile: "../test-automation/config/settings.js",
          noColor: false,
          args: {
            baseUrl: 'https://myapp.com/test/'
          }
      },
      execute: {}
  },
  shell: {
      virtualDisplay: {
          command: 'Xvfb :98 -ac -screen 0 1920x1080x24',
          options: {
              async: true
          }
      }
  },
  env: {
      virtualDisplay: {
          DISPLAY: ':98'
      }
  }
  });

  grunt.loadNpmTasks('grunt-webdriver-runner');
  grunt.loadNpmTasks('grunt-shell-spawn');
  grunt.loadNpmTasks('grunt-env');
  grunt.loadNpmTasks('grunt-webdriver-manager');
  grunt.registerTask('webdriver-chrome', ['webdriver:chrome']);

  grunt.registerTask('webdriver-headless', [   
    'shell:virtualDisplay',
    'env:virtualDisplay',
    'webdriver:execute',
    'shell:virtualDisplay:kill'
  ]);

}

When I remove the webdriver task from the headless command, it runs very quickly and shows the shell tasks executing but no actual tests run. I think I’m missing something in my configuration to make the headless testing work properly.

How can I properly configure this setup to run tests in the background without opening browser windows?

2

Your webdriver isn’t picking up the DISPLAY environment variable. I ran into this exact issue setting up headless testing on CentOS servers last year - the browser launches before the environment gets configured.

First, modify your webdriver options to explicitly pass the display config. In settings.js, set Chrome options for headless mode or make sure it reads the DISPLAY variable. Also verify Xvfb is running on port 98 before webdriver starts. There’s often a timing issue where the virtual display isn’t ready.

Double-check that your webdriver process uses the same environment where you set DISPLAY=:98. Grunt doesn’t always pass environment variables between tasks properly. You’ll probably need to set DISPLAY directly in your webdriver config instead of relying on the env task.

3

check if xvfb’s running before starting your webdriver. had the same issue last month - my shell task was failing silently. run ps aux | grep Xvfb to see if the display server’s actually up. also add options: {failOnError: true} to your grunt shell task so it stops when xvfb doesn’t start right.

4

Your task execution order is messed up, and Chrome isn’t configured right. I’ve hit this exact issue when running automated tests on production servers. You’re killing the virtual display right after starting webdriver - that’s why your tests either don’t run or browser windows pop up. Drop the kill command from your task array and handle cleanup somewhere else. The bigger issue is Chrome needs specific headless flags in your webdriver config. Add chromeOptions to settings.js with ‘–headless’, ‘–no-sandbox’, and ‘–disable-gpu’. Without these, Chrome completely ignores XVFB. Also throw in a short delay between starting Xvfb and launching webdriver. The virtual display needs a moment to spin up before Chrome tries connecting. Use grunt-wait or setTimeout in a custom task to get the timing right.

5

Your webdriver config file is probably the issue, not Grunt. I spent weeks on similar headless failures and found that even with proper XVFB and environment variables, you need explicit browser capabilities defined. In settings.js, make sure you’re setting browser capabilities for headless mode. For Chrome, add capabilities like --disable-web-security and --disable-features=VizDisplayCompositor with your standard headless flags. Firefox needs --headless in the binary path instead. Check your webdriver and browser versions too. Mismatched versions ignore headless configs completely and fall back to windowed mode. Run version checks first - incompatible combinations always show UI no matter what your XVFB setup looks like.

6

Been fighting this exact problem for years. Your Grunt setup’s way too complex and breaks constantly. You’ve got too many moving pieces that can fail.

The real issue? You’re trying to duct-tape XVFB, Grunt, and WebDriver together when you need something that just works headless from the start.

I jumped to Latenode after getting sick of these config nightmares. It runs browser automation headless by default - no XVFB setup, no environment variables. You build your automation flow visually and it handles all the headless browser stuff automatically.

No more wrestling with display servers or wondering why browsers randomly pop up. Everything runs in the cloud so you never see UI on your machine.

You can trigger tests from webhooks, schedules, or API calls without babysitting all that Grunt infrastructure.

7

The Problem:

You’re experiencing issues running headless tests using Grunt, WebDriver, and Xvfb. Your tests run fine in non-headless mode, but when attempting headless execution, the browser window still appears, or tests fail to execute altogether. The core issue is a race condition between starting the virtual display (Xvfb) and initiating the WebDriver, compounded by incorrect Chrome configuration for headless mode. Your current Grunt task also prematurely kills the virtual display.

:thinking: Understanding the “Why” (The Root Cause):

The problem lies in the timing and dependency management within your Grunt tasks. Xvfb needs time to initialize before the WebDriver attempts to connect to it. Your current setup starts Xvfb, attempts to set the DISPLAY environment variable, starts the WebDriver, and then immediately kills Xvfb, leading to inconsistent and unreliable headless execution. Furthermore, Chrome requires specific command-line flags (--headless, --no-sandbox, --disable-gpu) to operate correctly in a headless environment; without them, it will ignore the Xvfb server and open a window.

:gear: Step-by-Step Guide:

  1. Modify your Gruntfile.js and settings.js:

    • Gruntfile.js Changes: Remove the shell:virtualDisplay:kill task from the webdriver-headless task. This ensures the virtual display remains active throughout the test execution. Also, ensure you are consistently setting and using the DISPLAY environment variable. Avoid relying on Grunt’s environment management between tasks, instead set the DISPLAY variable directly within your webdriver configuration (see settings.js below). Add a delay using grunt-contrib-wait to give Xvfb sufficient time to start.
    grunt.initConfig({
    // ... other configurations ...
    wait: {
      xvfbReady: {
        options: {
          delay: 5000 // 5-second delay
        }
      }
    },
    webdriver: {
      options: {
        // ... other options ...
      },
      execute: {}
    },
    shell: {
      virtualDisplay: {
        command: 'Xvfb :98 -ac -screen 0 1920x1080x24',
        options: {
          async: true,
          failOnError: true // Important for error handling
        }
      }
    },
    // ... other configurations ...
});


grunt.registerTask('webdriver-headless', [
    'shell:virtualDisplay',
    'wait:xvfbReady', // Added delay
    'webdriver:execute'
]);
    • settings.js Changes: Add the necessary Chrome options to your WebDriver configuration. This ensures Chrome runs in headless mode, even if the DISPLAY environment variable is not set correctly within the webdriver process.
    module.exports = {
    // ... other settings ...
    chromeOptions: {
        args: ['--headless', '--no-sandbox', '--disable-gpu', '--disable-web-security', '--disable-features=VizDisplayCompositor']
    }
    // ... other settings ...
};

  2. Verify Xvfb is Running: Before running your tests, manually verify that Xvfb is running correctly by executing ps aux | grep Xvfb in your terminal. If Xvfb isn’t running, troubleshoot the shell:virtualDisplay task. Ensure you have the necessary permissions to run Xvfb.

  3. Install grunt-contrib-wait: If you don’t already have it, install this plugin: npm install grunt-contrib-wait --save-dev

:mag: Common Pitfalls & What to Check Next:

  • WebDriver and Chrome Version Compatibility: Ensure your WebDriver and Chrome versions are compatible. Mismatched versions can lead to headless mode not working correctly.
  • Firewall/Security Restrictions: Check if any firewalls or security software are interfering with Xvfb or the WebDriver.
  • System Resources: Ensure your system has sufficient resources (CPU, memory) to run Xvfb and the browser concurrently. Headless mode still consumes resources, especially memory.
  • Clean Up: Implement a dedicated task for cleaning up after your tests; this might involve explicitly stopping Xvfb (though leaving it running for multiple test executions could be more efficient).

:speech_balloon: Still running into issues? Share your (sanitized) config files, the exact command you ran, and any other relevant details. The community is here to help!

8

This topic was automatically closed 24 hours after the last reply. New replies are no longer allowed.