How to configure custom headers for headless Chrome/Firefox automation

[Mike71]

I’m working on automated testing using a headless browser setup. During my tests, I take screenshots and realize that the pages in my development environment appear blank, whereas well-known sites operate normally.

I suspect this is due to missing HTTP headers that my staging environment requires. How can I set custom request headers when using Selenium with a headless browser?

Here’s the code I have at the moment:

public void runHeadlessTest() throws Exception {
    ChromeOptions options = new ChromeOptions();
    options.addArguments("--headless");
    options.addArguments("--no-sandbox");
    WebDriver browser = new ChromeDriver(options);
    
    browser.get("https://dev-environment.example.com/");
    captureScreenshot("TestPage");
}

Any guidance on adding custom headers to support my staging environment would be appreciated!

[avamtz]

have u tried using Chrome DevTools Protocol? it allows u to modify headers more effectively. use commands like Network.setUserAgentOverride and Network.setRequestInterception. it’s def better for staging setups that need those auth headers.

[Noah_Fire]

Had the same issue with our staging servers. The problem was missing auth tokens in the headers. I fixed it using ChromeDriver’s Network domain to add headers. Enable Chrome DevTools Protocol, then use executeScript to set headers before navigating to your URL. You could also try a proxy approach - intercept requests and modify headers on the fly. That worked better for me, especially with complex auth flows. Just make sure you set headers before the initial page load, not after.

[Dave_17Sketch]

I ran into the same header problems testing against protected dev environments. Chrome DevTools Protocol alone wasn’t cutting it for me. Here’s what worked better: build a custom Chrome extension that intercepts requests and automatically injects the headers you need. Way more reliable, especially with those CORS preflight requests that staging environments love to throw at you. Load it through ChromeOptions with addExtensions. Or go with a local proxy server that handles header modification transparently - this works across any browser and keeps your test suite consistent without browser-specific hacks.