Element not found error when running headless Chrome automation on Google homepage

Other Questions Headless Browser
1

I keep getting an element not found error when trying to automate a simple search on Google using headless Chrome. The script works fine in normal mode but fails when running headless.

My setup:

  • Chrome Browser: 79.0.3945.79
  • ChromeDriver: 79.0.3945.36

Code I’m using:

System.setProperty("webdriver.chrome.driver", "D:\\selenium\\drivers\\chromedriver.exe");
ChromeOptions chromeOpts = new ChromeOptions();
chromeOpts.addArguments("--headless");
chromeOpts.addArguments("--window-size=1280,720");
DesiredCapabilities caps = DesiredCapabilities.chrome();
caps.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts);
WebDriver browser = new ChromeDriver(chromeOpts);
browser.manage().window().maximize();
browser.manage().deleteAllCookies();
browser.manage().timeouts().pageLoadTimeout(30, TimeUnit.SECONDS);
browser.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);
browser.get("https://www.google.com");
browser.findElement(By.xpath("//input[@title='Search']")).sendKeys("Selenium");

Error message:

org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//input[@title='Search']"}
(Session info: headless chrome=79.0.3945.79)

The error says it cannot find the search input field. Has anyone faced this issue with headless Chrome? Any suggestions on how to fix this?

2

Add --disable-blink-features=AutomationControlled to your Chrome options. Google detects automation and serves different layouts in headless mode. Also set a proper user agent string to mimic a real browser. Fixed the same issue for me last month.

3

Had the same problem last year with headless Google automation. Google serves different DOM structures based on your browser context and user agent - headless mode gets a stripped-down page without the same element attributes. Skip the title attribute and use By.name("q") instead. Way more reliable across different Google versions. This selector works consistently in both headless and regular modes. Also throw in a quick wait after navigating before you hunt for elements. Headless Chrome needs that extra beat to fully render the DOM, even with implicit waits set up.

4

Google changes their page structure when they spot headless browsers. Don’t fight their detection - just add --user-data-dir=/tmp/chrome-test and --no-sandbox to your chrome options. The user data directory makes Chrome act like a normal browser session. I hit this same problem six months ago and switching to By.cssSelector("input[name='q']") fixed it instantly. The name attribute doesn’t change, but title attributes do depending on localization and page variants. Also throw in a quick explicit wait before you interact with the search box - headless mode loads elements a bit differently than regular Chrome.