Selenium headless browser showing 'Unavailable' instead of actual page source

Other Questions Headless Browser
1

I’m developing a Python script to log into a website and check if there are available appointment slots. It works perfectly when I run the browser normally, but in headless mode, I’m only getting ‘Unavailable’ in the page source instead of the expected content.

Here’s an example of my code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
import time

# Website configuration
login_url = 'https://mywebsite.com/login'
username = '[email protected]'
password = 'your_secure_password'
expected_message = "No appointments available."

# Headless Chrome settings
chrome_options = Options()
chrome_options.add_argument("--headless")

# Launch the browser
driver = webdriver.Chrome(options=chrome_options)
driver.implicitly_wait(5)

# Go to the login page
driver.get(login_url)

# Find and fill in the login fields
email_field = driver.find_element('name', 'Email')
password_field = driver.find_element('name', 'Password')
email_field.send_keys(username)
password_field.send_keys(password)

# Log in
password_field.send_keys(Keys.RETURN)

# Wait for the page to load
time.sleep(5)

# Verify the content
if expected_message in driver.page_source:
    print('No availability')
else:
    print('Appointments are available!')

driver.quit()

When I run the code normally, I can see the expected content after logging in. However, in headless mode, the only text returned is ‘Unavailable’.

I have tried various user agent strings and added common flags like no-sandbox and disable-gpu. Additionally, I’ve explored different waiting methods, yet the issue persists. Is it possible that the website is blocking headless browsers? What alternatives can I explore to resolve this?

2

I’ve hit this same problem. The site’s using JavaScript fingerprinting to catch headless browsers. Add --disable-web-security and --disable-features=VizDisplayCompositor to your Chrome options. Better yet, switch to undetected-chromedriver - it’s built to dodge these checks. Set your window size with --window-size=1920,1080 too, since some sites flag missing dimensions. If Chrome keeps getting blocked, try Firefox headless instead. Many sites only bother detecting Chrome.

3

it seems like the site is blocking headless modes. try adding --disable-blink-features=AutomationControlled and set a proper user-agent. also, don’t forget to remove the webdriver property after starting the driver, some sites check that.

4

Had this exact issue last month with a booking site. The problem isn’t just detection - headless browsers handle JavaScript differently. Try adding --disable-extensions and --no-first-run to your options. More importantly, bump your wait time after login to at least 10 seconds or use WebDriverWait with expected conditions instead of time.sleep(). Some sites load content asynchronously and headless mode processes it slower. Also check if the site needs cookies from the initial page load - I had to hit the main page first before logging in. If nothing works, try --headless=new which uses the newer headless implementation that’s harder to detect.