Issues with file downloads in Selenium's headless mode due to unavoidable download prompts

Other Questions Headless Browser
1

I’ve been using Selenium with Python to download files from a website that blocks direct HTTP requests. My setup worked fine for a while but recently hit a snag.

The problem is that I can’t seem to turn off the ‘Save As’ popup when trying to download files in headless mode. This popup is getting in the way and stopping the downloads.

Here’s a simplified version of what I was doing:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument('--headless')
options.add_experimental_option('prefs', {
    'download.prompt_for_download': False,
    'download.default_directory': '/path/to/downloads'
})

browser = webdriver.Chrome(options=options)
browser.get('file_download_url')

This used to work but now the download prompt keeps showing up even with the experimental option set to False.

Has anyone else run into this problem lately? Any ideas on how to fix it or work around it? I’m on Windows 11 if that matters. Thanks for any help!

2

I’ve dealt with this headache before, and it can be frustrating. One thing that worked for me was switching to Firefox instead of Chrome. Firefox seems to handle headless downloads better in my experience.

Here’s what I did:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options

options = Options()
options.add_argument(‘-headless’)
options.set_preference(‘browser.download.folderList’, 2)
options.set_preference(‘browser.download.manager.showWhenStarting’, False)
options.set_preference(‘browser.download.dir’, ‘/path/to/downloads’)
options.set_preference(‘browser.helperApps.neverAsk.saveToDisk’, ‘application/octet-stream’)

driver = webdriver.Firefox(options=options)

This approach bypassed the download prompts for me. Just make sure you have the latest geckodriver installed. Also, you might need to tweak the MIME types in the ‘neverAsk.saveToDisk’ preference depending on your file types.

Hope this helps! Let me know if you need any clarification.

3

I’ve encountered similar issues with Selenium headless downloads. One effective workaround I’ve found is using the Chrome DevTools Protocol (CDP) to intercept and handle download requests. This method bypasses the Save As dialog entirely.

Here’s a basic implementation:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument('--headless')
browser = webdriver.Chrome(options=options)

browser.execute_cdp_cmd('Page.setDownloadBehavior', {
    'behavior': 'allow',
    'downloadPath': '/path/to/downloads'
})

browser.get('file_download_url')

This approach has proven more reliable for me in recent versions of Chrome. It’s worth noting that you may need to adjust your code to wait for the download to complete, as headless downloads can be asynchronous.

4

hey dancingbird, i’ve hit this issue too. headless mode can be tricky with downloads. have u tried using a custom chrome profile instead? it might bypass the popup. also, check if the site changed anything recently. sometimes they tweak stuff to block automation. gl!