I would like to know how to apply the following profile preferences for the PhantomJS headless browser in Python. Here’s a code snippet that demonstrates my attempt:
def setup_webdriver():
print('Establishing webdriver connection...')
global web_driver
if web_driver is None:
options = webdriver.FirefoxProfile()
options.accept_untrusted_certs = True
options.set_preference('network.proxy.type', 1)
options.set_preference('network.proxy.http', 'proxy.example.com')
options.set_preference('network.proxy.https', 'proxy.example.com')
options.set_preference('network.proxy.ssl', 'proxy.example.com')
options.set_preference('network.proxy.http_port', 8080)
options.set_preference('network.proxy.https_port', 8080)
options.set_preference('network.proxy.ssl_port', 8080)
options.update_preferences()
web_driver = webdriver.Firefox(firefox_profile=options)
web_driver.maximize_window()
yield web_driver
web_driver.quit()
I’m currently working with Python 2.7, Selenium 3.0, and PhantomJS 1.9.7.
PhantomJS is outdated and no longer maintained. Using Selenium with a more up-to-date WebDriver like Chrome or Firefox is recommended. However, if you need to use PhantomJS, you can configure it with the desired network preferences using the following:
from selenium import webdriver
service_args = [
'--proxy=proxy.example.com:8080',
'--proxy-type=http'
]
driver = webdriver.PhantomJS(service_args=service_args)
This sets the proxy for HTTP connections. Adjust 'proxy-type' based on your needs (http/https/socks/etc.).
The PhantomJS WebDriver indeed lacks continued support, but if you still need to use it, customizing network settings programmatically can be done directly via service arguments as indicated by CreatingStone. Here’s a more comprehensive setup for configuring proxies across multiple protocols:
from selenium import webdriver
service_args = [
'--proxy=proxy.example.com:8080',
'--proxy-type=http', # Change this to 'socks' if needed
'--ignore-ssl-errors=true'
]
driver = webdriver.PhantomJS(service_args=service_args)
The service arguments provided above handle proxy settings for HTTP connections and ignore SSL errors, which is crucial for untrusted certificates.
If migrating to a newer setup, it’s worth trying a combination involving the headless options of the Chrome or Firefox drivers in Selenium, which provide regularly updated support and security improvements while achieving similar headless browsing capabilities.
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--proxy-server=proxy.example.com:8080')
driver = webdriver.Chrome(options=chrome_options)
This code snippet sets up Chrome WebDriver in headless mode with similar proxy configurations. Transitioning to Chrome WebDriver is recommended for tool longevity and security enhancements.