I’m trying to create an automated script that runs through cron, but I’m having issues with browser automation. Even though I’m using a headless setup, Firefox still opens a visible window when the script runs.
The main problem is that when I execute this manually, everything works fine. But when cron tries to run it, the script fails because it can’t open the display.
Here’s my current approach:
#!/usr/bin/env ruby
require 'watir-webdriver'
require 'headless'
# Start headless session
headless_session = Headless.new
web_driver = Watir::Browser.start 'http://dashboard.example.com/login'
# Navigate to login page
web_driver.link(:xpath => '//div[@class="login-container"]/a').when_present.click
web_driver.text_field(:id => 'username').when_present.set '[email protected]'
web_driver.button(:id => 'continue').click
web_driver.text_field(:id => 'password').when_present.set 'mypassword'
web_driver.button(:id => 'login').click
# Go to main dashboard
web_driver.goto 'https://dashboard.example.com/main'
# Extract required data
data_points = []
# ... data extraction logic here ...
# Cleanup
web_driver.close
headless_session.destroy
# Output results
timestamp = Time.now
puts "Data extracted at #{timestamp}"
data_points.each_slice(2) { |pair| puts pair.join(" - ") }
My shell script looks like this:
#!/bin/bash
today=$(date +"%Y_%m_%d")
ruby data_scraper.rb > ~/reports/daily_$today.txt
I thought headless mode was supposed to run without opening any GUI windows. Am I configuring something wrong? The script completes but produces empty output when run via cron.
Note: I also tried using a different headless browser but ran into timeout issues with form fields, especially on authentication pages.