I’m trying to set up a script that can do stuff on a website automatically on a schedule. I know about browser add-ons like Greasemonkey and Selenium but I need something that works from the command line. I’m good with PowerShell, NodeJS, or any .NET stuff.
One thing I was wondering is if it’s possible to save web actions in a .HAR file and then run them later. Or is .HAR just for keeping track of network stuff and can’t be used to repeat actions?
I’ve looked into a few options but I’m not sure what’s the best way to go about this. Has anyone done something similar before? What worked for you? Any tips or tricks would be really helpful. Thanks!
hey, have u tried using curl? its pretty awesome for command-line web stuff. i use it with bash scripts to automate tasks. you can send POST requests, handle cookies, and even do file uploads. plus, its already installed on most systems. just a thought if you haven’t checked it out yet!
I’ve had great success using Python with the requests library for automated web interactions. It’s incredibly versatile and works well from the command line. You can easily schedule tasks with cron jobs on Linux or Task Scheduler on Windows.
For more complex scenarios, I’ve found Puppeteer (Node.js library) to be a powerful tool. It can handle JavaScript-heavy sites and even supports headless Chrome for server environments.
Regarding .HAR files, they’re primarily for logging network traffic, not for replaying actions. However, you can parse them to extract useful information for your scripts.
One tip: always respect website terms of service and implement proper delays to avoid overloading servers. Also, consider using API endpoints when available - they’re often more stable for automation than scraping web pages directly.
For command-line web automation, I’ve found Puppeteer to be incredibly effective. It’s a Node.js library that gives you programmatic control over Chrome or Chromium. You can script complex interactions, handle dynamic content, and even generate PDFs or screenshots.
If you’re comfortable with .NET, you might want to look into PuppeteerSharp. It’s a port of Puppeteer for .NET, offering similar functionality. This could integrate well with your existing skills.
As for .HAR files, they’re not designed for replaying actions. They’re more for analyzing network traffic. However, you could potentially use them to identify API endpoints or request patterns to incorporate into your automation scripts.
One crucial tip: Always implement proper error handling and retry mechanisms in your scripts. Web interactions can be unpredictable, and robust error management will save you a lot of headaches in the long run.