I’ve been running puppeteer scripts for data extraction for a couple of years now, and one thing that consistently bites me is how fragile everything becomes when a client’s website gets a redesign. Usually it takes maybe a week or two before selectors start failing, and then I’m back in there manually patching things.
The real pain point is that I’m maintaining this stuff across like 15 different workflows, so when one breaks, I have to hunt through code, figure out what changed on the site, update the selector, test it again. It’s tedious work that doesn’t really add value.
I’ve heard that some platforms can convert a plain English description of what you’re trying to do into a puppeteer workflow that somehow adapts better when sites change. Has anyone actually tried this? Does it actually reduce the maintenance burden, or is it just moving the problem somewhere else?