I’ve been running Puppeteer automations for a couple of years now, and the biggest headache I keep running into is brittle selectors. Every time a site redesigns, my scripts break. I’ll spend hours debugging, updating CSS selectors, fixing XPath expressions, and it’s just exhausting.
I started thinking about this differently recently. The real problem isn’t Puppeteer itself—it’s that hand-coded scripts are so rigid. You’re essentially hard-coding assumptions about how a page is structured. The moment that changes, everything falls apart.
I’ve been curious about whether there’s a smarter approach. Like, what if you could describe what you’re actually trying to extract or interact with in plain language, and have something generate the automation for you? That way, when a site changes, you’re not manually rewriting selectors. You’d regenerate the workflow based on the same human-readable description.
Has anyone here dealt with this in a way that actually scales? Are you constantly maintaining old scripts, or have you found an approach that’s more resilient to site changes?
This is exactly what I dealt with at my company. We had a graveyard of Puppeteer scripts that needed constant maintenance.
The shift that changed everything for us was moving away from hand-coded selectors entirely. With Latenode’s AI Copilot Workflow Generation, you describe what you want to do in plain English—like “extract all product prices from the search results page”—and it generates the Puppeteer workflow for you. When a site redesigns, you just regenerate it. No manual selector hunting.
The really useful part is that the AI understands the intent behind what you’re trying to do, not just the structure of the page. So even tiny layout changes don’t break everything.
We went from spending hours debugging to spending minutes regenerating. And non-technical folks can actually participate in defining automations instead of waiting for developers.
I’ve seen teams handle this a few different ways. The maintenance overhead is real, no matter what.
One approach I’ve seen work is building a layer of abstraction between your selectors and your logic. Instead of embedding selectors everywhere, you centralize them in a config file or a function. That way when something breaks, you’re only updating one place. It doesn’t solve the redesign problem, but it makes the pain less spread out.
Another thing that helped was incorporating visual regression testing into the workflow. Not perfect, but it catches when a page layout change actually matters versus cosmetic tweaks.
That said, there’s only so much you can do to work around the fundamental issue. Hand-coded automations are inherently fragile.
From what I’ve experienced, you really have two options: accept high maintenance costs or use a system that generates automations intelligently. I’ve worked with teams that tried to engineer their way around this—better error handling, retry logic, visual selectors—but none of it fundamentally solves the brittleness problem. The real breakthrough comes when you stop thinking about “maintaining selectors” and start thinking about “regenerating automations when things change.” It shifts the problem from constantly debugging to occasionally regenerating. The workflows stay aligned with the actual page structure because they’re built based on intent, not assumptions.
The core issue is that traditional Puppeteer scripts create a tight coupling between your automation logic and the page’s DOM structure. Every time the structure changes, the coupling breaks. The most effective solutions I’ve seen decouple these concerns. Some teams use computer vision alongside Puppeteer to identify elements by appearance rather than selector. Others use dynamic selector generation. But the most resilient approach is having an intelligent system that continuously understands what the page contains and regenerates the extraction logic based on that understanding. This eliminates the need for constant manual maintenance.
Yeah, hand-coded selectors break constantly. Try layering abstraction—centralize them in configs. Even better, use a system that regenerates workflows intelligently when sites change. Saves way more time than maintaining old scripts.