I’ve been running a headless browser automation for about six months now that regularly pulls data from a few e-commerce sites. It’s been stable, mostly, but I’ve had to rebuild selectors at least three times because the sites updated their layouts. Each time it breaks, I’m manually inspecting the page, finding new selectors, updating the workflow.
I know this is a common problem with selector-based automation, but I’m wondering what people actually do to make this more maintainable. Are you rebuilding manually every time? Are you using something like visual recognition instead of CSS selectors? Is there a smarter way to set this up that I’m missing?
The workflow itself is solid—it’s just this brittle dependency on the HTML structure that keeps giving me headaches. What’s your actual approach when you know the website is going to change?
Selector brittleness is real, and there are a few patterns that help.
First, use XPath instead of CSS selectors when you can. XPath can target elements by text content or attributes, which tend to be more stable than class names that change on redesigns.
Second, layer your selectors. Don’t just look for .product-card .price. Look for a div containing specific text, then find the price within it. Multiple fallbacks mean one redesign doesn’t break everything.
In Latenode, you can add JavaScript to your workflow to handle selector fallbacks programmatically. If your primary selector fails, the script tries alternates. You can also use visual element recognition for critical steps—the platform supports that.
The real solution though is to combine selectors with data validation. Instead of trusting the selector alone, verify the data makes sense. If you get a price that’s way off, you know something broke and can alert yourself.
I’ve dealt with this a lot, and honestly, the best approach depends on how critical the automation is. If it’s something you can afford to babysit, XPath with multiple fallbacks is solid. But if you need it running without intervention, you need monitoring built in.
I set up alerts that trigger when selectors fail. Usually that means a page changed. Then I have a simple script that logs what the page structure actually looks like now, which speeds up rebuilding. The rebuild itself still happens manually, but at least I’m not discovering failures a week later.
For sites I know update frequently, I’ve moved toward data-driven selectors instead of structure-driven ones. Like, find the element by searching for specific text content rather than relying on a class name. It’s slower but way more resilient.
The selector brittleness problem gets worse as sites use JavaScript frameworks that dynamically generate class names. One approach that’s helped me is combining multiple selection strategies in the same workflow. Primary selector fails? Try an XPath that targets by text content. That fails? Try a third method that looks for unique data attributes. Building in these layers takes more effort initially but saves constant maintenance later. You’re essentially creating a fault-tolerant selector system instead of relying on one brittle path.
Dynamic websites require adaptive selectors. Rather than hardcoding selectors, consider implementing a validation layer that checks if extracted data matches expected patterns. This provides early warning when selectors break. Additionally, some automation platforms now support image-based element recognition as a fallback, which is more resilient to HTML structure changes. Combining this with proper monitoring ensures you’re alerted to failures before they cascade.