I’m trying to pull the pricing information from a webpage using the importXML function in Google Sheets, but I’m facing some challenges. The HTML part I’m interested in looks like this:
I suspect that having both a class and an id might be causing the problem. How can I successfully reference an element like this? I’ve tried different approaches, but I’m not getting the results I need. Thanks!
Deal with this at work all the time. Google Sheets importXML fails on modern e-commerce sites because they’re built to block scraping.
Your XPath isn’t the problem. Most product sites load prices with JavaScript, plus they detect bots and serve different content to Google’s servers.
I gave up on importXML for price monitoring years ago. Now I use automated workflows that handle JavaScript rendering, rotate user agents, manage sessions, and work around CAPTCHAs.
Set up proper automation with headless browsers. Let it capture data after JavaScript loads and feed it straight into your sheets. Schedule it hourly or daily - way more reliable than hoping importXML works.
Handles all the complexity and just gives you clean data.
Your XPath looks correct, but it seems something else might be at play. Many e-commerce sites rely on JavaScript to dynamically load prices after the initial HTML loads, which means importXML can’t access this information because it only retrieves the static HTML. I’ve encountered this issue frequently while scraping product pages. Make sure to check the actual page source (not through inspect element) and look for ‘final_price’. If that span is empty or shows a placeholder, it indicates that JavaScript is populating it later. Additionally, the website could be blocking requests from Google’s servers; many retailers deliver different content to bots compared to regular browsers to prevent scraping.
I’ve hit this before - usually it’s rate limiting or server protections, not your XPath. Some sites need specific headers or cookies to serve the full HTML. Try adding a user agent string or check for CAPTCHA triggers from automated requests. What worked for me was combining both selectors: //span[@class='new_price' and @id='final_price'] to be more specific. Also, some sites serve different content by location, so the element might not exist when hit from Google’s servers. Test your XPath locally in browser dev tools first to make sure the element’s actually there.
Been dealing with this too - timing’s way more important than people think. ImportXML times out fast, and if the server’s slow to respond, you get nothing back even with perfect XPath. Happens a lot with smaller e-commerce sites that can’t handle traffic well. Hit the URL in your browser with dev tools open and check the network tab - see how long that initial HTML takes. If it’s over a few seconds, that’s probably why you’re getting blanks. Test your formula on a simpler, faster page first to make sure the XPath works before blaming bot detection.
I’ve hit this exact problem with hardware retailer sites. That domain’s probably running CloudFlare protection - it throws up a challenge page before you can see the real content. Your XPath looks fine though. Test it on a simpler site first to double-check. UK retail sites are tricky too - they check referrer headers and serve different HTML depending on how you got there. Quick fix: manually visit that URL and check the page source. Make sure the span actually has the price text instead of getting filled in later through AJAX calls to their inventory system.
some sites totally block google sheets requests. first, try wrapping your xpath with importhtml to see if you can pull any data from that page. if that fails too, then the site’s got anti-bot protection runnin. also worth checking - disable js in your browser and see if the price loads. that’ll tell ya if it’s a dynamic loadin issue.