What's the best way to verify selector existence in Puppeteer?

Other Questions Puppeteer
1

I’m struggling with Puppeteer and need help. I want to check if certain selectors exist on a page. If they don’t, I need to set default values. For example, I want to check if #idProductType is there. If it’s not, I want to set producttype to an empty string. I’ve tried a bunch of things, but nothing works. Here’s a simplified version of what I’m trying to do:

let scrapeData = await page.evaluate(() => {
  let productInfo = {};
  
  if (document.querySelector('#productCategory')) {
    productInfo.category = document.querySelector('#productCategory').innerText;
  } else {
    productInfo.category = 'Unknown';
  }
  
  if (document.querySelector('#itemName')) {
    productInfo.name = document.querySelector('#itemName').innerText;
  } else {
    productInfo.name = 'Unnamed Product';
  }
  
  return productInfo;
});

Can someone help me figure out the right way to do this in Puppeteer? Thanks!

2

hey charlottew, i’ve dealt with this before. try using page.$eval() instead. it’s more efficient and handles non-existent selectors gracefully. something like:

let productInfo = await page.evaluate(() => ({
  category: document.querySelector('#productCategory')?.innerText || 'Unknown',
  name: document.querySelector('#itemName')?.innerText || 'Unnamed Product'
}));

this should work better for ya. good luck!

3

I’ve had success using the waitForSelector() method in Puppeteer for this exact scenario. It’s a robust way to check for element existence and handles timing issues well. Here’s an approach you could try:

let productInfo = {};

try {
  await page.waitForSelector('#productCategory', { timeout: 5000 });
  productInfo.category = await page.$eval('#productCategory', el => el.innerText);
} catch (error) {
  productInfo.category = 'Unknown';
}

try {
  await page.waitForSelector('#itemName', { timeout: 5000 });
  productInfo.name = await page.$eval('#itemName', el => el.innerText);
} catch (error) {
  productInfo.name = 'Unnamed Product';
}

This method gives you more control over timeouts and error handling. It’s particularly useful when dealing with dynamically loaded content. Just adjust the timeout as needed for your specific use case.

4

I’ve faced similar challenges with Puppeteer, and I’ve found that using the page.$$eval() method can be quite effective for checking multiple selectors at once. Here’s an approach that might help:

const productInfo = await page.$$eval('body', (elements, selectors) => {
  return selectors.reduce((acc, selector) => {
    const element = elements[0].querySelector(selector.id);
    acc[selector.key] = element ? element.innerText.trim() : selector.default;
    return acc;
  }, {});
}, [
  { id: '#productCategory', key: 'category', default: 'Unknown' },
  { id: '#itemName', key: 'name', default: 'Unnamed Product' }
]);

This method is efficient as it only requires one trip to the page’s context. It also allows you to easily add more selectors to check without cluttering your code. Just remember to handle any potential errors, as network issues or page load problems can still occur.