How can I iterate over website elements using Puppeteer?

Other Questions Puppeteer
1

I’m working with Puppeteer on Node.js and need a way to identify certain elements on a website. The element IDs vary and might start at 1 and go up to 6, or they could begin at a different number like 5 and continue sequentially.

Here’s an example of what I tried:

let found = false;
if (await page.$('input[data-customid="element-1"]')) {
  found = true;
}
console.log(found ? 'Element found' : 'Element not found');

I want to loop through possible IDs until I detect an element that exists, setting a flag to true when it’s found. If the first element isn’t there, I need to check the next one consecutively. Can anyone help me find a more efficient way to confirm if a selector exists on the page before interacting with it?

2

I’ve faced similar challenges with Puppeteer, and here’s an approach that worked well for me:

You can use a while loop combined with a try-catch block to iterate through possible element IDs. This method is efficient and flexible, allowing you to handle varying starting points.

Here’s a code snippet that demonstrates this:

async function findElement(page, prefix, startIndex = 1) {
  let index = startIndex;
  let element = null;

  while (!element) {
    try {
      element = await page.$(`input[data-customid=\"${prefix}-${index}\"]`);
      if (element) {
        console.log(`Found element with ID: ${prefix}-${index}`);
        return element;
      }
    } catch (error) {
      // Element not found, continue to next index
    }
    index++;
    if (index > startIndex + 20) break; // Prevent infinite loop
  }

  console.log('Element not found');
  return null;
}

You can call this function like:

const element = await findElement(page, 'element', 5);

This approach is more dynamic and can handle various starting points and ID patterns. It’s also more efficient as it stops once an element is found.

3

hey, i’ve dealt with this before! try using page.$$eval() to grab all matching elements at once. something like:

const elements = await page.$$eval(‘input[data-customid^=“element-”]’, els => els.length > 0);

console.log(elements ? ‘Elements found’ : ‘No elements found’);

this checks for any inputs with data-customid starting with ‘element-’. Hope that helps!

4

From my experience with Puppeteer, a more efficient approach would be using page.evaluate() to run JavaScript directly in the browser context. This method can be faster and more flexible for iterating over elements.

Here’s a potential solution:

const found = await page.evaluate(() => {
  for (let i = 1; i <= 20; i++) {
    const element = document.querySelector(`input[data-customid=\"element-${i}\"]`);
    if (element) return true;
  }
  return false;
});

console.log(found ? 'Element found' : 'Element not found');

This code checks for elements with IDs from 1 to 20, but you can adjust the range as needed. It’s more efficient because it runs in the browser, reducing the back-and-forth communication between Node.js and the browser. Plus, it stops as soon as an element is found, saving unnecessary iterations.