Puppeteer cannot select element with dynamic ID - checkbox clicking issue

[Emma_Fluffy]

I need help with selecting a checkbox element using Puppeteer when the element has a dynamic ID that changes on every page load.

My Problem:
I’m trying to click a checkbox for a specific person, but the ID attribute changes each time I visit the page. I tried using what I thought was a regex pattern but got an error.

What I tried:

const target = '#id^="/^[A-Za-z0-9_]+-[0-9]+/g"';
await page.click(target);

Error message:

DOMException: Failed to execute 'querySelector' on 'Document': '#id^="/^[A-Za-z0-9_]+-[0-9]+/g"' is not a valid selector.

Testing my regex separately:

const testId = '7429-1';
const pattern = /^[A-Za-z0-9_]+-[0-9]+/g;
const result = testId.match(pattern);
console.log(result); // [ '7429-1' ]

HTML structure:

<span class="person-item-main">JOHN DOE (98765432)</span>
<span class="person-item-main">PENDING</span>
<span class="person-item-main">$25.50</span>

Environment:

• JavaScript with Node.js v20.16.0
• Puppeteer version 23.0.2

How can I properly select and click an element when its ID follows a pattern but changes dynamically? What’s the correct approach for this situation?

[mythicMuse]

CSS selectors don’t support regex patterns like that. You’re mixing CSS attribute selectors with regex syntax, which won’t work.

Here are your options in Puppeteer:

Option 1: Attribute starts-with selector

// If IDs always start with numbers
const target = '[id^="7429"]';
await page.click(target);

Option 2: XPath with contains

const [element] = await page.$x('//input[contains(@id, "7429")]');
if (element) await element.click();

Option 3: Evaluate in page context

const element = await page.evaluateHandle(() => {
  return document.querySelector('[id*="7429"]');
});
if (element) await element.click();

Honestly though, dynamic selectors in scraping get messy fast. You’ll end up writing tons of fallback logic and selector variations.

I’ve been using Latenode for this stuff because it handles dynamic elements way better than raw Puppeteer. It’s got built-in retry logic and smarter element detection that works even when IDs change. Plus you can set up the whole flow visually instead of debugging selector strings.

Check it out: https://latenode.com

[opalEcho]

The Problem:

You’re encountering difficulties using Puppeteer to click a checkbox with a dynamically changing ID. Your attempts to use regular expressions within CSS selectors are failing, resulting in a DOMException. You’ve correctly tested your regex separately, but it’s not correctly applied within the Puppeteer context.

Understanding the “Why” (The Root Cause):

CSS selectors, used by Puppeteer’s page.click() method, do not directly support regular expressions. Your attempt to use a regular expression within the selector string ('#id^="/^[A-Za-z0-9_]+-[0-9]+/g"') is fundamentally incorrect. CSS selectors have their own specific syntax for pattern matching, which differs from JavaScript regular expressions. This is why Puppeteer throws the DOMException. Trying to force a regex into a CSS selector will always fail.

Step-by-Step Guide:

1. Use page.evaluate() with a JavaScript Regular Expression: This approach allows you to leverage the power of JavaScript regular expressions within the browser context to find the correct element, and then click it. This bypasses the limitations of CSS selectors.

const element = await page.evaluate((pattern) => {
  const regex = new RegExp(pattern);
  const checkboxes = Array.from(document.querySelectorAll('input[type="checkbox"]')); // Get all checkboxes
  const targetCheckbox = checkboxes.find(checkbox => regex.test(checkbox.id));
  return targetCheckbox;
}, /^[A-Za-z0-9_]+-[0-9]+$/);

if (element) {
  await element.click();
} else {
  console.error("Checkbox not found!");
}

This code first defines a regular expression that matches your expected ID pattern. It then selects all checkboxes on the page using querySelectorAll. Finally, it uses the find method to locate the checkbox whose ID matches the regex.

2. (Alternative) Use XPath Selectors with contains() or matches(): If you are comfortable with XPath, this offers a more concise approach that can handle partial matches.

const checkbox = await page.$x(`//input[@type='checkbox'][contains(@id, '7429')]`); // Partial match
// Or, if you need more precise regex within XPath:
// const checkbox = await page.$x(`//input[@type='checkbox'][matches(@id, '^[A-Za-z0-9_]+-[0-9]+$')]`);

if (checkbox.length > 0) {
  await checkbox[0].click();
} else {
  console.error("Checkbox not found!");
}

XPath’s contains() provides a simpler way to find elements that have an ID that contains a specific substring, while matches() supports the use of regular expressions.

Common Pitfalls & What to Check Next:

• Incorrect Regex: Double-check your regular expression ^[A-Za-z0-9_]+-[0-9]+$ to ensure it accurately captures the structure of your dynamic IDs. Test it thoroughly with various sample IDs.
• Checkbox Selector: Make absolutely sure that input[type="checkbox"] accurately targets your checkbox. Inspect the page’s HTML to confirm the element’s structure and attributes. There might be other attributes besides the id you can use for selection.
• Element Visibility: Verify the checkbox is visible and not hidden by CSS or JavaScript before attempting to click it. You might need to use page.waitForSelector or a similar method to ensure it’s rendered.
• Multiple Checkboxes: If the page contains multiple checkboxes, ensure your selector (regex or XPath) is sufficiently specific to target the correct one.
• Synchronization: Ensure your script waits until the page is fully loaded and the checkbox element exists before attempting to click it. Use page.waitForSelector to wait for the element to appear.

Still running into issues? Share your (sanitized) config files, the exact command you ran, and any other relevant details. The community is here to help!

[Samuel87]

Been wrestling with the same dynamic ID mess for months. The issue isn’t your regex - you’re just attacking this wrong. Forget chasing those dynamic IDs. Look at your HTML structure again. You’ve got person data in spans but I don’t see any checkbox in your example. It’s probably sitting as a sibling or child element near those person-item-main spans. Find the person by name first, then grab the checkbox:

const personRow = await page.$x(`//span[contains(text(), 'JOHN DOE')]/ancestor::*[contains(@class, 'row') or contains(@class, 'item')][1]`);
if (personRow.length > 0) {
  const checkbox = await personRow[0].$('input[type="checkbox"]');
  if (checkbox) await checkbox.click();
}

This targets stable content (names) instead of unstable IDs. Way more maintainable since names don’t change but those generated IDs will keep breaking your selectors.

[Alex_Brave]

puppeteer’s waitForSelector with xpath handles this way better. try await page.waitForSelector('xpath///input[matches(@id, "^[A-Za-z0-9_]+-[0-9]+$")]') then click it. xpath supports regex matching - css selectors can’t handle patterns at all.

[JumpingRabbit]

querySelector doesn’t understand regex at all. You need to find elements programmatically first, then interact with them.

I’ve hit this same issue. Use page.$$eval() for dynamic ID patterns:

const targetElement = await page.$$eval('[id]', (elements, pattern) => {
  const regex = new RegExp(pattern);
  return elements.find(el => regex.test(el.id));
}, '^[A-Za-z0-9_]+-[0-9]+$');

if (targetElement) {
  await page.click(`#${targetElement.id}`);
}

Another option that’s saved me headaches: use partial attribute matching when part of the ID stays consistent. If your IDs look like “prefix-7429-1”, try [id*="7429"] to match any ID containing that substring.

Since your HTML shows person data, maybe target the parent container and find the checkbox relative to the person’s name instead of chasing dynamic IDs.

[Finn_Mystery]

CSS selectors don’t support regex patterns - that’s your problem. You’ll need to use Puppeteer’s page.evaluate() to run custom JavaScript that handles regex matching.

Here’s what worked for me:

const element = await page.evaluate(() => {
  const pattern = /^[A-Za-z0-9_]+-[0-9]+$/;
  const elements = document.querySelectorAll('[id]');
  for (let el of elements) {
    if (pattern.test(el.id)) {
      return el;
    }
  }
  return null;
});

if (element) {
  await page.click(`#${element.id}`);
}

Or try targeting something more stable in the HTML structure. From your example, you could target “JOHN DOE” text and navigate to the checkbox from there. I’ve found this more reliable since text content stays the same while IDs keep changing.