I’m working on a web scraping project where I need to check if a specific element shows up on the page. The tricky part is that this element might appear or it might not appear at all.
What I want to do is wait for a certain CSS selector for about 10 seconds. If the element shows up during this time, I’ll execute one set of actions. But if it doesn’t show up after the timeout, I need to run a different set of actions instead.
I’ve been thinking about using await page.waitForTimeout(10000) but I’m not sure if that’s the best approach. Is there a better way to handle this conditional waiting scenario in Puppeteer? Maybe something that can detect the element’s presence without blocking the entire execution?
You could also consider using page.waitForFunction() with a custom condition. This approach gives you more control over what exactly you’re waiting for. Something like await page.waitForFunction(() => document.querySelector('.your-selector'), {timeout: 10000}) wrapped in try-catch. I’ve found this particularly useful when dealing with dynamic content that loads unpredictably. The function keeps polling until your condition is met or times out, which is more efficient than fixed delays. Just make sure your selector logic inside the function is solid since debugging can be trickier than regular waitForSelector.
i’d suggest using page.waitForSelector() with a timeout parameter. put it in a try-catch block to manage the timeout case. like await page.waitForSelector('.your-selector', {timeout: 10000}) then catch the error for your alternate actions. much neater than waitForTimeout!
I ran into a similar issue last year and ended up using Promise.race() to handle this elegantly. You can race between page.waitForSelector() and a custom timeout promise. When the race resolves, check which promise won to determine your action path. The benefit is that you get immediate response when the element appears instead of waiting the full timeout period. I wrapped this in a helper function that returns a boolean indicating whether the element was found. This approach has served me well across multiple scraping projects where element presence varies depending on page state or user permissions.