Understanding Puppeteer's evaluate method

Other Questions Puppeteer
1

Hey everyone, I’m trying to get my head around Puppeteer. I want to grab the view count from a YouTube video. I wrote this bit of code that works fine in the Chrome console:

document.querySelector('#count > yt-view-count-renderer > span.view-count.style-scope.yt-view-count-renderer').innerText

But when I use it in my Puppeteer script, it can’t find the element. Here’s what I tried:

const puppeteer = require('puppeteer')

async function getViews() {
  const browser = await puppeteer.launch()
  const page = await browser.newPage()
  await page.goto('https://www.youtube.com/watch?v=dQw4w9WgXcQ')
  await page.waitForTimeout(1000)

  const result = await page.evaluate(() => {
    const viewCount = document.querySelector('#count > yt-view-count-renderer > span.view-count.style-scope.yt-view-count-renderer').innerText
    return { views: viewCount }
  })

  await browser.close()
  return result
}

getViews().then(console.log)

I ended up using the ytInitialData object to get it working. But I’m curious why my original code didn’t work in Puppeteer. Any ideas? Thanks!

2

yo, i’ve run into this too. youtube’s a pain with puppeteer cuz it loads stuff weird. try upping your wait time or use waitForSelector() instead. also, youtube might be onto you - try setting a user agent to trick it. your ytInitialData hack is smart tho, good thinking!

3

The issue you’re facing is likely due to YouTube’s dynamic content loading. Puppeteer runs in a headless environment by default, which can sometimes behave differently from a regular browser. To troubleshoot, try increasing the wait time or using page.waitForSelector() instead of a fixed timeout. You could also experiment with launching Puppeteer in non-headless mode to see what’s happening visually.

Another factor to consider is that YouTube might be serving different content to Puppeteer, possibly detecting it as a bot. In this case, you might need to set a user agent or use stealth plugins to mimic a real browser more closely.

Your ytInitialData approach is a smart workaround. It’s more reliable as it’s part of the initial page data. However, keep in mind that for frequently updated metrics like view count, it might not always reflect the most current data.

4

I’ve encountered similar issues with Puppeteer before, and it’s often related to timing or dynamic content loading. YouTube’s interface is complex and relies heavily on JavaScript to render elements.

You might try increasing the wait time since one second may not be enough for all elements to load. Instead of using waitForTimeout(), it’s helpful to use page.waitForSelector() to ensure the element is actually present before attempting to access it. Also, YouTube could be detecting Puppeteer as a bot, so launching it with {headless: false} may reveal what’s happening. The selector itself might change dynamically, so a more robust selector or XPath could be necessary.

Using ytInitialData is a clever workaround because it’s part of the initial page load, although it might not always reflect real-time changes.