I’m working on a VB.NET application that uses a WebBrowser control to load Twitch stream pages. My goal is to scrape the current viewer count from the loaded page and display it in a label on my form.
I’ve been trying different approaches but can’t seem to find the right way to extract this specific data from the webpage. The viewer count appears on the stream page but I’m struggling to access it programmatically.
Here’s what I’m trying to achieve:
Dim streamUrl As String = "https://twitch.tv/somechannel"
WebBrowser1.Navigate(streamUrl)
' Need code here to extract viewer count
Label1.Text = "Viewers: " & viewerCount
What’s the best method to parse the HTML content and grab the viewer number from the loaded Twitch page? Any guidance on accessing the WebBrowser document elements would be really helpful.
honestly, twitch’s api is ur best bet. scraping their site will b a nightmare - they’ve got dynamic loading n anti-bot stuff that’ll block u. just use their helix api instead. you can grab viewer counts with simple http requests rather than wrestling with browser controls.
Been there with Twitch scraping - it’s a pain. The viewer counts load through JavaScript after the page loads, so WebBrowser won’t grab them right away. You need to wait for the JS to run and fill in those elements. I’d throw a timer in there to delay scraping by a few seconds after the page finishes loading. Just heads up though - Twitch constantly changes their HTML structure and class names specifically to break scrapers, so your code will probably stop working at some point. Use WebBrowser1.Document.GetElementsByTagName to hunt for the viewer count elements in the DOM, but plan on updating your code pretty regularly when they switch things up.
WebBrowser control works, but you’ll hit timing issues since Twitch loads everything dynamically. I handled the DocumentCompleted event, then used Document.GetElementsByClassName or GetElementById to grab the viewer count element. The tricky part’s waiting for all scripts to load - I used a background worker with multiple attempts over several seconds. You can inspect the page source to find the viewer count element, but expect to maintain this regularly since they change their layout all the time. Also heads up - viewer counts might be in different spots depending on if the stream’s live or not.