I’m trying to scrape the currently playing game from a Twitch streamer’s page but having trouble locating the right element. I want to build a simple script that can check what game any streamer is currently playing.
Here’s my current attempt:
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.firefox.options import Options
browser = webdriver.Firefox()
browser.get('https://www.twitch.tv/shroud')
page_content = browser.page_source
parser = BeautifulSoup(page_content, 'html.parser')
current_game = parser.find('a', {'data-a-target': True})
print(current_game)
The selector doesn’t seem to work properly and returns None. What’s the correct way to target the game title element on Twitch? Should I use a different approach or wait for specific elements to load before scraping?
Your BeautifulSoup selector’s way too broad - {'data-a-target': True} grabs any element with that attribute. I’ve scraped Twitch before and the game title’s buried in a specific span inside the channel info section. Try targeting elements with game-like text or hunt for the category section with the game link. But here’s the real issue - Twitch loads everything via JavaScript and you’re probably running before the game info even shows up. You need explicit waits for those specific elements, not just page load. Also heads up: some streamers don’t set a game or they’re in ‘Just Chatting’, so handle those cases or your script will break.
Twitch obfuscates their class names and uses dynamic loading, so finding the game element is a pain. Don’t search for just data-a-target - look for the specific attribute data-a-target="browse-link-card-game" which usually has the game info. You’ll need proper wait conditions since content loads asynchronously. Use WebDriverWait with presence_of_element_located before parsing with BeautifulSoup. I’ve had luck targeting the h2 element with the game title, but Twitch changes their DOM structure constantly so your selectors will break. Add error handling and fallback selectors to make your script more robust.
twitch api beats scraping every time. their site’s constantly changing and kills scrapers. if you’re set on scraping tho, hunt for data-test-selector attributes - way more reliable than those scrambled class names. and throw in a sleep() after pages load since twitch is slow as hell to render.
Scraping Twitch is a pain - they constantly change their DOM and you’re dealing with dynamic content. You’ll also hit rate limits quick when checking multiple streamers.
I ran into this building a dashboard to track what games our team streamed during hackathons. Instead of fighting selectors that break every week, I automated it with Latenode.
Built a workflow that hits the Twitch API every few minutes, pulls game titles from the JSON, and dumps everything in a database. No broken selectors, no browser bloat, way more reliable.
Best part? It handles offline streamers and category switches automatically. Someone goes offline or switches to “Just Chatting”? Updates without crashing your script.
You can expand it to track multiple streamers, get notifications when someone starts a specific game, or analyze trending games.
Had the same problem building a stream monitoring tool last year. Your selector’s fine - the real issue is Twitch loads content in phases. The game info comes after the initial page loads, so you’re grabbing the source too early before JavaScript finishes. Don’t parse right after page load. Use WebDriverWait to wait for the actual game elements to show up. I had better luck targeting the link with the game name instead of hunting for data attributes. Wait for elements with ‘playing’ text or check if the category section’s populated. Also, streamers who aren’t live or don’t have a game set will break your script. Build in checks for those cases from the start.