How to fetch YouTube comments for multiple video IDs using YouTube Data API v3 through RapidAPI

Other Questions RapidAPI
1

I’m building a comment scraper for YouTube videos and need help with batch processing. Right now my code only handles one video at a time but I have hundreds of video IDs to process.

import requests
import json

def fetch_comments_batch(video_list):
    api_endpoint = "https://youtube-v31.p.rapidapi.com/commentThreads"
    
    request_headers = {
        'x-rapidapi-key': "your-api-key-here",
        'x-rapidapi-host': "youtube-v31.p.rapidapi.com"
    }
    
    for vid in video_list:
        params = {
            "maxResults": "100",
            "videoId": vid,
            "part": "snippet"
        }
        
        result = requests.get(api_endpoint, headers=request_headers, params=params)
        print(result.json())

video_ids = ["video1", "video2", "video3"]
fetch_comments_batch(video_ids)

The current setup requires me to manually change the videoId parameter each time. I want to pass an array of video IDs and automatically collect comments from all of them in one go. Is there a way to modify the API call to accept multiple video IDs at once, or do I need to loop through them individually? What’s the most efficient approach for handling large datasets of video IDs?

2

ur def right about the loop! make sure to add error handling and maybe some delays. rapidapi has rate limits that can hurt if u go too fast. also, try saving results to a list instead of just printing - you’ll be glad u did later!

3

The YouTube Data API doesn’t support batch requests for comments, so your loop approach is correct. But here are some key optimizations for handling hundreds of videos. First, handle pagination properly - you’re only getting 100 comments per request. Most videos have thousands of comments that need multiple API calls using nextPageToken. Second, add solid error handling for disabled comments, private videos, and quota limits. I learned this the hard way after losing hours of progress to unhandled exceptions on large datasets. For better performance, use threading or asyncio for concurrent requests (just respect rate limits). Also, save results to JSON files or a database instead of printing. This lets you resume if something breaks and makes the data way more useful for analysis.