How to fetch all records from Airtable when there are more than 100 entries

Other Questions Airtable
1

I’m working on a Python application that connects to my Airtable database using the requests library. The goal is to pull all the data and convert it into a Python list for further processing.

The problem I’m facing is that my API call only returns 100 records, even though my table contains way more entries. I know there should be around 300+ records in total.

Here’s the API endpoint I’m currently using:

import requests

api_url = "https://api.airtable.com/v0/appX8K2MnQwRtVsLp/Employees?api_key=YOUR_API_KEY"
response = requests.get(api_url)
data = response.json()
print(len(data['records']))  # This only shows 100

I’ve heard about using pagination and the maxRecords parameter, but I’m not sure how to implement this properly in my code. Has anyone dealt with this limitation before? What’s the best approach to get all records from a large Airtable base?

2

Yeah, Airtable caps at 100 records per request. You’ve got to handle pagination with the offset parameter they send back.

Here’s how I do it:

import requests

def get_all_airtable_records(base_url, headers):
    all_records = []
    offset = None
    
    while True:
        params = {}
        if offset:
            params['offset'] = offset
            
        response = requests.get(base_url, headers=headers, params=params)
        data = response.json()
        
        all_records.extend(data['records'])
        
        # Check if there's more data
        if 'offset' not in data:
            break
            
        offset = data['offset']
    
    return all_records

Just check for the offset field in the response. When there’s more data, Airtable includes this token - pass it in your next request.

I hit this same problem last year pulling employee data for a dashboard. Only 100 records showed up, then I realized we’d hired way more people than that.

Don’t bother with maxRecords to get more than 100 at once - Airtable won’t let you anyway.