Retrieving over 100 records from Airtable API using pagination

Other Questions Airtable
1

I’m new to the Airtable API and I’m trying to fetch more than 100 records. I’ve managed to connect to the API using requests like this:

get_url = 'https://api.airtable.com/v0/BASE_ID/TABLE_NAME'
get_headers = {
'Authorization': 'Bearer API_KEY' }

response = requests.get(get_url, headers=get_headers)
response_table = response.json()

This works fine but only gets the first 100 records. I’ve heard about offset and pagination but I’m not sure how to use them with this code. Can someone explain how to implement pagination to retrieve all the records? I’d really appreciate any help or examples. Thanks!

2

hey, you can use the offset param to paginate. try this:

offset = None
while True:
    params = {} if not offset else {'offset': offset}
    response = requests.get(get_url, headers=get_headers, params=params)
    data = response.json()
    offset = data.get('offset')
    if not offset:
        break

this should fetch all records!

3

To retrieve more than 100 records using the Airtable API, you’ll need to implement pagination. Here’s a more robust approach that handles potential API errors and rate limits:

import time

all_records = []
offset = None

while True:
    params = {'offset': offset} if offset else {}
    
    try:
        response = requests.get(get_url, headers=get_headers, params=params)
        response.raise_for_status()
        data = response.json()
        
        all_records.extend(data['records'])
        offset = data.get('offset')
        
        if not offset:
            break
        
        time.sleep(0.2)  # Respect rate limits
    except requests.exceptions.RequestException as e:
        print(f'Error occurred: {e}')
        break

print(f'Total records retrieved: {len(all_records)}')

This code handles pagination, respects rate limits, and catches potential errors. It’s a more reliable way to fetch all records from your Airtable base.