I’m working on a Python application that connects to Airtable using the requests module to pull data and convert it to a Python list format.
The problem I’m facing is that my API calls only return 100 records maximum, even though my table has more entries. I’ve been looking into the Airtable documentation about pagination and maxRecords parameters, but I can’t figure out how to modify my request properly.
Here’s the API endpoint I’m currently using:
import requests
api_url = "https://api.airtable.com/v0/appX8H2MNkQzVDoWr/Users?api_key=YOUR_API_KEY"
response = requests.get(api_url)
data = response.json()
print(len(data['records'])) # Always shows 100 max
I know there should be a way to handle pagination or increase the record limit, but I’m stuck on the implementation. Has anyone dealt with this before? What’s the correct approach to retrieve all records from an Airtable base?
The 100-record limit is a hard constraint from Airtable’s API - you can’t just bump up maxRecords beyond that. You need to loop through the pagination tokens instead. Each response gives you an ‘offset’ field when there’s more data to grab. I hit this same problem last year and fixed it with a function that keeps requesting until there’s no offset returned. Just save the offset from each response and stick it in your next request URL. Combine all the records from each page into one list. Takes about 3-4 requests for tables with ~300 records. Don’t forget error handling and maybe throw in a small delay between requests so you don’t hit rate limits.
Yeah, pagination is your only option here. Airtable’s API won’t budge on that 100 record limit.
I just use a simple while loop that grabs all pages automatically:
import requests
def get_all_records(api_url):
all_records = []
offset = None
while True:
params = {}
if offset:
params['offset'] = offset
response = requests.get(api_url, params=params)
data = response.json()
all_records.extend(data['records'])
if 'offset' not in data:
break
offset = data['offset']
return all_records
Saved me hours last month pulling customer data from multiple bases. Call it once and you’re done.
Watch out though - if you’ve got massive tables (10k+ records), add some rate limiting or process records in batches. Don’t dump everything into memory at once.
