How to retrieve all data entries from Airtable when count exceeds 100 using axios requests?

Other Questions Airtable
1

I’m working with Airtable API and running into a limitation issue. When I make requests to get data from my base, I only receive 100 entries maximum even though my table contains way more records than that. I need to pull all the available data, not just the first 100 rows. Here’s my current implementation:

fetchAllData(){
  const instance = this;
  const baseId = "**********";
  const apiToken = "**********";
  axios.get(
      "https://api.airtable.com/v0/"+baseId+"/Monthly%20Data?view=Grid%20View",
      {
          headers: { Authorization: "Bearer "+apiToken }
      }
  ).then(function(result){
    console.log(result.data.records);
    instance.dataList = result.data.records;
  }).catch(function(err){
    console.log(err)
  });
}

Is there a way to get past this 100 record limit? What’s the best approach to fetch complete datasets from Airtable?

2

I hit this same wall about 2 years ago when pulling customer data for a quarterly report. Ended up with around 800 records but only got the first 100.

Here’s what worked for me - I wrapped the whole thing in a function that keeps track of all records:

fetchAllData() {
  const instance = this;
  const baseId = "**********";
  const apiToken = "**********";
  let allRecords = [];
  
  function fetchBatch(offset = null) {
    let url = `https://api.airtable.com/v0/${baseId}/Monthly%20Data?view=Grid%20View`;
    if (offset) url += `&offset=${offset}`;
    
    return axios.get(url, {
      headers: { Authorization: "Bearer " + apiToken }
    }).then(function(result) {
      allRecords = allRecords.concat(result.data.records);
      
      if (result.data.offset) {
        return fetchBatch(result.data.offset);
      } else {
        instance.dataList = allRecords;
        return allRecords;
      }
    });
  }
  
  return fetchBatch();
}

The key difference is building up the allRecords array and only setting your final data when there’s no more offset to follow.

One heads up - if you’re dealing with thousands of records, this can take a while. I usually throw in a loading indicator so users know something’s happening.

3

Had this exact problem when migrating data from multiple bases. The solution involves modifying your function to handle pagination properly. Check if result.data.offset exists in your response - when it does, you need to make another request appending &offset=${offsetValue} to your URL. I typically use a while loop that keeps fetching until no offset is returned, pushing each batch of records into an array. One thing to watch out for is rate limiting - Airtable caps requests at 5 per second, so consider adding a small delay between calls if you’re dealing with massive datasets. Also make sure to handle errors gracefully since you’ll be making multiple sequential requests.

4

The 100 record limit is Airtable’s default pagination constraint. You need to implement pagination logic by checking for the offset parameter in the response. When present, this indicates there are additional records to fetch. Create a recursive function that continues making requests until no offset is returned. Each subsequent request should include the offset from the previous response as a query parameter. I encountered this same issue when pulling inventory data from a large table - the key is building a loop that accumulates all records across multiple API calls rather than expecting everything in a single request.

5

yep, u got it! each request returns an offset if there’s more data to fetch. just keep making calls with that offset till u hit the end. that’s how u get the whole dataset without missing any records. gl!