How to retrieve complete dataset from Airtable when records exceed 100 using axios requests?

Other Questions Airtable
1

I’m working with Airtable API and facing an issue where I can only pull 100 records at a time. My database has way more entries than that and I need to get all of them in my application. The API seems to have some kind of limit but I’m not sure how to work around it.

fetchAllData(){
  const instance = this;
  const base_id = "**********";
  const api_token = "**********";
  
  axios.get(
      "https://api.airtable.com/v0/"+base_id+"/Monthly%20Data?view=Grid%20View",
      {
          headers: { Authorization: "Bearer "+api_token }
      }
  ).then(function(result){
    console.log(result.data.records);
    instance.dataList = result.data.records;
  }).catch(function(err){
    console.log(err)
  });
}

Is there a way to paginate through all the results or increase the limit somehow? Any help would be great!

2

Had this exact problem last month pulling customer data for a dashboard. You need a loop that keeps going until it hits the end.

Here’s what works:

fetchAllData() {
  const instance = this;
  const base_id = "**********";
  const api_token = "**********";
  let allRecords = [];
  
  function makeRequest(offset = null) {
    let url = `https://api.airtable.com/v0/${base_id}/Monthly%20Data?view=Grid%20View`;
    if (offset) url += `&offset=${offset}`;
    
    return axios.get(url, {
      headers: { Authorization: `Bearer ${api_token}` }
    }).then(result => {
      allRecords = [...allRecords, ...result.data.records];
      
      if (result.data.offset) {
        return makeRequest(result.data.offset);
      }
      return allRecords;
    });
  }
  
  makeRequest().then(records => {
    instance.dataList = records;
    console.log(`Got ${records.length} total records`);
  }).catch(err => console.log(err));
}

Collects everything in one array and handles recursion cleanly. Way better than managing state across multiple separate calls.

3

totally get it, that 100 limit can be a pain! u just gotta keep callin with the offset until ya pull in all your data. its like a never-ending loop lol, just keep adding to your array from each response. it’ll work out in the end!

4

To get all records from Airtable past the first 100, you need to use pagination. Airtable’s API sends an offset parameter when there are more records than the limit allows. Make your first call, then check if there’s an offset in the response - if yes, there are more records to fetch. Keep making requests with ?offset=${offset} added to your URL until the response doesn’t include an offset anymore. You might want to add a small delay between requests so you don’t hit rate limits.

5

Airtable’s API paginates with an offset parameter when you’ve got more than 100 records. You’ll get an offset value back that you use for your next request. I’ve hit this same issue multiple times - a recursive function is your best bet. Check if result.data.offset exists in your response. If it does, make another request with that offset as a query parameter. Keep collecting records from each call until there’s no more offset. That’s how you pull thousands of records instead of getting stuck at 100.