Is there a way to retrieve more than 100 entries from Airtable using Axios?

Other Questions Airtable
1

I’m working on a project where I need to pull all the data from my Airtable base. The problem is I can only get 100 records at a time. I’ve tried using Axios to make the API call, but I’m stuck on how to fetch everything. Here’s what I’ve got so far:

function getAllEntries() {
  const baseId = 'your_base_id_here';
  const apiKey = 'your_api_key_here';
  const url = `https://api.airtable.com/v0/${baseId}/MyTable?view=GridView`;

  axios.get(url, {
    headers: { Authorization: `Bearer ${apiKey}` }
  }).then(response => {
    console.log(response.data.records);
    // But this only gives me 100 records!
  }).catch(error => {
    console.error('Oops, something went wrong:', error);
  });
}

Does anyone know how to get all the records? Is there a way to loop through pages or something? Any help would be awesome!

2

hey CreativePainter33, i’ve had the same issue before. have you tried using the ‘pageSize’ parameter? you can set it to like 1000 or something. then use a while loop to keep fetching until you get all records. something like:

let allRecords = [];
let offset = null;
do {
  const response = await axios.get(url + `&pageSize=1000&offset=${offset}`...);
  allRecords = [...allRecords, ...response.data.records];
  offset = response.data.offset;
} while (offset);

hope that helps!

3

You’re on the right track with Axios, but you’ll need to implement pagination to fetch all records. Airtable’s API uses an ‘offset’ parameter for this. Here’s how you can modify your code:

async function getAllEntries() {
  const baseId = 'your_base_id_here';
  const apiKey = 'your_api_key_here';
  const baseUrl = `https://api.airtable.com/v0/${baseId}/MyTable?view=GridView`;
  let allRecords = [];
  let offset = null;

  do {
    const url = offset ? `${baseUrl}&offset=${offset}` : baseUrl;
    try {
      const response = await axios.get(url, {
        headers: { Authorization: `Bearer ${apiKey}` }
      });
      allRecords = allRecords.concat(response.data.records);
      offset = response.data.offset;
    } catch (error) {
      console.error('Error fetching data:', error);
      break;
    }
  } while (offset);

  console.log(`Retrieved ${allRecords.length} records`);
  return allRecords;
}

This function will keep making requests until there’s no more ‘offset’, effectively retrieving all records. Remember to handle rate limits if you have a large dataset.

4

I’ve tackled this issue before, and there’s a nifty solution using the Airtable.js library. It’s much simpler than manually handling pagination with Axios. Here’s what worked for me:

const Airtable = require('airtable');
const base = new Airtable({apiKey: 'your_api_key_here'}).base('your_base_id_here');

function getAllRecords() {
  return new Promise((resolve, reject) => {
    let allRecords = [];
    base('MyTable').select({
      view: 'GridView'
    }).eachPage(function page(records, fetchNextPage) {
      allRecords = [...allRecords, ...records];
      fetchNextPage();
    }, function done(err) {
      if (err) {
        reject(err);
      } else {
        resolve(allRecords);
      }
    });
  });
}

getAllRecords()
  .then(records => console.log(`Retrieved ${records.length} records`))
  .catch(error => console.error('Error:', error));

This approach automatically handles pagination and rate limiting. It’s been incredibly reliable for me, even with large datasets. Just make sure to install the Airtable package first with npm or yarn.