How to iterate through several Airtable databases in Next.js getStaticProps

[davidw]

I’m working on a Next.js project where I need to fetch data from multiple Airtable databases inside getStaticProps. Right now my code works fine for a single database, but I can’t figure out how to properly loop through an array of database IDs.

I have an array of database identifiers like ["db001", "db002", "db003"] and I want to fetch records from each one. Here’s my current working code for one database:

const tableData = await airtableClient
  .base("db001")("Project Details")
  .select()
  .firstPage();

const projects = tableData.map((project) => {
  return {
    location: project.get("Location") || null,
    title: project.get("Title") || null,
    status: project.get("Status") || null,
  };
});

return {
  props: {
    projects,
  },
};

I tried a manual approach that works but isn’t scalable:

export async function getStaticProps() {
  const firstDbRecords = await getProjectBase("db001")
    .select({})
    .firstPage();
  const secondDbRecords = await getProjectBase("db002")
    .select({})
    .firstPage();

  const allProjects = [];
  allProjects.push(processRecords(firstDbRecords));
  allProjects.push(processRecords(secondDbRecords));

  return {
    props: {
      allProjects,
    },
  };
}

My helper function looks like this:

const getProjectBase = (dbId) =>
  airtableClient.base(dbId)("Project Details");

I think I need to use async/await with a loop but I’m not sure about the best approach. Any suggestions?

[henryg]

You could also use a simple for loop if you want better error handling. I tried Promise.all but when one db fails, everything crashes. This way you handle each db separately:

const projects = [];
for (const dbId of databaseIds) {
  try {
    const records = await getProjectBase(dbId).select().firstPage();
    projects.push(...processRecords(records));
  } catch (err) {
    console.log(`failed to fetch ${dbId}:`, err);
  }
}

Slower than parallel processing but way more reliable.

[liamj]

I hit the same problem building a portfolio site that pulled from multiple Airtable workspaces. Rate limiting got me - Airtable caps you at 5 requests per second per base, so hitting multiple databases at once triggers throttling.

For production, I’d add delays between requests or go with something more robust:

export async function getStaticProps() {
  const databaseIds = ["db001", "db002", "db003"];
  const allProjects = [];
  
  for (let i = 0; i < databaseIds.length; i++) {
    const dbId = databaseIds[i];
    const tableData = await airtableClient
      .base(dbId)("Project Details")
      .select()
      .firstPage();
    
    const processedData = tableData.map(project => ({
      dbId, // useful for debugging
      location: project.get("Location") || null,
      title: project.get("Title") || null,
      status: project.get("Status") || null,
    }));
    
    allProjects.push(...processedData);
    
    // Small delay to avoid rate limits
    if (i < databaseIds.length - 1) {
      await new Promise(resolve => setTimeout(resolve, 200));
    }
  }
  
  return { props: { projects: allProjects } };
}

This sequential approach works way better for me, especially with larger datasets or sketchy network conditions.

[Neo_Movies]

Hit the same issue building a client dashboard that pulled from multiple Airtable bases. The tricky part was managing data structure when combining results from different databases.

Your helper function works well, but I’d pull out the record processing logic too:

const processTableData = (tableData, dbId) => {
  return tableData.map((project) => ({
    sourceDb: dbId, // helps with debugging later
    location: project.get("Location") || null,
    title: project.get("Title") || null,
    status: project.get("Status") || null,
  }));
};

export async function getStaticProps() {
  const databaseIds = ["db001", "db002", "db003"];
  const projectsPromises = databaseIds.map(async (dbId) => {
    const tableData = await getProjectBase(dbId).select().firstPage();
    return processTableData(tableData, dbId);
  });
  
  const allProjectArrays = await Promise.all(projectsPromises);
  const projects = allProjectArrays.flat();
  
  return { props: { projects } };
}

Keeps your main function cleaner and makes tweaking the data processing way easier. That sourceDb field saved me hours debugging when I needed to track where records came from.

[charlottek]

Yeah, you’re on the right track with async/await. I’ve dealt with this exact thing pulling data from multiple sources for our dashboard.

Cleanest approach is Promise.all() with map(). Here’s how I’d refactor your code:

export async function getStaticProps() {
  const databaseIds = ["db001", "db002", "db003"];
  
  const allProjects = await Promise.all(
    databaseIds.map(async (dbId) => {
      const tableData = await airtableClient
        .base(dbId)("Project Details")
        .select()
        .firstPage();
      
      return tableData.map((project) => ({
        location: project.get("Location") || null,
        title: project.get("Title") || null,
        status: project.get("Status") || null,
      }));
    })
  );
  
  // Flatten the array if you want all projects in one list
  const flattenedProjects = allProjects.flat();
  
  return {
    props: {
      projects: flattenedProjects,
    },
  };
}

This runs all your database calls in parallel instead of sequentially - way faster. Need them grouped by database? Just return allProjects without flattening.

I hit some tricky stuff with loops and async operations before. This video helped me understand the different patterns:

Loops with Async / Await

Watch out though - if any databases might be empty or throw errors, wrap each call in try/catch inside the map function.