Duplicate entries appearing when pushing data to Airtable via API

Other Questions Airtable
1

I’m having trouble with my script that sends new student info to Airtable. It’s working, but it’s making duplicate entries. I’ve looked at the data I’m sending and can’t see any doubles.

Here’s what my code does:

function findNewStudents(studentList, existingStudents) {
  let newEntries = { records: [] };
  
  studentList.forEach(student => {
    if (['active', 'accepted'].includes(student.enrollmentStatus)) {
      if (!existingStudents.some(existing => existing.studentId === student.studentId)) {
        newEntries.records.push({ fields: { ...student } });
      }
    }
  });

  sendToAirtable('dbX123abc', 'StudentTable', newEntries);
}

function sendToAirtable(dbId, table, data) {
  const apiUrl = `https://api.airtable.com/v0/${dbId}/${encodeURIComponent(table)}`;
  const headers = {
    'Authorization': `Bearer ${airtableApiKey}`,
    'Content-Type': 'application/json'
  };

  const response = fetch(apiUrl, {
    method: 'POST',
    headers: headers,
    body: JSON.stringify(data)
  });

  return response.json();
}

Any ideas why it’s creating duplicates? I’m stumped!

2

hey jack, might be worth checking if ur script is running multiple times accidentally. i had a similar issue and it turned out my cron job was triggering the script more often than needed. also, try adding a unique identifier to each record, like studentId + timestamp. that way airtable can spot n skip dupes. good luck mate!

3

Have you considered implementing a debounce or throttle mechanism in your script? This could help prevent rapid-fire API calls that might be overwhelming Airtable’s processing capabilities. Another approach worth exploring is implementing a unique identifier for each student record, such as a combination of studentId and a timestamp. This way, even if a duplicate request is sent, Airtable could potentially recognize and ignore it. Additionally, you might want to double-check your existingStudents array - ensure it’s being properly populated and updated after each successful API call to maintain an accurate representation of your Airtable data.

4

I’ve run into this issue before with Airtable’s API. The problem might not be in your code, but in how Airtable handles concurrent requests. If your script runs multiple times in quick succession, Airtable might not have finished processing the first request before the second one comes in. This can lead to duplicates.

One solution is to introduce a delay between API calls or consider using an upsert-like functionality to identify records uniquely. Additionally, batching requests can minimize race conditions. Finally, ensure that your existingStudents array accurately reflects the current state to avoid flawed duplicate checks.