Duplicate Records Being Generated When Sending Data to Airtable via API

Other Questions Airtable
1

I built a function that transfers student data from our internal system to Airtable using their API. The data transfer works fine, but I’m getting duplicate records created in Airtable even though my source data doesn’t contain any duplicates.

Here’s my code that identifies which students need to be added:

for (let student of enrolledStudents) {
    if (student.enrollment_status === 'active' || student.enrollment_status === 'pending') {
        const isNewStudent = !existingRecords.find(record => record.student_id === student.student_id);
        if (isNewStudent) {
            studentsToAdd.records.push({
                fields: {
                    ...student
                },
            });
        }
    }
    
    sendToAirtable('appXYZ123ABC', 'Enrollments', studentsToAdd);
}

And here’s my function that sends the data to Airtable:

function sendToAirtable(baseId, table, data) {
    var requestOptions = {
        method: 'POST',
        headers: {
            'Authorization': 'Bearer ' + airtableToken,
            'Content-Type': 'application/json',
            'Accept': 'application/json'
        },
        payload: JSON.stringify(data)
    };
    
    var result = UrlFetchApp.fetch('https://api.airtable.com/v0/' + baseId + '/' + encodeURIComponent(table), requestOptions).getContentText();
    return result;
}

I’ve verified my payload multiple times and there are no duplicate entries in the data I’m sending. Any ideas why Airtable keeps creating duplicates?

2

yeah, move the sendToAirtable call outside the loop. you’re sending each student one by one right now, which creates duplicates when the api processes requests at different speeds. batch them all up first, then send everything in one go.

3

I hit this exact problem a few months ago. Yeah, that API call in your loop is the main culprit.

But here’s what everyone else missed - you’re not clearing your studentsToAdd array between runs. If this function runs multiple times, you’ll keep piling up records in that array.

I had the same thing happen when our batch job ran twice daily and we kept getting mystery duplicates. The array was holding onto data from previous runs.

Here’s the fix:

let studentsToAdd = { records: [] }; // Reset this every time

for (let student of enrolledStudents) {
    if (student.enrollment_status === 'active' || student.enrollment_status === 'pending') {
        const isNewStudent = !existingRecords.find(record => record.student_id === student.student_id);
        if (isNewStudent) {
            studentsToAdd.records.push({
                fields: {
                    ...student
                },
            });
        }
    }
}

// Move this outside and add a check
if (studentsToAdd.records.length > 0) {
    sendToAirtable('appXYZ123ABC', 'Enrollments', studentsToAdd);
}

Add some logging too - you’ll probably find records getting sent multiple times when you actually trace it.

4

Had this exact same issue with Airtable integration last month. You’re calling sendToAirtable inside your loop, so it’s firing off separate API requests for each student instead of batching everything together. That creates race conditions where multiple requests run at once and skip your duplicate checks. Here’s what fixed it for me: initialize studentsToAdd before the loop, collect all records during the loop, then make one API call after it’s done. Also double-check that your existingRecords array is getting populated right - I’ve seen comparisons fail silently when data types don’t match between your source and Airtable’s response.

5

Your loop structure is the problem. You’re calling sendToAirtable inside the for loop, so it runs for every single student. That creates multiple API requests and duplicates, especially if some records are empty or already sent. I had the same issue last year - moving the API call outside the loop fixed it completely. Move sendToAirtable after the loop ends, and make sure studentsToAdd.records = [] gets initialized before the loop starts. Also double-check that existingRecords is pulling fresh data - stale data causes duplicates too.