I’m working with HubSpot’s contact API and running into the pagination limit. The API restricts me to fetching only 100 contacts at a time.
Here’s my current approach for the first batch:
$first_batch = $hubspot_client->fetch_contacts(array('limit' => '100'));
For the second batch, I use the offset like this:
$next_offset = $first_batch->{'contact-offset'};
$second_batch = $hubspot_client->fetch_contacts(array('limit' => '100', 'contactOffset' => $next_offset));
The problem is I need to retrieve thousands of contacts, but I don’t want to manually create separate variables for each batch. I’m looking for a way to automatically extract the offset from each response and use it in the next API call. What’s the best approach to loop through all pages and collect the complete contact list without creating individual variables for each batch?
I use a pagination loop with offset tracking. Start with offset as null, then keep updating it from each API response. Here’s what works: create an array for all contacts, then loop through API calls using the current offset. Append each batch to your main array and grab the new offset for the next call. Stop when there’s no offset returned or when contacts are less than your limit. I throw in a 200ms sleep between calls for rate limits. This handles 50k+ contacts without memory problems and you don’t need to juggle multiple batch variables.
Just dealt with this on a client migration project. Build an iterator function that handles state management automatically. I create a simple function that takes the HubSpot client as a parameter and yields batches as it goes. Initialize your offset variable and total contacts array, then loop while the API returns data. Each iteration updates the offset from the response and merges new contacts into your master collection. You can process contacts in chunks without loading everything into memory at once - critical for large databases. I’ve pulled 100k+ contacts this way without memory issues or timeouts.
I’ve hit this same issue with HubSpot’s API. Use a while loop that runs until there’s no more data. Check for the ‘has-more’ property in the response, or see if you got fewer contacts than your limit - that’s when you stop. Dump all contacts into one array and keep adding to it each loop. Don’t forget error handling and throw in a small delay between requests so you don’t slam their rate limits. I always set a max iteration count too - saves you from infinite loops if their API acts up. This method’s never failed me when pulling big contact lists from HubSpot.
Use a do-while loop and keep fetching until the offset comes back empty or null. This worked for me - start with an empty offset, make the call, grab the new offset from response, repeat. Store everything in one array. HubSpot returns null offset when you’re at the end, so that’s your exit condition. Way cleaner than tracking separate batch variables.