Verifying HTTP response codes for URLs in an Airtable column

Other Questions Airtable
1

We maintain an Airtable database consisting of more than 24,000 records that correspond to various websites. Many of these URLs have encountered issues such as missing slashes or unnecessary spaces. Our goal is to identify these problematic websites prior to making any manual corrections.

What we have attempted
To date, we have utilized the fetch API to access each URL and obtain its error status. Here’s the code snippet we’ve implemented:

const config = input.config();  
const website = config.url;  
let httpStatus;  

try {  
    const response = await fetch(website);  
    httpStatus = response.status;  
} catch (err) {  
    httpStatus = 'error';  
}  

output.set('httpStatus', httpStatus);

Problems faced

  1. The script does not handle redirects, resulting in an ‘error’ status even if the destination URL is functional.
  2. Currently, we only receive a ‘200’ status code for functional URLs or ‘error’ messages, which does not provide the specific error codes we’d prefer.

We would greatly appreciate any assistance on how to enhance this process. Thank you!

2

hey emma, you might wanna try using a library like axios or node-fetch instead of fetch directly, they handle redirects more smoothly. also, check the response.statusText for more detailed error codes instead of just ‘error’. This might give a clearer picture of what’s goin wrong.