How to validate HTTP status codes for website URLs in Airtable records

Other Questions Airtable
1

I’m working with an Airtable base that contains approximately 24,000 website URLs. Many of these links have formatting issues like missing slashes or extra spaces that cause them to break. I need a way to check which URLs are problematic so I can fix them manually.

Current approach
I’ve been using a fetch-based script to test each URL and get the response status:

const config = input.config();
const website = config.website;
let responseCode;

try {
    const result = await fetch(website);
    responseCode = result.status;
} catch (err) {
    responseCode = 'failed';
}

output.set('responseCode', responseCode);

Problems I’m facing

  1. When a URL redirects to another page, my script treats it as an error instead of following the redirect to check if it ultimately works
  2. I only get back either “200” for working URLs or “failed” for broken ones. I’d prefer to see the actual HTTP status codes (like 404, 500, etc.) to better understand what’s wrong

Any suggestions on how to improve this approach would be really helpful!

2

I’ve faced similar URL validation issues. The problem with your script is that fetch follows redirects by default, so you’re missing the intermediate status codes. A better approach would be:

const result = await fetch(website, { redirect: 'follow' });
if (result.redirected) {
    responseCode = `${result.status} (redirected from ${result.url})`;
} else {
    responseCode = result.status;
}

Additionally, to catch actual HTTP error codes, you should improve your error handling. Remember, network errors and HTTP errors are different; for example, a 404 is a successful fetch with an error status, while connection timeouts and DNS failures will throw exceptions. This approach will help you better identify problematic URLs for manual fixes.