We maintain an Airtable database consisting of more than 24,000 records that correspond to various websites. Many of these URLs have encountered issues such as missing slashes or unnecessary spaces. Our goal is to identify these problematic websites prior to making any manual corrections.
What we have attempted
To date, we have utilized the fetch API to access each URL and obtain its error status. Here’s the code snippet we’ve implemented:
const config = input.config();
const website = config.url;
let httpStatus;
try {
const response = await fetch(website);
httpStatus = response.status;
} catch (err) {
httpStatus = 'error';
}
output.set('httpStatus', httpStatus);
Problems faced
- The script does not handle redirects, resulting in an ‘error’ status even if the destination URL is functional.
- Currently, we only receive a ‘200’ status code for functional URLs or ‘error’ messages, which does not provide the specific error codes we’d prefer.
We would greatly appreciate any assistance on how to enhance this process. Thank you!