Unexpected CORS issue with Puppeteer automation

Other Questions Puppeteer
1

Hey everyone, I’m having trouble with my Puppeteer script. It’s supposed to automate a purchase on a website, but I keep running into a CORS error. It happens in about 80% of my test runs.

The error message says something about ‘Access-Control-Allow-Origin’ header missing. I’m not sure why this is happening or how to fix it.

Here’s a simplified version of my code:

const puppeteer = require('puppeteer');

async function automate() {
  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();
  
  await page.goto('https://example.com/product-page');
  await page.click('#add-to-cart-button');
  await page.waitForSelector('#size-dropdown');
  await page.click('#size-dropdown');
  await page.click('#size-10');
  await page.click('#checkout-button');
  await page.click('#proceed-to-payment');
  
  await browser.close();
}

automate();

Any ideas on what might be causing this CORS issue? Thanks for your help!

2

hey there, have u tried using a headless browser like browserless? it might help bypass those pesky CORS issues. also, adding some random delays between actions could make ur script look more human-like. just a thought!

3

The CORS error you’re encountering is likely due to the website’s security measures. Puppeteer simulates a browser environment, but it doesn’t perfectly mimic a real user’s browser. Some websites have strict CORS policies to prevent automated interactions.

To address this, you could try setting a user agent string to make your requests appear more like a regular browser:

await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36');

Additionally, you might want to add some randomized delays between actions to mimic human behavior:

await page.waitForTimeout(Math.floor(Math.random() * 1000) + 500);

If these don’t work, you may need to look into using a proxy or rotating IP addresses to bypass the website’s anti-automation measures. Remember to respect the website’s terms of service and robot.txt file when automating interactions.

4

I’ve dealt with similar CORS issues in my Puppeteer projects. One thing that helped me was intercepting network requests and modifying headers. You can use page.setRequestInterception(true) and then add an event listener for ‘request’ events. In the listener, you can modify the headers to include the necessary CORS headers.

Here’s a snippet that might help:

await page.setRequestInterception(true);
page.on('request', request => {
  const headers = request.headers();
  headers['Access-Control-Allow-Origin'] = '*';
  request.continue({headers});
});

This approach worked for me in most cases. If it doesn’t solve your problem, you might want to look into using a headless browser that more closely mimics a real browser, like Playwright. It has better built-in support for handling these kinds of issues.