Puppeteer 24.1.0 and Chrome 132.0.6834.83 failing to access basic auth pages with client-side blocking error

Other Questions Puppeteer
1

I’m having trouble with Puppeteer 24.1.0 and Chrome 132.0.6834.83 in headless mode. They can’t get pages that need basic auth. I keep getting a net::ERR_BLOCKED_BY_CLIENT error.

I found this problem while testing our customer service site crawler. I even made a simple test case that shows the same issue.

Here’s what I’ve tried:

  • Puppeteer 22.15.0 to 24.0.0 work fine
  • Chrome versions up to 131.0.6778.264 are okay
  • Chrome headless shell versions all work

But Puppeteer 24.1.0 with Chrome 132.0.6834.83 and later versions fail.

I’m using Rocky Linux 9.5 and Apache 2.4.62. Node.js is version 20.18.2.

Is there a new setting I need for these newer versions? Or is this a bug?

I can share my test setup and code if needed. Any ideas on how to fix this?

2

yo, i’ve run into this too. seems like chrome 132 messed up basic auth in headless mode. try this:

await page.setExtraHTTPHeaders({
‘Authorization’: 'Basic ’ + Buffer.from(‘user:pass’).toString(‘base64’)
});

if that dont work, maybe downgrade to chrome 131 for now. pain in the butt, but it might be ur best bet til they fix it.

3

I’ve been wrestling with a similar issue in our web scraping project. From what I’ve gathered, there’s been a change in how Chrome 132+ handles basic auth in headless mode.

One workaround that’s worked for us is setting up a custom Chrome executable path. We downloaded an older Chrome version (131.0.6778.264) and pointed Puppeteer to use that instead. It’s not ideal, but it got us back up and running.

Another thing to try is explicitly setting the authentication headers before navigation:

await page.setExtraHTTPHeaders({
‘Authorization’: 'Basic ’ + Buffer.from(‘username:password’).toString(‘base64’)
});

If neither of these work, you might want to consider using a proxy server that handles the authentication before passing requests to your target site. It’s a bit more setup, but it can sidestep browser-level auth issues.

Have you tried running Chrome in non-headless mode to see if the problem persists? Sometimes that can provide clues about what’s going on. If all else fails, downgrading to Puppeteer 24.0.0 might be your best bet until this gets resolved upstream.

4

I’ve encountered a similar issue with Puppeteer and Chrome recently. It seems the newer versions have indeed changed how they handle basic auth in headless mode. One workaround I found effective was to use the page.authenticate() method before navigating to the page. Something like:

await page.authenticate({
  username: 'your_username',
  password: 'your_password'
});
await page.goto('your_url');

If that doesn’t work, you might want to try setting the --disable-web-security flag when launching Chrome. This bypasses some security measures, but use it cautiously:

const browser = await puppeteer.launch({
  args: ['--disable-web-security']
});

Lastly, check if your Apache server’s configuration for basic auth has changed. Sometimes server-side changes can cause unexpected client-side errors. If none of these solve it, it might be worth filing a bug report with the Puppeteer team.