Navigating to webpage with Puppeteer results in ERR_EMPTY_RESPONSE

Other Questions Puppeteer
1

I’m having trouble with Puppeteer. I’m trying to visit a website, but I keep getting an error. Here’s what I did:

const puppeteer = require('puppeteer');

async function browsePage() {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  page.on('response', resp => {console.log(resp.request().url())});
  page.on('error', err => {console.error(err.message)});
  await page.goto('https://www.example.com');
  await browser.close();
}

browsePage();

But when I run this, I get:

UnhandledPromiseRejectionWarning: Error: net::ERR_EMPTY_RESPONSE at https://www.example.com

The weird thing is, the page loads fine in my regular browser. It takes about 5-7 seconds and makes around 135 requests.

I’m using Puppeteer 1.10.0 on macOS High Sierra 10.13.6 with Node.js 10.13.

Any ideas why this isn’t working? Is it a Puppeteer issue, a Chrome problem, or am I doing something wrong?

2

Have you tried increasing the timeout and adjusting the waitUntil option? Sometimes complex sites need more time to load fully. Here’s what I’d suggest:

await page.goto(‘https://www.example.com’, {
waitUntil: ‘networkidle2’,
timeout: 60000
});

This gives the page up to 60 seconds to load and waits until there are no more than 2 network connections for at least 500 ms. If that doesn’t work, you might want to check if the site is using any bot detection that could be blocking Puppeteer. Setting a custom user-agent or running in non-headless mode might help bypass that.

3

I’ve encountered similar issues with Puppeteer before. One thing that often helps is adjusting the network idle settings. Try modifying your page.goto() call like this:

await page.goto(‘https://www.example.com’, { waitUntil: ‘networkidle0’, timeout: 60000 });

This tells Puppeteer to wait until there are no more than 0 network connections for at least 500 ms before considering navigation complete. The longer timeout (60 seconds) gives slower sites more time to load.

Another trick is to add a small delay before closing the browser:

await page.goto(‘https://www.example.com’, { waitUntil: ‘networkidle0’, timeout: 60000 });
await page.waitForTimeout(5000);
await browser.close();

This extra pause can sometimes allow final requests to complete. If you’re still having trouble after trying these, it might be worth investigating if the site is actively blocking headless browsers.

4

hey, i’ve run into this before. try setting a custom user-agent, like:

await page.setUserAgent(‘Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36’);

some sites block headless browsers. this might trick it into thinking ur using a normal browser. worth a shot!