Setting up custom proxy server with Puppeteer browser automation

[aroberts]

I’m trying to configure a custom proxy with Puppeteer but running into issues. Here’s my proxy setup:

const httpModule = require('http');
const proxyServer = require('http-proxy');
const urlParser = require('url');

const myProxy = proxyServer.createProxyServer({});
myProxy.on('proxyReq', function (proxyRequest, request, response, opts) {
    response.setHeader("Custom-Header", "9876");
});

myProxy.listen(7788);

httpModule.createServer(function (request, response) {
    const parsedUrl = urlParser.parse(request.url);
    const destination = parsedUrl.protocol + "//" + parsedUrl.host;
    myProxy.web(request, response, { target: destination });
}).listen(4222);

For the browser launch configuration I’m using:

const browserConfig = {
    args: [
        "--no-sandbox",
        "--disable-setuid-sandbox",
        "--proxy-server=localhost:4222",
    ],
    timeout: 0,
    ignoreHTTPSErrors: true
};

When I try to navigate to a page, I get this error:

Error: net::ERR_EMPTY_RESPONSE at http://example.com/
    at navigate (.../node_modules/puppeteer/lib/Page.js:598:37)

Any ideas what might be causing this connection issue? Thanks for any help!

[Finn_Mystery]

Your proxy chain setup is the problem. You’ve got two servers (7788 and 4222), but your HTTP server can’t extract the target from the request URL - that won’t work with Puppeteer’s requests. Puppeteer sends requests differently through proxies than you’re expecting. Try this: just use http-proxy directly on port 4222. Skip the HTTP server wrapper entirely. Also, add some console.log statements in your request handler to see if the proxy’s actually getting requests. ERR_EMPTY_RESPONSE usually means the connection starts but gets dropped immediately. Your proxy logic is probably failing silently somewhere in the request processing.

[Alex_Brave]

your proxy event handling’s missing an ‘error’ listener on myProxy - that’ll cause empty responses. also, ditch that timeout: 0 setting or bump it to something reasonable like 30000. and parsedUrl.protocol can be undefined sometimes, so watch out for that.

[benmoore]

Had the same issue with custom Puppeteer proxies. Your HTTP server’s the problem - you’re parsing the full URL then messing up the target reconstruction. Puppeteer doesn’t send URLs the way you think it does. Just pass the original request URL straight to your proxy target. Don’t parse and rebuild it. Also, your proxy probably isn’t handling HTTPS right, and most sites use HTTPS now. You’ll need SSL handling or CONNECT request support for HTTPS traffic. Add some logging to see if requests even reach your proxy before that ERR_EMPTY_RESPONSE hits. That’ll tell you if it’s a connection issue or something else.