I’m working on a web automation project with Puppeteer and I’m stuck on a couple of things.
First, I need to figure out how to send POST requests with form data while also setting custom headers. I’ve been looking through the docs but I’m not sure about the proper way to handle this.
Second, after getting the response back, I want to save it as a PDF document. I know Puppeteer has PDF generation features but I’m not clear on how to combine this with form submissions.
Has anyone done something similar? I’m particularly confused about working with the Request and Response objects in Puppeteer. What’s the right approach for handling form posts and then converting the result to PDF format?
Any code examples or guidance would be really helpful. Thanks!
Another approach worth considering is using page.setRequestInterception(true) combined with request.continue() to modify requests on the fly. This gives you more control over the entire request lifecycle. When you intercept the form submission request, you can override the postData and headers before allowing it to proceed. Something like await request.continue({postData: yourFormData, headers: customHeaders}). The advantage here is that you maintain the natural browser behavior while still customizing the request parameters. Once the form response loads, you might want to add a slight delay with await page.waitForTimeout(2000) before calling page.pdf() to ensure any dynamic content has finished loading. I found this method particularly useful when dealing with forms that have JavaScript validation or when the response includes charts or images that need time to render properly.
actually you can also use page.goto() with the POST method directly. just pass the postData and headers in the options object like page.goto(url, {method: 'POST', postData: 'field1=value1&field2=value2', headers: {'Content-Type': 'application/x-www-form-urlencoded'}}). once the page loads with your response data, call page.pdf({path: 'output.pdf'}) to save it. way simpler than evaluate imo
I ran into this exact scenario last year when automating report downloads from our company portal. The key is to use page.evaluate() to submit your form programmatically rather than trying to intercept network requests. You can set your custom headers using page.setExtraHTTPHeaders() before navigation, then use page.evaluate() to populate form fields and trigger submission. After the form processes and the response page loads, simply call page.pdf() with your desired options. The tricky part is timing - you need to wait for the response content to fully render before generating the PDF, so make sure to use page.waitForSelector() or page.waitForLoadState() to ensure everything is loaded. This approach worked much better than trying to manually construct POST requests through Puppeteer’s request interception.