How can I download a file specified by the Content-Disposition header using a headless browser like PhantomJS and CasperJS?

Other Questions Headless Browser
1

I need help with downloading files that use the Content-Disposition header while using a headless browser. I am making requests and receiving responses that include the header for file attachments. However, I’m unsure about the proper way to save these files to my computer.

Here’s how my response object appears:

Response {
    "contentType": "text/plain",
    "headers": [
        {
            "name": "Cache-Control",
            "value": "private"
        },
        {
            "name": "Content-Length",
            "value": "256"
        },
        {
            "name": "Content-Type",
            "value": "text/plain"
        },
        {
            "name": "Server",
            "value": "Microsoft-IIS/7.0"
        },
        {
            "name": "Content-Disposition",
            "value": "attachment; filename=\"ContactList_08-25-14.csv\""
        },
        {
            "name": "Date",
            "value": "Mon, 25 Aug 2014 12:19:25 GMT"
        }
    ],
    "id": 104,
    "status": 200,
    "statusText": "OK",
    "url": "http://xxxxxxxxxx.com/xxxxxxxx/ExportSubscribers.aspx"
}

What are the steps to retrieve the filename from the Content-Disposition header and successfully download the file content? I’ve attempted various methods but haven’t found anything that works.

2

Had the same problem with headless browser downloads. Most solutions miss the key part - you’ve got to handle the response encoding properly or your files get corrupted. Set it to binary before processing. For PhantomJS, use page.onResourceReceived to catch the response and look for the Content-Disposition header. When you find it, make another request with binary encoding. Don’t use page.evaluate() - it can’t handle binary data. Use the native HTTP module to re-fetch the URL with proper encoding, then grab the filename and save. Watch out for encoded filenames in headers too. Some servers use RFC 5987 encoding that needs special handling.

3

Here’s what worked for me: use response.body to get the file content after parsing the header. Extract the filename with regex on the Content-Disposition header, then write the response body to disk with the filesystem module. In CasperJS: var fs = require('fs'); var filename = response.headers.filter(function(h) { return h.name === 'Content-Disposition'; })[0].value.match(/filename="(.+)"/)[1]; fs.write(filename, response.body, 'wb'); The ‘wb’ flag is crucial - it writes in binary mode and keeps file integrity intact, especially for non-text files.

4

been there! parse the content-disposition header to grab the filename, then write the response body to a file. try var filename = response.headers['content-disposition'].match(/filename="(.+)"/)[1]; and use fs.write() to save it. works great in phantomjs.