How to extract localStorage data using Puppeteer?

Other Questions Puppeteer
1

I’m trying to use Puppeteer to grab all the data a website stores, including cookies and localStorage (especially after logging in). I’ve gone through the Puppeteer docs but can’t find anything about accessing localStorage.

I can get cookies just fine, but I’m stuck on how to retrieve localStorage data. Here’s a basic example of what I’m working with:

const puppeteer = require('puppeteer');

async function scrapeData() {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://example.com');

  // Login code would go here

  const cookies = await page.cookies();
  console.log(cookies);

  // How do I get localStorage here?
  // const localStorage = ???
  // console.log(localStorage);

  await browser.close();
}

scrapeData();

Any ideas on how to access localStorage using Puppeteer? I’d really appreciate some help figuring this out!

2

To extract localStorage data using Puppeteer, you can leverage the page.evaluate() method. This allows you to execute JavaScript code in the context of the page. Here’s how you can modify your code:

const localStorage = await page.evaluate(() => {
  let items = {};
  for (let i = 0; i < localStorage.length; i++) {
    const key = localStorage.key(i);
    items[key] = localStorage.getItem(key);
  }
  return items;
});

console.log(localStorage);

This snippet iterates through all localStorage items and returns them as an object. You can place this code right after your cookie extraction. Remember that localStorage is domain-specific, so ensure you’re on the correct page before attempting to access it. Also, some sites might use encryption for sensitive data in localStorage, so be prepared to handle that if necessary.

3

hey, u can use page.evaluate() to grab localStorage. just do smthin like:

const localData = await page.evaluate(() => Object.assign({}, localStorage));
console.log(localData);

this gets all the localStorage stuff in one go. hope it helps!

4

I’ve dealt with this issue before, and there’s actually a neat trick you can use. Instead of directly accessing localStorage, which can be tricky, you can inject a script into the page that captures the localStorage data and sends it back to Puppeteer. Here’s what I’ve found works well:

const localStorageData = await page.evaluate(() => {
const json = {};
for (let i = 0; i < localStorage.length; i++) {
const key = localStorage.key(i);
json[key] = localStorage.getItem(key);
}
return json;
});

console.log(localStorageData);

This approach has been reliable for me across different websites. Just make sure you’re on the right page after login before running this. Also, keep in mind that some sites might use sessionStorage instead of localStorage, so you might want to check for that too if you’re not getting the expected data.