Issues with Puppeteer not saving cookies after using page.setCookie() and rebooting the browser

Other Questions Puppeteer
1

I’m experiencing an issue with my Puppeteer script where I set cookies for an authenticated session and then close the browser. When I reopen it later, the cookies don’t seem to be retained. However, if I authenticate during the session, I can still access the account upon reopening the browser. I suspect it may be related to user data directory settings but I’m unsure about it. Can someone provide insights on this problem?

(async () => {
    const newBrowser = await launchBrowser();
    const newTab = await newBrowser.newPage();

    // Load cookies to the page if available
    const storedCookies = fetchStoredCookies();
    if (storedCookies) {
        const parsedCookies = JSON.parse(storedCookies);
        const standardizedCookies = parsedCookies.map(
            ({ name, value, domain, path, expires, httpOnly, secure, sameSite }) => ({
                name,
                value,
                domain,
                path,
                expires,
                httpOnly,
                secure,
                sameSite,
            })
        );

        await newTab.setCookie(...standardizedCookies);
    }

    // Load the target URL
    await newTab.goto(TARGET_URL, {
        waitUntil: 'domcontentloaded',
        timeout: TIMEOUT_LIMIT,
    });

    await pause(5000);

    await newBrowser.close();
    await pause(5000);

    const secondBrowser = await launchBrowser();
    const secondTab = await secondBrowser.newPage();

    await secondTab.goto(TARGET_URL, {
        waitUntil: 'domcontentloaded',
        timeout: TIMEOUT_LIMIT,
    });
})();

export const launchBrowser = async ({
    disableUserData = false,
    isHeadless = false,
    additionalArgs = []
} = {}) => {
    if (activeBrowser) {
        return activeBrowser;
    }

    const launchOptions = {
        headless: isHeadless,
        ...(!disableUserData && {
            userDataDir: './session_data/user_data',
        }),
        args: [
            '--disable-infobars',
            '--disable-notifications',
            '--disable-default-browser-check',
            ...additionalArgs,
        ],
    };

    activeBrowser = await puppeteer.launch(launchOptions);

    return activeBrowser;
};

Node Version: 21.2.0
Puppeteer Dependencies:
“puppeteer”: “^22.8.2”,
“puppeteer-extra”: “^3.3.6”,
“puppeteer-extra-plugin-session”: “^1.0.1”,
“puppeteer-extra-plugin-stealth”: “^2.11.2”,
“puppeteer-extra-plugin-user-preferences”: “^2.4.1”,

I have attempted the following solutions:

  1. Used both page.setCookie(…args) and network.send(“setCookie”, cookies).
  2. Tried manually modifying the SQLite database in userDataDir, but it did not yield any results.
2

Hi FlyingLeaf, it looks like the issue might stem from inconsistent usage of the userDataDir option, which is crucial for persisting cookies across browser restarts.

Here's a streamlined approach to consistently manage the user data directory:

(async () => { const browserOptions = { userDataDir: './session_data/user_data', // Ensure consistency here headless: false, }; 
const firstBrowser = await puppeteer.launch(browserOptions);
const firstTab = await firstBrowser.newPage();
// Add your cookie handling here
await firstBrowser.close();

const secondBrowser = await puppeteer.launch(browserOptions);
const secondTab = await secondBrowser.newPage();
await secondTab.goto(TARGET_URL, {
    waitUntil: 'domcontentloaded',
    timeout: TIMEOUT_LIMIT,
});

})();

Ensure that both instances of the browser are launched with the exact same path in userDataDir. This consistency will help maintain cookie persistence across sessions. Additionally, you might want to double-check access permissions for the '../../../session_data/user_data' directory to ensure the browser can read and write successfully.

3

Hey FlyingLeaf, it sounds like the issue might be with the userDataDir not being utilized consistently across browser sessions.

Ensure that the browser is launched with the same userDataDir every time for cookie persistence. Here's a slightly modified snippet:

(async () => { const browserOptions = { userDataDir: './session_data/user_data', headless: false, }; 
const firstBrowser = await puppeteer.launch(browserOptions);
const firstTab = await firstBrowser.newPage();
// Set and navigate with cookies...
await firstBrowser.close();

const secondBrowser = await puppeteer.launch(browserOptions);
const secondTab = await secondBrowser.newPage();
await secondTab.goto(TARGET_URL);

})();

This ensures the cookies are saved and loaded from the same directory. Let me know if this helps!

4

FlyingLeaf, I see you’ve already received excellent advice regarding the consistent use of the userDataDir for maintaining session persistence across browser restarts. Let me add a couple of points that could further assist you in addressing the cookie persistence issue.

Firstly, ensure that when you launch a browser session, the userDataDir points to a valid and accessible directory path, enabling Puppeteer to write session data there. Incorrect paths or insufficient write permissions can lead to data, including cookies, not being saved effectively.

Consider the following improvements to enhance cookie management:

(async () => { const browserOptions = { userDataDir: './session_data/user_data', // Must remain consistent headless: false, }; 
// Launch the browser with the user data directory
const browser = await puppeteer.launch(browserOptions);
const page = await browser.newPage();

// Check existing cookies
const cookies = await page.cookies();
console.log(cookies);

// Set cookies if needed
await page.setCookie(...standardizedCookies);

await page.goto(TARGET_URL, {
    waitUntil: 'domcontentloaded',
    timeout: TIMEOUT_LIMIT,
});

// Optionally, close or leave the browser session open depending on your use case
await browser.close();

})();

By using page.cookies(), you can verify what cookies are present in the session and log them for debugging purposes. This helps ensure that the cookies have been set correctly and are readable within the session.

If your application logic permits, consider deploying a more advanced session management approach that combines userDataDir with explicit read and write operations to a secured storage location. This way, you can control exactly which cookies are saved and retrieved across sessions.

Remember to verify directory permissions and paths rigorously, as any misconfigurations could prevent proper data persistence. I hope this provides a fresh perspective and aids in achieving the cookie persistence functionality you're aiming for.