How can I retrieve information from a JSON file in JavaScript using Puppeteer?

FlyingStar · December 19, 2024, 5:09pm

Query:

What is the method to extract data like usernames and passwords from a JSON file in a Puppeteer script?

Here’s a sample of my code:

const { errors } = require('puppeteer');
const puppeteer = require('puppeteer-extra');

(async () => {
    const browserInstance = await puppeteer.launch({
      headless: false,
      ignoreHTTPSErrors: true,
      slowMo: 3,
      executablePath: "C:\Program Files\Google\Chrome\Application\chrome.exe"
    });

    const newPage = await browserInstance.newPage();
    await installPointerHelper(newPage); // Makes cursor visible
    await newPage.setDefaultNavigationTimeout(0);
    await newPage.evaluate(() => { debugger; });
    await newPage.goto('https://example.com/login');

    await newPage.waitFor(500);
    const emailField = '[data-identifier="emailAddress"]';
    const emailButton = await newPage.waitForSelector(emailField);
    await emailButton.click();
    await emailButton.type("[email protected]");

    await newPage.waitFor(500);
    const passwordField = '[data-identifier="password"]';
    const passwordButton = await newPage.waitForSelector(passwordField);
    await passwordButton.click();
    await passwordButton.type("Password123.");

    await newPage.waitFor(500);
    const submitButton = '[class="submit-button login"]';
    const submitBtn = await newPage.waitForSelector(submitButton);
    await submitBtn.click();

})();

Here’s how my JSON file is structured:

{
  "appUsers": {
    "credentials": [
      {
        "note": "",
        "details": [
          {
            "username": "[email protected]",
            "credentialsPass": "Password123.",
            "token": "",
            "refreshToken": "",
            "expiry": "",
            "type": "",
            "countryCode": "",
            "contact": "",
            "validUntil": "???",
            "proxyDetails": ""
          }
        ]
      }
    ]
  }
}

ClimbingLion · December 24, 2024, 2:06pm

To retrieve data from a JSON file in your Puppeteer script, you should first read the file using Node.js's fs module and parse the JSON content. Here's how you can modify your script:

const puppeteer = require('puppeteer-extra');
const fs = require('fs');

(async () => {
    const rawData = fs.readFileSync('path-to-your-json-file.json');
    const jsonData = JSON.parse(rawData);

    const browserInstance = await puppeteer.launch({
        headless: false,
        ignoreHTTPSErrors: true,
        slowMo: 3,
        executablePath: 'C:\Program Files\Google\Chrome\Application\chrome.exe'
    });

    const newPage = await browserInstance.newPage();
    await newPage.goto('https://example.com/login');

    const credentials = jsonData.appUsers.credentials[0].details[0];

    await newPage.type('[data-identifier="emailAddress"]', credentials.username);
    await newPage.type('[data-identifier="password"]', credentials.credentialsPass);
    await newPage.click('[class="submit-button login"]');

    await browserInstance.close();
})();

Replace 'path-to-your-json-file.json' with the actual path to your JSON file.

CreativePainter33 · December 25, 2024, 6:52pm

Building on ClimbingLion’s approach, here’s a slightly more structured way to handle reading from a JSON file and using its contents in a Puppeteer script:

First, ensure you have the fs module available in your script to read the JSON file, similar to what ClimbingLion suggested.

If your JSON structure is subject to change or may contain multiple user credentials, you may want to wrap your logic in a function for flexibility. Here’s an updated version of your script:

const puppeteer = require('puppeteer-extra');
const fs = require('fs');

(async () => {
    function getCredentials(path) {
        const rawData = fs.readFileSync(path);
        const jsonData = JSON.parse(rawData);
        // Adjust index [0] if you want to loop over multiple users
        return jsonData.appUsers.credentials[0].details[0];
    }
    
    const credentials = getCredentials('path-to-your-json-file.json');

    const browserInstance = await puppeteer.launch({
        headless: false,
        ignoreHTTPSErrors: true,
        slowMo: 3,
        executablePath: 'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe'
    });

    const newPage = await browserInstance.newPage();
    await newPage.goto('https://example.com/login');

    await newPage.type('[data-identifier="emailAddress"]', credentials.username);
    await newPage.type('[data-identifier="password"]', credentials.credentialsPass);
    await newPage.click('[class="submit-button login"]');

    await browserInstance.close();
})();

Notice the getCredentials function, which provides an abstraction to easily change or loop through multiple credentials if needed.
Ensure that the path in 'path-to-your-json-file.json' points to the correct JSON file. Also, mind the double backslashes in the file path on Windows.

This way, your script can easily adapt to future changes in the JSON data structure or add functionalities to handle multiple users.

HappyDancer99 · December 28, 2024, 9:24pm

To retrieve data from a JSON file in your Puppeteer script and use it for automating tasks, you can incorporate the fs module to read and parse your JSON file efficiently. Here’s how you can streamline your Puppeteer script setup:

const puppeteer = require('puppeteer-extra');
const fs = require('fs');

(async () => {
    // Function to get credentials from JSON file
    function getCredentials(filePath) {
        const rawData = fs.readFileSync(filePath);
        const jsonData = JSON.parse(rawData);
        // Fetch user details; adjust index to navigate different users
        return jsonData.appUsers.credentials[0].details[0];
    }

    const credentials = getCredentials('path-to-your-json-file.json');

    const browserInstance = await puppeteer.launch({
        headless: false,
        ignoreHTTPSErrors: true,
        slowMo: 3,
        executablePath: 'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe'
    });

    const newPage = await browserInstance.newPage();
    await newPage.goto('https://example.com/login');

    // Use credentials from JSON to automate login
    await newPage.type('[data-identifier="emailAddress"]', credentials.username);
    await newPage.type('[data-identifier="password"]', credentials.credentialsPass);
    await newPage.click('[class="submit-button login"]');

    await browserInstance.close();
})();

Ensure to replace 'path-to-your-json-file.json' with your actual file path and maintain the double backslashes in paths for Windows systems. This approach enhances your script's adaptability and efficiency, allowing future adjustments to the JSON structure or processing multiple users easily.

Alex_Brave · December 25, 2024, 9:24pm

To extract data from a JSON file in your Puppeteer script, use Node's fs module. Update your script like this:

const puppeteer = require('puppeteer-extra');
const fs = require('fs');

(async () => {
    const rawData = fs.readFileSync('path-to-your-json-file.json');
    const jsonData = JSON.parse(rawData);
    const credentials = jsonData.appUsers.credentials[0].details[0];

    const browserInstance = await puppeteer.launch({
        headless: false,
        ignoreHTTPSErrors: true,
        slowMo: 3,
        executablePath: 'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe'
    });

    const newPage = await browserInstance.newPage();
    await newPage.goto('https://example.com/login');

    await newPage.type('[data-identifier="emailAddress"]', credentials.username);
    await newPage.type('[data-identifier="password"]', credentials.credentialsPass);
    await newPage.click('[class="submit-button login"]');

    await browserInstance.close();
})();

Replace 'path-to-your-json-file.json' with your file's actual path, and ensure to use double backslashes (\\) for paths in Windows.

Emma_Fluffy · December 27, 2024, 12:45pm

In addition to the solutions provided, you might want to consider implementing environment-specific configurations for improved flexibility and security in handling sensitive data like usernames and passwords.

Here's an alternative approach using environment variables, which is particularly useful for scenarios where multiple environments (development, testing, production) are used:

First, store your JSON file path and any sensitive data in a .env file. This ensures you're not hardcoding sensitive information directly within your script:
```
<pre><code>
```
.env file

JSON_FILE_PATH=path-to-your-json-file.json

Use the dotenv package to load these environment variables into your Node.js application:

<pre><code>npm install dotenv</code></pre>

<pre><code>const puppeteer = require('puppeteer-extra');

const fs = require(‘fs’);
require(‘dotenv’).config();

(async () => {
// Load credentials from JSON
const rawData = fs.readFileSync(process.env.JSON_FILE_PATH);
const jsonData = JSON.parse(rawData);
const credentials = jsonData.appUsers.credentials[0].details[0];

const browserInstance = await puppeteer.launch({
    headless: false,
    ignoreHTTPSErrors: true,
    slowMo: 3,
    executablePath: 'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe'
});

const newPage = await browserInstance.newPage();
await newPage.goto('https://example.com/login');

await newPage.type('[data-identifier="emailAddress"]', credentials.username);
await newPage.type('[data-identifier="password"]', credentials.credentialsPass);
await newPage.click('[class="submit-button login"]');

await browserInstance.close();

})();

This setup not only organizes your secret data but also facilitates easier management of environment-specific settings by simply changing environment variables.

Remember to include your .env file in your .gitignore to prevent sensitive information from being uploaded to version control.

How can I retrieve information from a JSON file in JavaScript using Puppeteer?

Query:

.env file