Trouble filling dropdown and input fields with Puppeteer for web scraping

Other Questions Puppeteer
1

I’m stuck on a web scraping project using Puppeteer and JavaScript. There’s this website I need to grab info from, but I can’t get the dropdown to work or type in the input field for the ‘Comuna’ (that’s like a town or district).

I’ve tried a bunch of different ways to fill out the form, but nothing’s working. Here’s what I’ve attempted:

// Try 1: Using focus and keyboard
await page.focus('input[data-ng-model="communeName"]');
await page.keyboard.type('SAN', { delay: 50 });

// Try 2: Direct typing
page.type('input[data-ng-model="communeName"]', 'SANTIAGO', { delay: 50 });

// Try 3: JavaScript injection
await page.evaluate(() => {
  let communeInput = document.querySelector('input[data-ng-model="communeName"]');
  communeInput.value = 'VALPARAISO';
  communeInput.dispatchEvent(new Event('input', { bubbles: true }));
});

// Try 4: Using locator
await page.locator('input[data-ng-model="communeName"]').fill(communeName);

The other fields like ‘Block’ and ‘Property’ were easy to fill, but this ‘Comuna’ field is giving me a headache. Any ideas on how to make this work?

2

I’ve run into similar issues with Puppeteer before, especially on sites using Angular (which I suspect this one might be, given the ‘data-ng-model’ attribute). Here’s what worked for me:

Try using page.evaluate() to directly manipulate the Angular model:

await page.evaluate(() => {
    angular.element('input[data-ng-model="communeName"]').scope().communeName = 'SANTIAGO';
    angular.element('input[data-ng-model="communeName"]').scope().$apply();
});

This approach bypasses Puppeteer’s usual input methods and directly sets the Angular model value. Then it triggers a $apply() to update the view.

If that doesn’t work, you might need to wait for the Angular app to fully load before interacting with the form. Try adding a wait before your input attempts:

await page.waitForFunction(() => {
    return window.angular && angular.element(document.body).injector();
});

Hope this helps! Let me know if you need any more details or if you’re still stuck.

3

I’ve encountered similar challenges with Puppeteer, particularly when dealing with dynamic forms. One approach that’s proven effective for me is using page.waitForSelector() before interacting with the element. This ensures the element is fully loaded and ready for input. Here’s an example:

await page.waitForSelector('input[data-ng-model=\"communeName\"]');
await page.type('input[data-ng-model=\"communeName\"]', 'SANTIAGO', {delay: 100});

If that doesn’t work, you might need to trigger events manually:

await page.evaluate(() => {
    const input = document.querySelector('input[data-ng-model=\"communeName\"]');
    input.value = 'SANTIAGO';
    input.dispatchEvent(new Event('input'));
    input.dispatchEvent(new Event('change'));
});

These methods have worked well in my experience with tricky forms. Let me know if you need further assistance.

4

hey ameliat, have u tried using page.waitForNavigation() before interacting with the form? sometimes the page needs time to load fully. also, try clearing the field first:

await page.evaluate(() => document.querySelector(‘input[data-ng-model=“communeName”]’).value = ‘’);

then fill it:

await page.$eval(‘input[data-ng-model=“communeName”]’, (el, value) => el.value = value, ‘SANTIAGO’);

hope this helps!