Automating form submission with headless browsing in Symfony

Other Questions Headless Browser
1

Hey everyone! I’m stuck with a tricky problem. I need to register users through a headless browser because the platform I’m working with doesn’t have an API. Bummer, right?

I’ve got it working with Selenium and PHPUnit in my Symfony project, but I’m not sure if keeping Selenium running all the time is the best way to go.

Here’s what I’ve tried:

public function autoFillForm($data, $pageUrl, $formClass)
{
  $browserProcess = new Process('xvfb-run -a --server-args=\"-screen 0 1280x1024x24\" chromium-browser --headless');
  $browserProcess->start();

  sleep(3);

  if (!$browserProcess->isRunning()) {
    throw new \Exception('Browser process failed to start');
  }

  $driver = new ChromeDriver();
  $driver->get($pageUrl);

  $form = $driver->findElement(WebDriverBy::className($formClass));

  foreach ($data as $field) {
    $element = $form->findElement(WebDriverBy::id($field['id']));

    if ($field['type'] === 'dropdown') {
      $select = new WebDriverSelect($element);
      $select->selectByValue($field['value']);
    } elseif ($field['type'] === 'checkbox') {
      $element->click();
    } else {
      $element->sendKeys($field['value']);
    }
  }

  $form->submit();
  $driver->quit();
  $browserProcess->stop();
}

Any ideas on how to make this work better or more efficiently? I’m all ears!

2

Have you considered using Goutte for this task? It’s a headless browser library that’s lighter than Selenium and integrates well with Symfony. You won’t need to manage a separate browser process, which could simplify your setup.

Here’s a basic example of how you might refactor your code with Goutte:

use Goutte\Client;

public function autoFillForm($data, $pageUrl, $formClass)
{
    $client = new Client();
    $crawler = $client->request('GET', $pageUrl);

    $form = $crawler->filter('.' + $formClass)->form();

    foreach ($data as $field) {
        $form[$field['id']] = $field['value'];
    }

    $client->submit($form);
}

This approach is more efficient and doesn’t require managing external processes. It might not handle JavaScript-heavy pages as well as Selenium, but for most form submissions, it should work fine and be much faster.

3

have u tried using Mink? it’s pretty cool for this kinda stuff. it can use different drivers like goutte or selenium, so u can switch easily. here’s a quick example:

$session = $this->mink->getSession();
$session->visit($pageUrl);
$page = $session->getPage();
$form = $page->find('css', '.' . $formClass);

foreach ($data as $field) {
  $form->fillField($field['id'], $field['value']);
}

$form->submit();

might be worth checkin out!

4

I’ve dealt with a similar situation before, and I can share some insights that might help you out.

Instead of using Selenium, you might want to consider using Symfony’s Panther component. It’s specifically designed for Symfony applications and provides a more seamless integration for headless browsing.

Panther uses ChromeDriver under the hood, but it manages the browser process for you, which simplifies your code quite a bit. It also provides a nice API that’s more aligned with Symfony’s style.

Here’s a rough idea of how you could refactor your code using Panther:

use Symfony\Component\Panther\Client;

public function autoFillForm($data, $pageUrl, $formClass)
{
    $client = Client::createChromeClient();
    $client->request('GET', $pageUrl);

    $form = $client->getCrawler()->filter('.' . $formClass)->form();

    foreach ($data as $field) {
        if ($field['type'] === 'dropdown') {
            $form[$field['id']]->select($field['value']);
        } elseif ($field['type'] === 'checkbox') {
            $form[$field['id']]->tick();
        } else {
            $form[$field['id']] = $field['value'];
        }
    }

    $client->submit($form);
    $client->close();
}

This approach is more efficient as Panther handles browser management, and it integrates better with Symfony’s testing tools. Give it a shot and see if it improves your workflow!