Can I use headless browser automation with locust for load testing?

I’m working on performance testing for a web application that heavily relies on JavaScript. The problem is that when pages load, there’s client-side code that makes extra HTTP requests after the initial page render. These additional calls are crucial for the complete user experience.

I’ve been using locust for my load testing, but it doesn’t execute JavaScript by default. This means I’m missing all those secondary requests that happen through client-side scripts. I’m wondering if there’s a way to combine something like Selenium with headless Chrome or Firefox alongside locust to capture the full request pattern.

Has anyone successfully set up this kind of hybrid approach? I need to simulate real user behavior including all the AJAX calls and dynamic content loading that happens after the page initially loads.

yeah you can do it but honestly its gonna be painfull. tried using playwright with locust last year and the setup was messy. you basically need to wrap the browser calls in your locust tasks but expect everything to run way slower. maybe consider using k6 browser module instead? it handles js execution better than trying to frankenstein selenium into locust.

There’s another approach worth considering if you want to maintain higher concurrency while still capturing those JavaScript-generated requests. Instead of running headless browsers during the actual load test, you can use them in a preprocessing phase to record the complete request patterns. Set up a headless Chrome session to navigate through your application flows while capturing all network traffic using the Chrome DevTools Protocol or a proxy like mitmproxy. Export these recorded request sequences and then recreate them as standard HTTP requests in your locust tasks. This way you get the benefit of understanding the full JavaScript behavior without the performance overhead during testing. The downside is that you’ll need to update your request patterns whenever the frontend logic changes, and you might miss some dynamic elements that depend on real-time data. But for most scenarios, this hybrid recording approach gives you much better scalability than running browsers in real-time while still accounting for the AJAX calls and dynamic loading patterns.

I’ve implemented exactly this setup in my previous project and it works well with some caveats. You can integrate Selenium WebDriver with headless Chrome into your locust tasks, but you need to be careful about resource management since each browser instance consumes significant memory and CPU compared to simple HTTP requests. The approach I used was creating a custom locust user class that initializes a headless Chrome driver in the on_start method and closes it properly in on_stop. Inside the tasks, I’d navigate to pages and wait for all network activity to complete using WebDriverWait with custom conditions. This captures all the AJAX calls and dynamic loading you mentioned. However, expect much lower concurrent user counts than traditional HTTP-only tests. Where I could simulate thousands of users with regular locust, the headless browser approach maxed out around 50-100 concurrent users per machine depending on the complexity of the pages. You’ll also need to handle browser crashes gracefully and implement proper cleanup to avoid memory leaks during long test runs.