How to execute JavaScript code in Python without using headless browsers?

Other Questions Headless Browser
1

I need to scrape information from a web page that contains interactive charts built with a popular charting library. The data I want is only accessible through JavaScript execution since it’s dynamically generated.

Currently I can extract the chart data using browser automation tools like this:

result = browser.execute_script('''
chartData = [];
Highcharts.charts[0].series[0].data.forEach((point) => chartData.push(point.y));
return chartData;
''')

However, using full browser automation seems like too much overhead for this simple task. I tried using requests_html for JavaScript support but ran into issues:

response = my_session.html.render(script='''
chartData = [];
Highcharts.charts[0].series[0].data.forEach((point) => chartData.push(point.y));
return chartData;
''')

This throws an error saying the session object doesn’t have an html attribute. Are there any lightweight alternatives to execute JavaScript and retrieve data without launching a full browser instance?

2

I’ve dealt with this exact problem scraping JS-heavy sites. Here’s what actually works: First, try intercepting the network requests that feed those charts. Most charting libraries pull data through AJAX calls - you can catch these with mitmproxy or just check the Network tab in dev tools. Once you find the API endpoints, hit them directly for the raw JSON. No JavaScript needed. If that doesn’t work, use Playwright in headless mode but block unnecessary resources. Disable images, CSS, and other junk while keeping JS execution. Way less overhead than full browser automation. When you absolutely need JS execution with DOM access, go with Pyppeteer and minimal Chrome flags. It’s lighter than most alternatives. The trick is balancing what you need vs. resource usage for your specific situation.

3

PyMiniRacer is your best bet here. It’s a lightweight Python wrapper around Google’s V8 JavaScript engine - lets you run JS code without spinning up a full browser. I’ve used it for similar data extraction tasks where I needed to process dynamically generated content. Main limitation: PyMiniRacer runs isolated, so no access to the actual DOM or libraries like Highcharts that load on the webpage. You’d need to extract raw data from the page source first, then manipulate it with JavaScript if needed. Another approach that works well: Node.js subprocess calls from Python. Write a small Node script that handles the JavaScript execution and returns results to your Python app. Still lightweight compared to browser automation, but gives you access to the full JavaScript ecosystem.

4

You’re overcomplicating this. First, check if the chart data comes from an API endpoint - most sites load chart data through XHR requests you can hit directly. Open browser dev tools, go to the network tab, and see what calls happen when the page loads. If that doesn’t work, try the execjs library. It runs JavaScript through different engines without browser overhead. Worked for me when I needed to process JSON transforms that were originally in JS.