Why does my Python request time out while the curl command works fine for the same API?

Other Questions API
1

I’m encountering a timeout issue when trying to make an API call using Python’s requests library. The CURL command I use runs without any issues in the terminal and successfully returns the JSON response.

Here’s how the successful curl command looks:

curl -X GET "https://my.domain.com/api/product/?name=product" \
    -H "Accept: application/json" \
    -H "Content-Type: application/json" \
    -H "x-api-key: 02987eafdeef73450982734"

Unfortunately, when I replace it with the equivalent request in Python, I get a timeout error:

base_url = 'https://my.domain.com'
timeout_duration = 10
request_headers = {
    'content-type': 'application/json',
    'Accept': 'application/json',
    'Content-Type': 'application/json',
    'x-api-key': '02987eafdeef73450982734'
}

response = requests.get(
    f'{base_url}/api/product/?name=product',
    headers=request_headers,
    timeout=timeout_duration
)

Can you help me figure out why the curl command succeeds while the Python version fails? Is there something I might be overlooking?

2

I’ve encountered similar issues before; curl and requests indeed manage connections quite differently. Curl tends to be more aggressive with its connections by default.

Consider using a session object—this allows your requests to reuse connections more effectively and handle keep-alive requests better. Additionally, the 10-second timeout may be too short when compared to curl’s default behavior, which is typically less strict.

Don’t overlook SSL verification either; curl is often more tolerant of certificate issues than requests. You could test your code with verify=False temporarily, but be cautious not to use this in production. Lastly, check for any proxy settings or firewall configurations that might affect how Python and curl interact with your network; I’ve seen discrepancies in how corporate networks handle these two tools.

3

It’s usually about connection pooling. Requests handles connections differently than curl. I hit this same issue last month - API worked fine with curl but kept timing out in Python. Requests was creating new connections every time while curl reused them more efficiently. Try using requests.adapters.HTTPAdapter with retry strategies, or just bump up your timeout way higher. What took curl 3 seconds sometimes needed 25+ seconds in requests. Also check if Python’s using proxy settings that curl ignores.

4

bumping the timeout to 30 secs might help—10 is kinda tight if the server is slow. also, curl and requests treat user-agents differently, so adding a custom user-agent header could do the trick. some APIs get picky about that stuff.