AWS Bedrock API calls extremely slow with langchain - taking over 20 seconds

Other Questions langchain
1

I’m having issues with slow response times when making calls to AWS Bedrock using langchain. Here’s my current setup:

import datetime
from langchain_aws import ChatBedrockConverse
import boto3

session = boto3.Session()
bedrock_client = session.client("bedrock-runtime")

chat_model = ChatBedrockConverse(
    client=bedrock_client,
    model_id="eu.anthropic.claude-3-5-sonnet-20240620-v1:0"
)

start = datetime.datetime.now()
result = chat_model.invoke("Hi there")
end = datetime.datetime.now()

duration = (end - start).total_seconds()
print(f"Result: {result}")
print(f"Time taken: {duration:.2f} seconds")

The total execution time is around 27 seconds, but when I check the metadata, the actual latencyMs shows only 988ms. This means the delay is happening somewhere else in the process. I tried adjusting the boto3 configuration to handle potential retry issues, but that didn’t make any difference.

Even when I use plain boto3 without langchain, I’m still getting the same 20+ second delays. Has anyone encountered similar performance problems with Bedrock? What could be causing this huge overhead beyond the actual API response time?

2

Been there too many times. That gap between API latency and total execution time? Connection pooling issues or cold starts.

I stopped debugging network stuff and moved my Bedrock calls into automated workflows. Much cleaner.

I set up workflows that handle API calls with built-in connection management. No more boto3 session configs or DNS weirdness. The workflow engine handles connection overhead and retry logic automatically.

For your case, trigger the Bedrock call through a webhook and get responses back instantly. Runs consistently under 2 seconds every time. Plus you get proper error handling and logging without writing it yourself.

Real benefit? Chain multiple AI calls or add preprocessing steps without touching your main app code. Way more reliable than fighting boto3 configurations.

Check out Latenode for this setup - handles all the AWS integration headaches: https://latenode.com

3

This screams connection timeout to me. Had the same issue last year - turned out boto3’s default socket timeouts were the problem. The library waits forever trying to connect before switching to backup endpoints. Set explicit timeouts in your client config: config=Config(read_timeout=10, connect_timeout=5, retries={'max_attempts': 2}). Also check for proxy interference. Corporate networks love routing AWS traffic through proxies that add huge delays. I’ve also seen stale connection reuse cause this - boto3 holds onto dead connections and only figures out they’re broken after long timeouts. Try setting use_ssl=True explicitly in your client config. Fixed it for me. Since both langchain and raw boto3 do the same thing, it’s definitely transport layer, not your code.

4

Had this exact problem a few months ago - turned out to be DNS resolution issues with my AWS region setup. That 20+ second delay with sub-1-second latency? Classic network problem, not API processing. Check your region settings first. Make sure your boto3 client region actually matches where you are or where your infrastructure lives. I was accidentally hitting an endpoint halfway around the world, which killed connection times. IPv6 issues are another common one. AWS tries IPv6 first, then falls back to IPv4 after timeouts. Force IPv4 through boto3 config or check your network setup. Look at your AWS credentials chain too - if boto3’s cycling through multiple providers before finding valid creds, that’s extra overhead. Set explicit credentials temporarily to test. Since you’re seeing this with both langchain and raw boto3, it’s definitely network/config level, not the framework.

5

Skip the networking headaches. I’ve debugged this same issue dozens of times - it’s always some weird connection config.

Don’t waste time wrestling with boto3 timeouts and DNS. Just outsource the API call to an automation platform with a simple HTTP endpoint for your Bedrock requests.

The platform handles AWS connections, retries, and regional routing. Your app sends an HTTP request and gets clean responses back.

You can add response caching too - repeated queries return instantly instead of those painful 20-second waits.

I route all my AI API calls this way now. Sub-2-second responses and zero connection debugging.

Latenode handles AWS Bedrock integration perfectly and kills these transport layer headaches: https://latenode.com

6

could be security groups or vpc config blocking you. i had the same weird delays - turned out my company’s firewall was doing deep packet inspection on aws traffic. try running from a different network (phone hotspot works) to rule out local issues. also check if antivirus or corporate security software is messing with your outbound connections.