GPT-3.5 API response times extremely slow - what's causing the delay?

Other Questions OpenAI
1

I’m building a mobile app with Flutter and integrated the ChatGPT API for my chatbot feature. The responses work fine but I’m getting really frustrated with how long it takes to receive answers. Sometimes it’s taking like 45 seconds or even a full minute just to get a simple response back from the API.

Here’s my current implementation:

Future<String> fetchAIResponse(String userInput) async {
  OpenAI.apiKey = myApiKey;
  try {
    final completion = await OpenAI.instance.chat.create(
      model: 'gpt-3.5-turbo',
      messages: [
        OpenAIChatCompletionChoiceMessageModel(
          content: userInput,
          role: OpenAIChatMessageRole.user,
        ),
      ],
    );
    return completion.choices.first.message.content;
  } catch (error) {
    return "Error occurred, please retry.";
  }
}

I’m currently using the free tier account. Could this be why responses are so sluggish? Would upgrading to a premium plan actually speed things up or is there something wrong with how I’m calling the API?

2

for sure, the free tier can be slow due to rate limits. when i upgraded, it was way better. also, consider setting timeout in your request. it might help get past those laggy moments quicker.

3

I experienced similar delays when working with GPT-3.5 in production. The issue is likely a combination of network latency and API load rather than just the free tier limitations. Try adding a max_tokens parameter to your request - something like 150 tokens for shorter responses can significantly reduce processing time. Also worth checking your internet connection stability since mobile networks can introduce additional delays. In my experience, response times vary greatly throughout the day depending on OpenAI’s server load. The upgrade to paid tiers does help with priority queuing, but won’t eliminate all delays if the underlying network or prompt complexity is the bottleneck.