OpenAI Realtime API - Cannot modify transcription settings mid-session

Other Questions OpenAI
1

I’m trying to change the transcription prompt while my OpenAI Realtime session is running but keep getting errors.

Here’s what I’m attempting:

const updateConfig = {
  type: "session.update",
  session: {
    input_audio_format: "pcm16",
    input_audio_transcription: {
      model: "whisper-1",
      prompt: newPromptText,
      language: "en"
    },
    turn_detection: {
      type: "server_vad",
      threshold: 0.6,
      prefix_padding_ms: 200,
      silence_duration_ms: 800
    }
  }
};

But I receive this error response:

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "code": "invalid_parameter",
    "message": "Passing a transcription session update event to a realtime session is not allowed."
  }
}

Is it actually possible to modify transcription parameters during an active WebSocket session? The documentation suggests it should work but I can’t get it functioning. Any ideas what I might be doing wrong?

2

Yes, I understand the frustration. It’s a common issue when working with the Realtime API. After initiating the WebSocket connection, you can’t alter the transcription settings until a new session is created. I recommend managing your conversation state independently so that when you need to make changes, you can terminate the existing session, preserve the context, and initiate a new connection with the updated settings. Although it isn’t the most elegant solution, given the API’s restrictions, this method minimizes disruption for users.

3

Had this exact problem a few weeks ago and wasted way too much time on it. The error’s pretty clear though - you can’t change transcription settings while a session’s running. That’s just how the Realtime API works right now. I had to create a completely new WebSocket connection whenever I needed different transcription settings. Not great, and it makes transitions trickier, but it’s the only thing that actually works. Just make sure you close the current session first, then start fresh with your new config. The docs really should mention this upfront. I spent hours thinking my session.update format was wrong when the API just doesn’t support live parameter changes at all.