Skip to content
OpenAI

gpt-realtime-whisper

gpt-realtime-whisper is a streaming speech-to-text model for realtime transcription, returning transcript text while audio is still arriving, with a tunable latency and accuracy tradeoff and pricing based on audio duration rather than tokens. Your use is subject to OpenAI's Terms & Privacy Policies.

Websockets
Use with AI GatewayView docs
import { experimental_streamTranscribe as streamTranscribe } from 'ai';
import { createGateway, gateway } from '@ai-sdk/gateway';
import { readFile } from 'node:fs/promises';


const modelId = 'openai/gpt-realtime-whisper';


// Mint this on your server, then send only the short-lived token to the client.
const { token } = await gateway.experimental_transcription.getToken({
  model: modelId,
});
const clientGateway = createGateway({ apiKey: token });


// Raw 24 kHz, 16-bit signed little-endian mono PCM audio.
const bytes = await readFile('audio.pcm');
const audio = new ReadableStream<Uint8Array>({
  start(controller) {
    controller.enqueue(new Uint8Array(bytes));
    controller.close();
  },
});


const result = streamTranscribe({
  model: clientGateway.transcription(modelId),
  audio,
  inputAudioFormat: { type: 'audio/pcm', rate: 24000 },
});


for await (const part of result.fullStream) {
  if (part.type === 'transcript-delta') {
    process.stdout.write(part.delta);
  }
}


console.log('\nFinal:', await result.text);
Read docs
OverviewAboutProvidersSimilarFAQ

Frequently Asked Questions

  • What does gpt-realtime-whisper do?

    gpt-realtime-whisper transcribes live audio into text while someone is still speaking. Transcript text arrives incrementally through a streaming session rather than after the recording ends.

  • How is gpt-realtime-whisper different from Whisper?

    Whisper-generation models transcribe completed audio, which fits recordings and post-session processing. gpt-realtime-whisper is the streaming counterpart, built for live audio where the transcript has to appear during the conversation.

  • How do I access gpt-realtime-whisper through AI Gateway?

    Audio support is in beta through AI SDK 7. Mint a short-lived token on your server with the AI SDK's gateway provider, then stream audio from the client over a WebSocket connection. Your AI Gateway API key never reaches the browser.

  • Can I trade latency for transcript accuracy?

    Yes. A delay setting controls how much audio gpt-realtime-whisper hears before emitting text. Lower delay surfaces partial text sooner, and higher delay gives the model more context and improves transcript quality.

  • How do I improve accuracy on names and product terms?

    Pass vocabulary hints and the languages you expect in the audio. Product names, acronyms, and identifiers come through more reliably when gpt-realtime-whisper knows to expect them.

  • What context window does gpt-realtime-whisper support?

    0 tokens, with up to varies of transcript output per turn. Streaming transcription commits audio in turns, so a long session produces many turns instead of one large response.

  • How is gpt-realtime-whisper priced?

    By audio duration rather than text tokens. Rates are listed on this page and update when providers change list prices, and AI Gateway adds no markup.

  • Does gpt-realtime-whisper support zero data retention through AI Gateway?

    Zero Data Retention is not currently available for this model. Zero Data Retention is offered on a per-provider basis. See https://vercel.com/docs/ai-gateway/capabilities/zdr for details.

  • What are typical latency characteristics?

    This page shows live performance metrics measured across real AI Gateway traffic. Your delay setting also shapes how quickly partial transcript text appears.