ElevenLabs

The ElevenLabs provider contains language model support for the ElevenLabs transcription and speech generation APIs. ElevenLabs models are not currently available through AI Gateway, so the opening example uses @ai-sdk/elevenlabs directly. See Using AI Gateway below for supported alternatives.

import { generateSpeech } from 'ai';
import { elevenLabs } from '@ai-sdk/elevenlabs';
import { writeFile } from 'node:fs/promises';

const { audio } = await generateSpeech({
  model: elevenLabs.speech('eleven_multilingual_v2'),
  text: 'Hello, world!',
  outputFormat: 'mp3',
  voice: '21m00Tcm4TlvDq8ikWAM',
});

await writeFile('speech.mp3', audio.uint8Array);

Using AI Gateway

The current AI Gateway model catalog does not list ElevenLabs models. Use @ai-sdk/elevenlabs for the APIs on this page. To use other speech models through Gateway, install ai and configure a Gateway API key:

pnpm add ai

npm install ai

yarn add ai

bun add ai

Set an AI Gateway API key in your environment:

export AI_GATEWAY_API_KEY=your_api_key_here

Vercel deployments can also use OIDC authentication. See the Gateway documentation for supported models and APIs.

Using @ai-sdk/elevenlabs

For direct access to ElevenLabs, install ai and @ai-sdk/elevenlabs:

pnpm add ai @ai-sdk/elevenlabs

npm install ai @ai-sdk/elevenlabs

yarn add ai @ai-sdk/elevenlabs

bun add ai @ai-sdk/elevenlabs

Set your ElevenLabs API credential in the ELEVENLABS_API_KEY environment variable:

export ELEVENLABS_API_KEY=your_api_key_here

Provider Instance

You can import the default provider instance elevenLabs from @ai-sdk/elevenlabs:

import { elevenLabs } from '@ai-sdk/elevenlabs';

If you need a customized setup, you can import createElevenLabs from @ai-sdk/elevenlabs and create a provider instance with your settings:

import { createElevenLabs } from '@ai-sdk/elevenlabs';

const elevenLabs = createElevenLabs({
  // custom settings, e.g.
  fetch: customFetch,
});

You can use the following optional settings to customize the ElevenLabs provider instance:

• apiKey string

  API key that is being sent using the xi-api-key header. It defaults to the ELEVENLABS_API_KEY environment variable.

• headers Record<string,string>

  Custom headers to include in the requests.

• fetch (input: RequestInfo, init?: RequestInit) => Promise<Response>

  Custom fetch implementation. Defaults to the global fetch function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing.

• webSocket WebSocketConstructor

  Custom WebSocket implementation for realtime transcription. A header-capable implementation is required in runtimes whose native WebSocket constructor cannot send the xi-api-key authentication header.

Speech Models

You can create models that call the ElevenLabs speech API using the .speech() factory method.

The first argument is the model id e.g. eleven_multilingual_v2.

const model = elevenLabs.speech('eleven_multilingual_v2');

The voice argument can be set to a voice ID from the ElevenLabs Voice Library. You can find voice IDs by selecting a voice in the library and copying its ID.

import { generateSpeech } from 'ai';
import { elevenLabs } from '@ai-sdk/elevenlabs';

const result = await generateSpeech({
  model: elevenLabs.speech('eleven_multilingual_v2'),
  text: 'Hello, world!',
  voice: '21m00Tcm4TlvDq8ikWAM', // Rachel voice
});

You can also pass additional provider-specific options using the providerOptions argument:

import { generateSpeech } from 'ai';
import {
  elevenLabs,
  type ElevenLabsSpeechModelOptions,
} from '@ai-sdk/elevenlabs';

const result = await generateSpeech({
  model: elevenLabs.speech('eleven_multilingual_v2'),
  text: 'Hello, world!',
  voice: '21m00Tcm4TlvDq8ikWAM',
  providerOptions: {
    elevenlabs: {
      voiceSettings: {
        stability: 0.5,
        similarityBoost: 0.75,
      },
    } satisfies ElevenLabsSpeechModelOptions,
  },
});

• languageCode string or null
  Optional. Language code (ISO 639-1) used to enforce a language for the model. Currently, only Turbo v2.5 and Flash v2.5 support language enforcement. For other models, providing a language code will result in an error.

• voiceSettings object or null
  Optional. Voice settings that override stored settings for the given voice. These are applied only to the current request.

  • stability double or null
    Optional. Determines how stable the voice is and the randomness between each generation. Lower values introduce broader emotional range; higher values result in a more monotonous voice.
  • useSpeakerBoost boolean or null
    Optional. Boosts similarity to the original speaker. Increases computational load and latency.
  • similarityBoost double or null
    Optional. Controls how closely the AI should adhere to the original voice.
  • style double or null
    Optional. Amplifies the style of the original speaker. May increase latency if set above 0.

• pronunciationDictionaryLocators array of objects or null
  Optional. A list of pronunciation dictionary locators to apply to the text, in order. Up to 3 locators per request.
  Each locator object:

  • pronunciationDictionaryId string (required)
    The ID of the pronunciation dictionary.
  • versionId string or null (optional)
    The version ID of the dictionary. If not provided, the latest version is used.

• seed integer or null
  Optional. If specified, the system will attempt to sample deterministically. Must be between 0 and 4294967295. Determinism is not guaranteed.

• previousText string or null
  Optional. The text that came before the current request's text. Can improve continuity when concatenating generations or influence current generation continuity.

• nextText string or null
  Optional. The text that comes after the current request's text. Can improve continuity when concatenating generations or influence current generation continuity.

• previousRequestIds array of strings or null
  Optional. List of request IDs for samples generated before this one. Improves continuity when splitting large tasks. Max 3 IDs. If both previousText and previousRequestIds are sent, previousText is ignored.

• nextRequestIds array of strings or null
  Optional. List of request IDs for samples generated after this one. Useful for maintaining continuity when regenerating a sample. Max 3 IDs. If both nextText and nextRequestIds are sent, nextText is ignored.

• applyTextNormalization enum
  Optional. Controls text normalization.
  Allowed values: 'auto' (default), 'on', 'off'.

  • 'auto': System decides whether to apply normalization (e.g., spelling out numbers).
  • 'on': Always apply normalization.
  • 'off': Never apply normalization.
    For eleven_turbo_v2_5 and eleven_flash_v2_5, can only be enabled with Enterprise plans.

• applyLanguageTextNormalization boolean
  Optional. Defaults to false. Controls language text normalization, which helps with proper pronunciation in some supported languages (currently only Japanese). May significantly increase latency.

• enableLogging boolean
  Optional. Whether to enable request logging for this API call. Defaults to the account-level setting.

Model Capabilities

ElevenLabs speech models do not support instructions; the option is ignored with a warning.

The speech model reports the ElevenLabs output_format it requested in providerMetadata.elevenlabs.outputFormat.

Dialogue

Set providerOptions.elevenlabs.inputs to generate multi-speaker dialogue with the Text-to-Dialogue API and an eleven_v3 or eleven_v4 model. Each input pairs a line of text with the voice that speaks it. The inputs replace text and voice: pass an empty text; a non-empty text or a voice is ignored with a warning.

const result = await generateSpeech({
  model: elevenLabs.speech('eleven_v3'),
  text: '',
  providerOptions: {
    elevenlabs: {
      inputs: [
        { text: '[giggling] Knock knock', voiceId: alice },
        { text: '[curious] Who is there?', voiceId: bob },
      ],
      voiceSettings: { stability: 0.5, similarityBoost: 0.75 },
    } satisfies ElevenLabsSpeechModelOptions,
  },
});

voiceSettings.stability and voiceSettings.similarityBoost map to the dialogue settings; nextText is sent as future_text. speed, voiceSettings.style, voiceSettings.useSpeakerBoost, and applyLanguageTextNormalization are ignored with a warning.

Music

The music_* models compose music with the Music API. text is the music prompt; voice, speed, language, and the text-to-speech options are ignored with a warning. The Music API requires a paid ElevenLabs plan.

const result = await generateSpeech({
  model: elevenLabs.speech('music_v2_5'),
  text: 'An upbeat synthwave intro with driving bass',
  providerOptions: {
    elevenlabs: {
      musicLengthMs: 10000,
      forceInstrumental: true,
    } satisfies ElevenLabsSpeechModelOptions,
  },
});

• musicLengthMs integer
  Optional. Length of the song in milliseconds, between 3000 and 600000. When omitted, the model picks a length from the prompt.

• forceInstrumental boolean
  Optional. Guarantees an instrumental song.

• compositionPlan object
  Optional. A native ElevenLabs composition_plan (snake_case), validated and sent in place of the prompt: sections for music_v1, chunks (generation and audio-reference chunks) for music_v2 and later. It cannot be combined with musicLengthMs or forceInstrumental. seed is only accepted with a composition plan.

When the length is known, from musicLengthMs or the plan's sections or chunks, it is reported in providerMetadata.elevenlabs.requestedDurationMs.

Sound effects

The eleven_text_to_sound_* models generate sound effects with the Sound Effects API. text describes the sound; voice, speed, language, seed, and the text-to-speech options are ignored with a warning.

const result = await generateSpeech({
  model: elevenLabs.speech('eleven_text_to_sound_v2'),
  text: 'Glass shattering on a concrete floor',
  providerOptions: {
    elevenlabs: {
      durationSeconds: 2,
      promptInfluence: 0.5,
    } satisfies ElevenLabsSpeechModelOptions,
  },
});

• durationSeconds number
  Optional. Length of the sound between 0.5 and 30 seconds. When omitted, the model picks a length from the prompt.

• promptInfluence number
  Optional. How closely the sound follows the prompt, from 0 to 1. Defaults to 0.3.

• loop boolean
  Optional. Creates a sound that loops smoothly.

Music and sound effects models do not stream; experimental_streamSpeech throws an UnsupportedFunctionalityError. Streaming speech ignores the inputs, music, and sound effects options with a warning.

Streaming Speech Models

experimental_streamSpeech uses the ElevenLabs text-to-speech WebSocket (stream-input). Text deltas are forwarded as they arrive, so you can pipe an LLM text stream straight into speech, and audio chunks stream back as they are generated.

import { elevenLabs } from '@ai-sdk/elevenlabs';
import { experimental_streamSpeech as streamSpeech, streamText } from 'ai';

const { textStream } = streamText({
  model: 'openai/gpt-5.4-nano',
  prompt: 'In three sentences, explain why the sky is blue.',
});

const result = streamSpeech({
  model: elevenLabs.speech('eleven_flash_v2_5'),
  text: textStream, // a string, or an array or stream of strings and/or text parts
  voice: '21m00Tcm4TlvDq8ikWAM',
  outputFormat: { type: 'audio/pcm', rate: 24000 },
  providerOptions: {
    elevenlabs: {
      streaming: {
        chunkLengthSchedule: [50, 120, 160],
        syncAlignment: true,
      },
    },
  },
});

for await (const part of result.fullStream) {
  if (part.type === 'audio') {
    // part.audio: Uint8Array, part.audioFormat: { type: 'audio/pcm', rate: 24000 }
  }
}

console.log(await result.usage); // { inputCharacters: ... }

The API key is sent in the WebSocket's first message, so the native WebSocket works in every runtime. Run streaming speech on the server: in browser code the API key would be exposed to the client.

outputFormat maps to an ElevenLabs output_format: audio/mpeg (44.1 kHz, or 22.05 kHz), audio/pcm (8, 16, 22.05, 24 (default), 44.1, or 48 kHz), audio/pcmu and audio/pcma (8 kHz), and audio/opus or audio/ogg (48 kHz Opus, delivered in an Ogg container and reported as audio/ogg). Other formats are rejected. Set providerOptions.elevenlabs.streaming.outputFormat to a native ElevenLabs format (for example mp3_44100_192) to choose one directly. Without either, audio is mp3_44100_128.

The text-to-speech WebSocket speaks one voice per stream. A text part whose voice differs from the call's voice errors the stream; use dialogue (below) for several voices.

Text deltas that end mid-word (common in LLM streams) are held back until the next whitespace before they are sent, and the rest is flushed when the text stream ends. CJK sentence punctuation also counts as a boundary, and held text is sent after 100 characters regardless. While the text stream is idle, the provider sends keepalive messages so the connection outlives the server's default inactivity timeout (for example while a reasoning model thinks). Setting inactivityTimeout turns keepalives off, so the timeout you set applies.

usage.inputCharacters is the number of Unicode code points sent for synthesis. Audio parts carry ElevenLabs alignment and normalizedAlignment data in providerMetadata.elevenlabs when the server returns it.

The streaming provider options mirror the WebSocket settings: autoMode, chunkLengthSchedule, enableSsmlParsing, inactivityTimeout (seconds, up to 180), outputFormat, and syncAlignment. generateSpeech ignores them with a warning. voiceSettings, pronunciationDictionaryLocators, seed, applyTextNormalization, languageCode, and enableLogging apply as for generateSpeech. previousText, nextText, previousRequestIds, nextRequestIds, and applyLanguageTextNormalization are not supported over the WebSocket and produce warnings.

See the speech model capabilities table above for which models stream. The eleven_v3 and eleven_v4 model families are not served by the text-to-speech WebSocket; stream them as dialogue (below).

Streaming dialogue

Set providerOptions.elevenlabs.dialogue to stream multi-speaker dialogue over the Text-to-Dialogue WebSocket with an eleven_v3 or eleven_v4 model. List the voices in dialogue.voices (the socket needs them up front), then give each text part its speaker with voice. Parts without a voice use the call's voice, or else the first dialogue voice; a voice that is not listed errors the stream. A change of voice starts a new turn. To start a new turn with the same voice, set providerOptions: { elevenlabs: { newTurn: true } } on the part.

const result = streamSpeech({
  model: elevenLabs.speech('eleven_v4'),
  // An array, or a stream of parts arriving from an LLM.
  text: [
    {
      type: 'text',
      text: 'Did you hear we can stream dialogue now? ',
      voice: alice,
    },
    { type: 'text', text: 'Over one socket? That is neat.', voice: bob },
  ],
  providerOptions: {
    elevenlabs: {
      dialogue: { voices: [alice, bob], voiceSettings: { stability: 0.5 } },
    },
  },
});

Text is held back to word boundaries within each speaker's text and flushed when the speaker changes. Text-to-Dialogue ignores speed, top-level voiceSettings (use dialogue.voiceSettings.stability), and the autoMode, chunkLengthSchedule, enableSsmlParsing, and inactivityTimeout streaming options; each produces a warning. eleven_v4 models receive the text incrementally. eleven_v3 models accept a single message per dialogue session, so the provider reads the whole text stream before sending it. eleven_v3_conversational supports a single voice. Text-to-Dialogue has a fixed inactivity timeout, so keepalives are always sent while the text stream is idle. Audio parts carry alignment (camelCase) and isFinalAudioForTurn in providerMetadata.elevenlabs when the server returns them; a turn boundary without audio arrives as an empty audio part carrying only isFinalAudioForTurn.

Transcription Models

You can create models that call the ElevenLabs transcription API using the .transcription() factory method.

The first argument is the model id e.g. scribe_v2.

const model = elevenLabs.transcription('scribe_v2');

You can also pass additional provider-specific options using the providerOptions argument. For example, supplying the input language in ISO-639-1 (e.g. en) format can sometimes improve transcription performance if known beforehand.

import { transcribe } from 'ai';
import {
  elevenLabs,
  type ElevenLabsTranscriptionModelOptions,
} from '@ai-sdk/elevenlabs';

const result = await transcribe({
  model: elevenLabs.transcription('scribe_v2'),
  audio: new Uint8Array([1, 2, 3, 4]),
  providerOptions: {
    elevenlabs: {
      languageCode: 'en',
    } satisfies ElevenLabsTranscriptionModelOptions,
  },
});

The following provider options are available:

• languageCode string

  An ISO-639-1 or ISO-639-3 language code corresponding to the language of the audio file. Can sometimes improve transcription performance if known beforehand. Defaults to null, in which case the language is predicted automatically.

• tagAudioEvents boolean

  Whether to tag audio events like (laughter), (footsteps), etc. in the transcription. Defaults to true.

• numSpeakers integer

  The maximum amount of speakers talking in the uploaded file. Can help with predicting who speaks when. The maximum amount of speakers that can be predicted is 32. Defaults to null, in which case the amount of speakers is set to the maximum value the model supports.

• timestampsGranularity enum

  The granularity of the timestamps in the transcription. Defaults to 'word'. Allowed values: 'none', 'word', 'character'.

• diarize boolean

  Whether to annotate which speaker is currently talking in the uploaded file. Defaults to true.

• fileFormat enum

  The format of input audio. Defaults to 'other'. Allowed values: 'pcm_s16le_16', 'other'. For 'pcm_s16le_16', the input audio must be 16-bit PCM at a 16kHz sample rate, single channel (mono), and little-endian byte order. Latency will be lower than with passing an encoded waveform.

Model Capabilities

Streaming Transcription Models

Scribe v2 Realtime supports live transcription through experimental_streamTranscribe. It accepts raw PCM audio at 8, 16, 22.05, 24, 44.1, or 48 kHz, or 8 kHz mu-law audio.

import {
  createElevenLabs,
  type ElevenLabsProviderSettings,
} from '@ai-sdk/elevenlabs';
import { experimental_streamTranscribe as streamTranscribe } from 'ai';
import { WebSocket } from 'ws';

const elevenLabs = createElevenLabs({
  webSocket: WebSocket as unknown as ElevenLabsProviderSettings['webSocket'],
});

const result = streamTranscribe({
  model: elevenLabs.transcription('scribe_v2_realtime'),
  audio,
  inputAudioFormat: { type: 'audio/pcm', rate: 16000 },
  providerOptions: {
    elevenlabs: {
      languageCode: 'en',
      streaming: {
        includeLanguageDetection: true,
        includeTimestamps: true,
      },
    },
  },
});

for await (const part of result.fullStream) {
  if (part.type === 'transcript-partial') {
    console.log('partial:', part.text);
  }

  if (part.type === 'transcript-final') {
    console.log('final:', part.text);
  }
}

The ElevenLabs realtime endpoint authenticates with a WebSocket header. Use a header-capable WebSocket implementation, such as ws, in runtimes whose native WebSocket constructor cannot send headers.

The streaming provider options mirror ElevenLabs' realtime transcription settings: commitStrategy, enableLogging, filterBackgroundAudio, includeLanguageDetection, includeTimestamps, keyterms, minSilenceDurationMs, minSpeechDurationMs, noVerbatim, previousText, secondaryLanguages, vadSilenceThresholdSecs, and vadThreshold.

ElevenLabs delivers detected language on its timestamp-bearing committed event. When includeLanguageDetection is enabled, the provider requests that event internally, but only returns timestamp segments when includeTimestamps is also enabled. filterBackgroundAudio cannot be combined with either option.

See Transcription for more information about streaming transcription.

Model Capabilities