Cartesia

The Cartesia provider contains Sonic speech generation, Ink-Whisper batch transcription, and Ink 2 realtime and streaming transcription support. Cartesia models are not currently available through AI Gateway, so the opening example uses @ai-sdk/cartesia directly. See Using AI Gateway below for supported alternatives.

import { generateSpeech } from 'ai';
import { cartesia } from '@ai-sdk/cartesia';
import { writeFile } from 'node:fs/promises';

const { audio } = await generateSpeech({
  model: cartesia.speech('sonic-3.5'),
  text: 'Hello, world!',
  outputFormat: 'mp3',
  voice: '694f9389-aac1-45b6-b726-9d9369183238',
});

await writeFile('speech.mp3', audio.uint8Array);

Using AI Gateway

The current AI Gateway model catalog does not list Cartesia models. Use @ai-sdk/cartesia for the APIs on this page. To use other speech models through Gateway, install ai and configure a Gateway API key:

pnpm add ai

npm install ai

yarn add ai

bun add ai

Set an AI Gateway API key in your environment:

export AI_GATEWAY_API_KEY=your_api_key_here

Vercel deployments can also use OIDC authentication. See the Gateway documentation for supported models and APIs.

Using @ai-sdk/cartesia

For direct access to Cartesia, install ai and @ai-sdk/cartesia:

pnpm add ai @ai-sdk/cartesia

npm install ai @ai-sdk/cartesia

yarn add ai @ai-sdk/cartesia

bun add ai @ai-sdk/cartesia

Set your Cartesia API credential in the CARTESIA_API_KEY environment variable:

export CARTESIA_API_KEY=your_api_key_here

Provider Instance

You can import the default provider instance cartesia from @ai-sdk/cartesia:

import { cartesia } from '@ai-sdk/cartesia';

If you need a customized setup, you can import createCartesia from @ai-sdk/cartesia and create a provider instance with your settings:

import { createCartesia } from '@ai-sdk/cartesia';

const cartesia = createCartesia({
  // custom settings, e.g.
  fetch: customFetch,
});

You can use the following optional settings to customize the Cartesia provider instance:

• apiKey string

  API key that is being sent using the Authorization header. It defaults to the CARTESIA_API_KEY environment variable.

• version string

  The Cartesia API version to use (sent via the Cartesia-Version header).

• headers Record<string,string>

  Custom headers to include in the requests.

• fetch (input: RequestInfo, init?: RequestInit) => Promise<Response>

  Custom fetch implementation. Defaults to the global fetch function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing.

• webSocket WebSocketConstructor

  Custom WebSocket implementation for Ink 2 streaming transcription. Defaults to the global WebSocket constructor.

Speech Models

You can create models that call the Cartesia text-to-speech API using the .speech() factory method.

The first argument is the model id, e.g. sonic-3.5.

const model = cartesia.speech('sonic-3.5');

You can use the model with the generateSpeech function. Cartesia requires a voice id:

import { generateSpeech } from 'ai';
import { cartesia } from '@ai-sdk/cartesia';

const result = await generateSpeech({
  model: cartesia.speech('sonic-3.5'),
  text: 'Hello, world!',
  voice: '694f9389-aac1-45b6-b726-9d9369183238',
});

You can also pass additional provider-specific options using the providerOptions argument:

import { generateSpeech } from 'ai';
import { cartesia, type CartesiaSpeechModelOptions } from '@ai-sdk/cartesia';

const result = await generateSpeech({
  model: cartesia.speech('sonic-3.5'),
  text: 'Hello, world!',
  voice: '694f9389-aac1-45b6-b726-9d9369183238',
  providerOptions: {
    cartesia: {
      container: 'wav',
      encoding: 'pcm_s16le',
      sampleRate: 24000,
    } satisfies CartesiaSpeechModelOptions,
  },
});

The following provider options are available:

• container string

  Container format for the output audio. Supported values: 'raw', 'wav', 'mp3'. Optional.

• encoding string

  Encoding type for the audio output. Supported values: 'pcm_f32le', 'pcm_s16le', 'pcm_mulaw', 'pcm_alaw'. Optional.

• sampleRate number

  Sample rate for the output audio in Hz (e.g. 8000, 16000, 22050, 24000, 44100, 48000). Optional.

• bitRate number

  Bitrate for mp3 output in bits per second (e.g. 32000, 64000, 128000, 192000). Optional.

• speed number

  Controls the speed of the generated speech between 0.6 and 1.5. Optional.

• language string

  The language to generate speech in (ISO 639-1 code). Optional.

Model Capabilities

Realtime Transcription Models

You can create a model for Cartesia's realtime Ink 2 API using the .experimental_realtime() factory method:

import { cartesia } from '@ai-sdk/cartesia';

const model = cartesia.experimental_realtime('ink-2');

Realtime sessions run in the browser and require a short-lived access token created on your server with cartesia.experimental_realtime.getToken():

const token = await cartesia.experimental_realtime.getToken({
  model: 'ink-2',
  sessionConfig: {
    inputAudioFormat: { type: 'audio/pcm', rate: 24000 },
    inputAudioTranscription: { language: 'en' },
    turnDetection: { type: 'server-vad' },
  },
});

Ink 2 produces input transcription events and supports English audio. Turn detection is enabled by default. To use manual finalization instead, set turnDetection to null in the session config and call commitAudio() when the current input is complete.

See Realtime for the complete browser session setup.

Model Capabilities

Streaming Transcription Models

Ink 2 is also available through the transcription streaming API. Create the model with .transcription() and pass it to experimental_streamTranscribe:

import { cartesia } from '@ai-sdk/cartesia';
import { experimental_streamTranscribe as streamTranscribe } from 'ai';

const result = streamTranscribe({
  model: cartesia.transcription('ink-2'),
  audio, // ReadableStream<Uint8Array | string> containing raw audio chunks
  inputAudioFormat: { type: 'audio/pcm', rate: 24000 },
  providerOptions: {
    cartesia: {
      language: 'en',
    },
  },
});

for await (const part of result.fullStream) {
  if (part.type === 'transcript-partial') {
    console.log('partial:', part.text);
  }

  if (part.type === 'transcript-final') {
    console.log('final:', part.text);
  }
}

The provider creates a short-lived, STT-scoped Cartesia access token before opening the WebSocket, so the API key is not included in the WebSocket URL. Send audio chunks at approximately their playback rate; Cartesia's realtime endpoint is intended for live audio rather than bulk file upload.

Ink 2 supports English audio and uses Cartesia's native turn detection by default. To finalize only when the input audio stream ends, pass providerOptions.cartesia.streaming.turnDetection: false.

By default, the provider maps audio/pcm, audio/pcmu, and audio/pcma to 16-bit PCM, G.711 μ-law, and G.711 A-law respectively. To stream another raw PCM representation, set providerOptions.cartesia.streaming.encoding to pcm_s32le, pcm_f16le, or pcm_f32le. For example:

const result = streamTranscribe({
  model: cartesia.transcription('ink-2'),
  audio,
  inputAudioFormat: { type: 'audio/pcm', rate: 48000 },
  providerOptions: {
    cartesia: {
      streaming: {
        encoding: 'pcm_f32le',
      },
    },
  },
});

See Transcription for more information about streaming transcription.

Model Capabilities

Batch Transcription Models

You can create models that call the Cartesia transcription API using the .transcription() factory method.

The first argument is the model id e.g. ink-whisper.

const model = cartesia.transcription('ink-whisper');

You can use the model with the transcribe function:

import { transcribe } from 'ai';
import { cartesia } from '@ai-sdk/cartesia';
import { readFile } from 'fs/promises';

const result = await transcribe({
  model: cartesia.transcription('ink-whisper'),
  audio: await readFile('audio.mp3'),
});

You can also pass additional provider-specific options using the providerOptions argument:

import { transcribe } from 'ai';
import {
  cartesia,
  type CartesiaTranscriptionModelOptions,
} from '@ai-sdk/cartesia';
import { readFile } from 'fs/promises';

const result = await transcribe({
  model: cartesia.transcription('ink-whisper'),
  audio: await readFile('audio.mp3'),
  providerOptions: {
    cartesia: {
      language: 'en',
    } satisfies CartesiaTranscriptionModelOptions,
  },
});

The following provider options are available:

• language string

  The language of the audio (ISO 639-1 code). If not specified, it defaults to English. Optional.

• timestampGranularities array of strings

  The timestamp granularities to populate. Currently only 'word' is supported. Optional.

Model Capabilities