experimental_generateVideo

Generates videos based on a given prompt using a video model.

It is ideal for use cases where you need to generate videos programmatically, such as creating visual content, animations, or generating videos from images.

import { experimental_generateVideo as generateVideo } from 'ai';

const { videos } = await generateVideo({
  model: 'google/veo-3.1-generate-001',
  prompt: 'A cat walking on a treadmill',
  aspectRatio: '16:9',
});

console.log(videos);

Import

import { experimental_generateVideo } from "ai"

API Signature

Parameters

• model (VideoModelV4): The video model to use.
• prompt (string | GenerateVideoPrompt): The input prompt to generate the video from.
  • object: A prompt object for video generation with optional input image
    • image (DataContent): Input image for image-to-video generation. Can be a URL string, base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer.
    • text (string): The text prompt.
• n? (number): Number of videos to generate. Default: 1.
• aspectRatio? (string): Aspect ratio of the videos to generate. Format: {width}:{height}, or 'adaptive' to inherit the ratio from the input media (e.g. a first-frame image or a source video). Support for 'adaptive' is provider-specific.
• resolution? (string): Resolution of the videos to generate. Format: {width}x{height}.
• duration? (number): Duration of the video in seconds.
• fps? (number): Frames per second for the video.
• seed? (number): Seed for the video generation.
• frameImages? (Array<{ image: DataContent; frameType: "first_frame" | "last_frame" }>): Role-tagged image inputs for first-last-frame generation.
• inputReferences? (Array<DataContent | { data: DataContent; mediaType?: string }>): Reference image or video inputs for reference-to-video generation. Use the object form with an explicit mediaType for URL-based video references. Providers route each reference by its media type and warn when a reference kind is unsupported.
• generateAudio? (boolean): Whether the model should generate audio alongside the video.
• providerOptions? (ProviderOptions): Additional provider-specific options.
• maxVideosPerCall? (number): Maximum number of videos to generate per API call. When n exceeds this value, multiple API calls will be made.
• maxRetries? (number): Maximum number of retries. Default: 2.
• abortSignal? (AbortSignal): An optional abort signal to cancel the call.
• headers? (Record<string, string>): Additional HTTP headers for the request.
• download? ((options: { url: URL; abortSignal?: AbortSignal }) => Promise<{ data: Uint8Array; mediaType: string | undefined }>): Custom download function for fetching videos from URLs. Use createDownload() from ai to create a download function with custom size limits, e.g. createDownload({ maxBytes: 50 * 1024 * 1024 }). Default: built-in download with 2 GiB limit.
• poll? (object): Polling configuration for the asynchronous start/status flow.
  • object
    • intervalMs? (number): Interval between status checks in milliseconds. Default: 5000.
    • timeoutMs? (number): Maximum time to wait for completion in milliseconds, including while waiting for a webhook notification. Default: 600000 (10 minutes).
    • delay? ((delayInMs: number, options?: { abortSignal?: AbortSignal }) => PromiseLike<void>): Custom delay implementation for polling intervals and webhook timeouts. Useful for durable workflow sleep functions. Default: built-in timer-based delay.
• webhook? (() => PromiseLike<{ url: string; received: PromiseLike<VideoModelV4OperationWebhook> }>): Webhook factory for providers that support webhook notifications. When provided and the model supports webhooks, the SDK uses the webhook instead of polling. The factory should return a URL for the provider to notify and a received promise that resolves when the notification arrives. The poll option can also be provided to configure the webhook timeout and polling fallback.

Returns

• video (GeneratedFile): The first video that was generated.
  • GeneratedFile
    • base64 (string): Video as a base64 encoded string.
    • uint8Array (Uint8Array): Video as a Uint8Array.
    • mediaType (string): The IANA media type of the video (e.g., video/mp4).
• videos (Array<GeneratedFile>): All videos that were generated.
  • GeneratedFile
    • base64 (string): Video as a base64 encoded string.
    • uint8Array (Uint8Array): Video as a Uint8Array.
    • mediaType (string): The IANA media type of the video (e.g., video/mp4).
• warnings (Warning[]): Warnings from the model provider (e.g. unsupported settings).
• providerMetadata? (VideoModelProviderMetadata): Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. A videos key is typically present in the metadata and is an array with the same length as the top level videos key. Details depend on the provider.
• responses (Array<VideoModelResponseMetadata>): Response metadata from the provider. There may be multiple responses if we made multiple calls to the model.
  • VideoModelResponseMetadata
    • timestamp (Date): Timestamp for the start of the generated response.
    • modelId (string): The ID of the response model that was used to generate the response.
    • headers? (Record<string, string>): Response headers.
    • providerMetadata? (VideoModelProviderMetadata): Provider-specific metadata for this individual API call. Useful for accessing per-call metadata when multiple calls are made.