Groq

The Groq provider contains language model support for the Groq API. Start with AI Gateway for models in its catalog (recommended), or use @ai-sdk/groq to connect directly to Groq.

import { generateText } from 'ai';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-5.5',
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});

console.log(text);

Using AI Gateway

The AI SDK uses AI Gateway by default for model strings. Gateway provides access to models across providers through a single API key, with usage and cost tracking. Install ai:

pnpm add ai

npm install ai

yarn add ai

bun add ai

Set an AI Gateway API key in your environment:

export AI_GATEWAY_API_KEY=your_api_key_here

Vercel deployments can also use OIDC authentication. See the AI Gateway provider documentation for configuration and the model catalog for available models.

The example sets providerOptions.gateway.order to ['groq']. This sends the request to Groq first, then falls back to other available providers of the same model if Groq is unavailable or the request fails. order sets a preference; it does not restrict fallback to Groq. See provider ordering.

The dropdown lists models served by Groq through Gateway with a corresponding direct-provider example. Provider availability depends on the model and your team's Provider Allowlist.

Using @ai-sdk/groq

For direct access to Groq, install ai and @ai-sdk/groq:

pnpm add ai @ai-sdk/groq

npm install ai @ai-sdk/groq

yarn add ai @ai-sdk/groq

bun add ai @ai-sdk/groq

Set your Groq API credential in the GROQ_API_KEY environment variable:

export GROQ_API_KEY=your_api_key_here

AI Gateway does not expose every API in @ai-sdk/groq. Use the direct provider for provider-specific APIs or custom connection settings.

Provider Instance

You can import the default provider instance groq from @ai-sdk/groq:

import { groq } from '@ai-sdk/groq';

If you need a customized setup, you can import createGroq from @ai-sdk/groq and create a provider instance with your settings:

import { createGroq } from '@ai-sdk/groq';

const groq = createGroq({
  // custom settings
});

You can use the following optional settings to customize the Groq provider instance:

• baseURL string

  Use a different URL prefix for API calls, e.g. to use proxy servers. The default prefix is https://api.groq.com/openai/v1.

• apiKey string

  API key that is being sent using the Authorization header. It defaults to the GROQ_API_KEY environment variable.

• headers Record<string,string>

  Custom headers to include in the requests.

• fetch (input: RequestInfo, init?: RequestInit) => Promise<Response>

  Custom fetch implementation. Defaults to the global fetch function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing.

Language Models

You can create Groq models using a provider instance. The first argument is the model id, e.g. llama-3.1-8b-instant.

const model = groq('llama-3.1-8b-instant');

Reasoning Models

Groq offers reasoning models such as qwen/qwen3.6-27b and openai/gpt-oss-120b. You can configure how the reasoning is exposed in the generated text by using the reasoningFormat option. It supports the options parsed, hidden, and raw.

import { groq, type GroqLanguageModelChatOptions } from '@ai-sdk/groq';
import { generateText } from 'ai';

const result = await generateText({
  model: groq('qwen/qwen3.6-27b'),
  providerOptions: {
    groq: {
      reasoningFormat: 'parsed',
      reasoningEffort: 'default',
      parallelToolCalls: true, // Enable parallel function calling (default: true)
      user: 'user-123', // Unique identifier for end-user (optional)
      serviceTier: 'flex', // Use flex tier for higher throughput (optional)
    } satisfies GroqLanguageModelChatOptions,
  },
  prompt: 'How many "r"s are in the word "strawberry"?',
});

The following optional provider options are available for Groq language models:

• reasoningFormat 'parsed' | 'raw' | 'hidden'

  Controls how reasoning is exposed in the generated text. Supported by reasoning models such as qwen/qwen3.6-27b; available formats depend on the model.

  For a complete list of reasoning models and their capabilities, see Groq's reasoning models documentation.

• reasoningEffort 'low' | 'medium' | 'high' | 'none' | 'default'

  Controls the level of effort the model will put into reasoning.

  • qwen/qwen3.6-27b
    • Supported values:
      • none: Disable reasoning. The model will not use any reasoning tokens.
      • default: Enable reasoning.
  • openai/gpt-oss-20b and openai/gpt-oss-120b
    • Supported values:
      • low: Use a low level of reasoning effort.
      • medium: Use a medium level of reasoning effort.
      • high: Use a high level of reasoning effort.

  Defaults to default for qwen/qwen3.6-27b.

• structuredOutputs boolean

  Whether to use structured outputs.

  Defaults to true.

  When enabled, object generation will use the json_schema format instead of json_object format, providing more reliable structured outputs.

• strictJsonSchema boolean

  Whether to use strict JSON schema validation. When true, the model uses constrained decoding to guarantee schema compliance.

  Defaults to true.

  Only used when structuredOutputs is enabled and a schema is provided. See Groq's Structured Outputs documentation for details on strict mode limitations.

• parallelToolCalls boolean

  Whether to enable parallel function calling during tool use. Defaults to true.

• user string

  A unique identifier representing your end-user, which can help with monitoring and abuse detection.

• serviceTier 'on_demand' | 'performance' | 'flex' | 'auto'

  Service tier for the request. Defaults to 'on_demand'.

  • 'on_demand': Default tier with consistent performance and fairness
  • 'performance': Prioritized tier for latency-sensitive workloads
  • 'flex': Higher throughput tier (10x rate limits) optimized for workloads that can handle occasional request failures
  • 'auto': Uses on_demand rate limits first, then falls back to flex tier if exceeded

  For more details about service tiers and their benefits, see Groq's service tiers documentation.

Structured Outputs

Structured outputs are enabled by default for Groq models. You can disable them by setting the structuredOutputs option to false.

import { groq } from '@ai-sdk/groq';
import { generateText, Output } from 'ai';
import { z } from 'zod';

const result = await generateText({
  model: groq('moonshotai/kimi-k2-instruct-0905'),
  output: Output.object({
    schema: z.object({
      recipe: z.object({
        name: z.string(),
        ingredients: z.array(z.string()),
        instructions: z.array(z.string()),
      }),
    }),
  }),
  prompt: 'Generate a simple pasta recipe.',
});

console.log(JSON.stringify(result.output, null, 2));

You can disable structured outputs for models that don't support them:

import { groq, type GroqLanguageModelChatOptions } from '@ai-sdk/groq';
import { generateText, Output } from 'ai';
import { z } from 'zod';

const result = await generateText({
  model: groq('llama-3.1-8b-instant'),
  providerOptions: {
    groq: {
      structuredOutputs: false,
    } satisfies GroqLanguageModelChatOptions,
  },
  output: Output.object({
    schema: z.object({
      recipe: z.object({
        name: z.string(),
        ingredients: z.array(z.string()),
        instructions: z.array(z.string()),
      }),
    }),
  }),
  prompt: 'Generate a simple pasta recipe in JSON format.',
});

console.log(JSON.stringify(result.output, null, 2));

Example

You can use Groq language models to generate text with the generateText function:

import { groq } from '@ai-sdk/groq';
import { generateText } from 'ai';

const { text } = await generateText({
  model: groq('llama-3.1-8b-instant'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});

Image Input

Groq's multi-modal models like meta-llama/llama-4-scout-17b-16e-instruct support image inputs. You can include images in your messages using either URLs or base64-encoded data:

import { groq } from '@ai-sdk/groq';
import { generateText } from 'ai';

const { text } = await generateText({
  model: groq('meta-llama/llama-4-scout-17b-16e-instruct'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What do you see in this image?' },
        {
          type: 'file',
          mediaType: 'image',
          data: 'https://example.com/image.jpg',
        },
      ],
    },
  ],
});

You can also use base64-encoded images:

import { groq } from '@ai-sdk/groq';
import { generateText } from 'ai';
import { readFileSync } from 'fs';

const imageData = readFileSync('path/to/image.jpg', 'base64');

const { text } = await generateText({
  model: groq('meta-llama/llama-4-scout-17b-16e-instruct'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe this image in detail.' },
        {
          type: 'file',
          mediaType: 'image',
          data: `data:image/jpeg;base64,${imageData}`,
        },
      ],
    },
  ],
});

Model Capabilities

Browser Search Tool

Groq provides a browser search tool that offers interactive web browsing capabilities. Unlike traditional web search, browser search navigates websites interactively, providing more detailed and comprehensive results.

Supported Models

Browser search is available on Groq's GPT-OSS models, for example:

• openai/gpt-oss-20b
• openai/gpt-oss-120b

Basic Usage

import { groq } from '@ai-sdk/groq';
import { generateText } from 'ai';

const result = await generateText({
  model: groq('openai/gpt-oss-120b'), // Must use supported model
  prompt:
    'What are the latest developments in AI? Please search for recent news.',
  tools: {
    browser_search: groq.tools.browserSearch({}),
  },
  toolChoice: 'required', // Ensure the tool is used
});

console.log(result.text);

Streaming Example

import { groq } from '@ai-sdk/groq';
import { streamText } from 'ai';

const result = streamText({
  model: groq('openai/gpt-oss-120b'),
  prompt: 'Search for the latest tech news and summarize it.',
  tools: {
    browser_search: groq.tools.browserSearch({}),
  },
  toolChoice: 'required',
});

for await (const delta of result.stream) {
  if (delta.type === 'text-delta') {
    process.stdout.write(delta.text);
  }
}

Key Features

• Interactive Browsing: Navigates websites like a human user
• Comprehensive Results: More detailed than traditional search snippets
• Server-side Execution: Runs on Groq's infrastructure, no setup required
• Powered by Exa: Uses Exa search engine for optimal results
• Currently Free: Available at no additional charge during beta

Best Practices

• Use toolChoice: 'required' to ensure the browser search is activated
• Use a model that supports browser search, such as openai/gpt-oss-20b or openai/gpt-oss-120b
• The tool works automatically - no configuration parameters needed
• Server-side execution means no additional API keys or setup required

Model Validation

The provider ignores the tool for known model IDs without browser search support:

// ✅ Supported - will work
const result = await generateText({
  model: groq('openai/gpt-oss-120b'),
  tools: { browser_search: groq.tools.browserSearch({}) },
});

// ❌ Unsupported - will show warning and ignore tool
const result = await generateText({
  model: groq('llama-3.1-8b-instant'),
  tools: { browser_search: groq.tools.browserSearch({}) },
});
// Warning: "Browser search is not supported on model llama-3.1-8b-instant."

Transcription Models

You can create models that call the Groq transcription API using the .transcription() factory method.

The first argument is the model id e.g. whisper-large-v3.

const model = groq.transcription('whisper-large-v3');

You can also pass additional provider-specific options using the providerOptions argument. For example, supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency.

import { transcribe } from 'ai';
import { groq, type GroqTranscriptionModelOptions } from '@ai-sdk/groq';
import { readFile } from 'fs/promises';

const result = await transcribe({
  model: groq.transcription('whisper-large-v3'),
  audio: await readFile('audio.mp3'),
  providerOptions: {
    groq: { language: 'en' } satisfies GroqTranscriptionModelOptions,
  },
});

The following provider options are available:

• timestampGranularities string[] The granularity of the timestamps in the transcription. Defaults to ['segment']. Possible values are ['word'], ['segment'], and ['word', 'segment']. Note: There is no additional latency for segment timestamps, but generating word timestamps incurs additional latency. Important: Requires responseFormat to be set to 'verbose_json'.

• responseFormat string The format of the response. Set to 'verbose_json' to receive timestamps for audio segments and enable timestampGranularities. Set to 'text' to return only the transcribed text. Optional.

• language string The language of the input audio. Supplying the input language in ISO-639-1 format (e.g. 'en') will improve accuracy and latency. Optional.

• prompt string An optional text to guide the model's style or continue a previous audio segment. The prompt should match the audio language. Optional.

• temperature number The sampling temperature, between 0 and 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. If set to 0, the model will use log probability to automatically increase the temperature until certain thresholds are hit. Defaults to 0. Optional.

Model Capabilities