LLM API / Gateway

Optionally, you can configure Langfuse to use an external LLM API or gateway for add-on features. Langfuse tracing does not need access to the LLM API as traces are captured client-side.

Supported LLM APIs

Langfuse supports:

• OpenAI
• Azure OpenAI
• Anthropic
• Google Vertex
• Amazon Bedrock

Via the OpenAI API, many other LLM services and proxies can be used, including any LLM gateway that exposes an OpenAI-compatible endpoint.

Features powered by LLM API

• Playground
• LLM-as-a-Judge Evaluation
• Prompt Experiments

Halo and Ask AI do not use project LLM connections. Configure an instance-wide Langfuse AI model instead. See self-hosted Halo setup.