Trace TanStack AI with Langfuse

TanStack AI is TanStack's framework for building AI-powered applications in TypeScript, with a provider-agnostic chat() API, streaming, and tool calling. By combining TanStack AI with Langfuse, you can trace, monitor, and analyze every agent step, tool call, and model response in development and production.

This notebook demonstrates how to use the openInferenceMiddleware from OpenInference to automatically instrument TanStack AI calls and send OpenTelemetry spans to Langfuse.

What is TanStack AI?
TanStack AI is a TypeScript framework for building AI applications. It provides a unified, provider-agnostic chat() API with built-in support for streaming, tool calling, and pluggable provider adapters (OpenAI, Anthropic, and more).

What is Langfuse?
Langfuse is an open source platform for LLM observability and monitoring. It helps you trace and monitor your AI applications by capturing metadata, prompt details, token usage, latency, and more. Use Langfuse Cloud (hosted by Langfuse, free tier, no infrastructure to run) or self-host it.

Step 1: Install Dependencies

Install the necessary packages:

npm install @tanstack/ai @tanstack/ai-openai @arizeai/openinference-tanstack-ai @langfuse/otel @opentelemetry/sdk-node

Note: This cookbook uses Deno.js for execution, which requires different syntax for importing packages and setting environment variables. For Node.js applications, the setup process is similar but uses standard npm packages and process.env.

Step 2: Configure Environment

Set up your Langfuse and OpenAI API keys. You can get Langfuse keys by signing up for a free Langfuse Cloud account or by self-hosting Langfuse. Get your OpenAI API key from the OpenAI Platform.

Deno.env.set("OPENAI_API_KEY", "sk-proj-...");

Deno.env.set("LANGFUSE_PUBLIC_KEY", "pk-lf-...");
Deno.env.set("LANGFUSE_SECRET_KEY", "sk-lf-...");

Deno.env.set("LANGFUSE_BASE_URL", "https://cloud.langfuse.com"); // 🇪🇺 EU region
// Other Langfuse data regions include 🇺🇸 US: https://us.cloud.langfuse.com, 🇯🇵 Japan: https://jp.cloud.langfuse.com and ⚕️ HIPAA: https://hipaa.cloud.langfuse.com

Step 3: Initialize OpenTelemetry with Langfuse

Set up the OpenTelemetry SDK with the LangfuseSpanProcessor. The openInferenceMiddleware from OpenInference uses your application's existing OpenTelemetry tracer provider, so this setup must run before any TanStack AI calls.

We also provide a custom shouldExportSpan filter to include spans from the @arizeai/openinference-tanstack-ai instrumentation scope alongside the default Langfuse filter.

import { NodeSDK } from "npm:@opentelemetry/sdk-node";
import { LangfuseSpanProcessor, isDefaultExportSpan } from "npm:@langfuse/otel";

const sdk = new NodeSDK({
  spanProcessors: [
    new LangfuseSpanProcessor({
      shouldExportSpan: ({ otelSpan }) =>
        isDefaultExportSpan(otelSpan) ||
        otelSpan.instrumentationScope.name ===
          "@arizeai/openinference-tanstack-ai",
    }),
  ],
});

sdk.start();

Step 4: Run the Agent

Add openInferenceMiddleware() to the middleware option of TanStack AI's chat() call. All agent steps, tool calls, and model completions are automatically traced and sent to Langfuse. The middleware works for both streaming and non-streaming calls.

import { chat } from "npm:@tanstack/ai";
import { openaiText } from "npm:@tanstack/ai-openai";
import { openInferenceMiddleware } from "npm:@arizeai/openinference-tanstack-ai";

const text = await chat({
  adapter: openaiText("gpt-4o-mini"),
  stream: false,
  systemPrompts: ["You are a concise technical explainer."],
  messages: [
    { role: "user", content: "What is the capital of France? Answer in a single sentence." },
  ],
  middleware: [openInferenceMiddleware()],
});

console.log(text);

await sdk.shutdown();

Step 5: View Traces in Langfuse

After running the agent, navigate to your Langfuse Trace Table. You will find detailed traces of the agent's execution, providing insights into every agent step, tool call, input, output, and performance metric.

Interoperability with the JS/TS SDK

You can use this integration together with the Langfuse SDKs to add additional attributes or group observations into a single trace.

Context Manager

Observe Wrapper

The Context Manager allows you to wrap your instrumented code using context managers (with with statements), which allows you to add additional attributes to the trace. Any observation created inside the callback will automatically be nested under the active observation, and the observation will be ended when the callback finishes.

import { startActiveObservation, propagateAttributes } from "npm:@langfuse/tracing";

await startActiveObservation("context-manager", async (span) => {
  span.update({
    input: { query: "What is the capital of France?" },
  });

  // Propagate userId to all child observations
  await propagateAttributes(
    {
      userId: "user-123",
      sessionId: "session-123",
      metadata: {
        source: "api",
        region: "us-east-1",
      },
      tags: ["api", "user"],
      version: "1.0.0",
    },
    async () => {

      // YOUR CODE HERE
      const { text } = await generateText({
        model: openai("gpt-5"),
        prompt: "What is the capital of France?",
        experimental_telemetry: { isEnabled: true },
      });
    }
  );
  span.update({ output: "Paris" });
});

Learn more about using the Context Manager in the Langfuse SDK instrumentation docs.

The observe wrapper is a powerful tool for tracing existing functions without modifying their internal logic. It acts as a decorator that automatically creates a span or generation around the function call. You can use the propagateAttributes function to add attributes to the observation from within the wrapped function.

import { observe, propagateAttributes } from "@langfuse/tracing";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";

// An existing function
const processUserRequest = observe(
  async (userQuery: string) => {

    // Propagate attributes to all child observations
    return await propagateAttributes(
      {
        userId: "user-123",
        sessionId: "session-123",
        metadata: {
          source: "api",
          region: "us-east-1",
        },
        tags: ["api", "user"],
        version: "1.0.0",
      },
      async () => {

        // YOUR CODE HERE
        const { text } = await generateText({
          model: openai("gpt-5"),
          prompt: userQuery,
          experimental_telemetry: { isEnabled: true },
        });

        return text;
      }
    );
  },
  { name: "process-user-request" }
);

const result = await processUserRequest("some query");

Learn more about using the Decorator in the Langfuse SDK instrumentation docs.

Troubleshooting

First, enable debug mode in the JS/TS SDK:

export LANGFUSE_LOG_LEVEL="DEBUG"

Then run your application and check the debug logs:

OTel spans appear in the logs: Your application is instrumented correctly but traces are not reaching Langfuse. To resolve this:
1. Call forceFlush() at the end of your application to ensure all traces are exported. This is especially important in short-lived environments like serverless functions.
2. Verify that you are using the correct API keys and base URL.
No OTel spans in the logs: Your application is not instrumented correctly. Make sure the instrumentation runs before your application code.

Next Steps

Once you have instrumented your code, you can manage, evaluate and debug your application:

Manage prompts in Langfuse Add evaluation scores Run LLM-as-a-judge Evaluators Create datasets Create custom dashboards Test queries in the Playground