Fault tolerance | LangChain 简体中文

Fault tolerance middleware keeps your deep agent running when things go wrong. Not all errors should be handled the same way: transient failures (network timeouts, rate limits) should be retried automatically, errors the LLM can recover from (bad tool output, parsing failures) should be fed back to the model, and errors that need human input should pause the agent.

Error handling strategies

Different errors need different handling strategies:

Error type	Who fixes it	Strategy	Middleware or feature
Transient errors (network issues, rate limits)	System (automatic)	Retry with exponential backoff	ModelRetryMiddleware, ToolRetryMiddleware
User-fixable errors (missing information, unclear instructions)	Human	Pause with `interrupt()`	Human-in-the-loop
Provider outage	System (automatic)	Fall back to an alternative model	ModelFallbackMiddleware
Excessive calls (runaway loops)	System (automatic)	Cap model and tool calls per run	ModelCallLimitMiddleware, ToolCallLimitMiddleware
Unexpected errors	Developer	Let them bubble up	No middleware; let the exception propagate

The sections below cover each strategy with code examples.

Transient errors

User-fixable

Provider outage

Excessive calls

Unexpected

Add retry middleware to automatically retry network issues and rate limits. Model calls and tool calls each have their own retry middleware with exponential backoff:

import { createAgent, modelRetryMiddleware, toolRetryMiddleware } from "langchain";

const agent = createAgent({
  model: "google:gemini-3.6-flash",
  tools: [searchTool, fetchUrlTool],
  middleware: [
    modelRetryMiddleware({ maxRetries: 3, backoffFactor: 2.0, initialDelayMs: 1000 }),
    toolRetryMiddleware({
      maxRetries: 2,
      tools: ["search", "fetch_url"],
      retryOn: [TimeoutError, TypeError],
    }),
  ],
});

Pause and collect information from the user when needed (like account IDs, order numbers, or clarifications). Use interrupt_on to pause the agent before specific tool calls:

import { createDeepAgent } from "deepagents";

const agent = createDeepAgent({
  model: "google:gemini-3.6-flash",
  tools: [sendEmailTool, deleteRecordTool],
  interruptOn: {
    send_email: true,
    delete_record: true,
  },
});

For the full human-in-the-loop guide, see Human-in-the-loop.

If your primary model provider goes down entirely, use ModelFallbackMiddleware to switch to an alternative model:

import { createAgent, modelFallbackMiddleware } from "langchain";

const agent = createAgent({
  model: "google:gemini-3.6-flash",
  tools: [searchTool],
  middleware: [
    modelFallbackMiddleware("gpt-5.5"),
  ],
});

Without limits, a confused agent can burn through your LLM API budget in minutes by looping on the same tool call or making hundreds of model calls. Set caps on both model calls and tool executions per run:

import { createAgent, modelCallLimitMiddleware, toolCallLimitMiddleware } from "langchain";

const agent = createAgent({
  model: "google:gemini-3.6-flash",
  tools: [searchTool],
  middleware: [
    modelCallLimitMiddleware({ runLimit: 50 }),
    toolCallLimitMiddleware({ runLimit: 200 }),
  ],
});

Let them bubble up for debugging. Do not catch what you cannot handle. This pattern applies to custom middleware in the JavaScript SDK as well.

Rate limiting

There are two complementary ways to limit resource usage: controlling the request rate to your model provider, and capping the total number of calls per run.

Provider rate limiting

Chat model providers impose a limit on the number of invocations that can be made in a given time period. To control the rate at which requests are made, initialize your model with a rate_limiter:

Call limits

import { createAgent, modelCallLimitMiddleware, toolCallLimitMiddleware } from "langchain";

const agent = createAgent({
  model: "google:gemini-3.6-flash",
  middleware: [
    modelCallLimitMiddleware({ runLimit: 50 }),
    toolCallLimitMiddleware({ runLimit: 200 }),
  ],
});

Use run_limit to cap calls within a single invocation (resets each turn). Use thread_limit to cap calls across an entire conversation (requires a checkpointer). See ModelCallLimitMiddleware and ToolCallLimitMiddleware for the full configuration.

Retries

Transient failures (network timeouts, rate limits) should be retried automatically. Model calls and tool calls each have their own retry middleware with exponential backoff:

import {
  createAgent,
  modelRetryMiddleware,
  toolRetryMiddleware,
} from "langchain";

const agent = createAgent({
  model: "google:gemini-3.6-flash",
  middleware: [
    // Retry model calls on rate limits, timeouts, and 5xx errors
    modelRetryMiddleware({ maxRetries: 3, backoffFactor: 2.0, initialDelayMs: 1000 }),
    // Retry specific tools that hit external APIs (not all tools)
    toolRetryMiddleware({
      maxRetries: 2,
      tools: ["search", "fetch_url"],
      retryOn: [TimeoutError, TypeError],
    }),
  ],
});

Scope ToolRetryMiddleware to specific tools rather than retrying everything. A filesystem read_file that fails will not benefit from a retry, but a web search that times out probably will. See ModelRetryMiddleware for the full configuration.

Fallbacks

If your primary model provider goes down entirely, the fallback middleware switches to an alternative model:

import {
  createAgent,
  modelFallbackMiddleware,
} from "langchain";

const agent = createAgent({
  model: "google:gemini-3.6-flash",
  middleware: [
    // If the primary model is fully down, fall back to an alternative
    modelFallbackMiddleware("gpt-5.5"),
  ],
});

See ModelFallbackMiddleware for the full configuration.

Error handling

When a tool raises an exception during execution, the agent run halts by default. ToolErrorMiddleware is not yet available in the JavaScript SDK.