AWS middleware integration

Middleware specifically designed for models hosted on AWS Bedrock. Learn more about middleware.

Middleware	Description
Prompt caching	Reduce costs by caching repetitive prompt prefixes

Prompt caching

Reduce inference latency and input token costs by caching frequently reused prompt prefixes on Amazon Bedrock. bedrockPromptCachingMiddleware enables caching through modelSettings. ChatBedrockConverse then translates that into the correct AWS wire format at request time. Cache checkpoints are placed after the system prompt, tool definitions, and the most recent message where supported, so that the model can skip recomputation of previously seen content on subsequent requests. Cache placement varies by model family: for example, Nova skips some tool definition and tool-result cases.

Prompt caching is useful for the following:

Multi-turn conversations with long, consistent system prompts
Agents with many tool definitions that remain constant across invocations
Document-based Q&A where users ask multiple questions over the same uploaded context
Batch processing workloads with repeated static content

Supported models:

Anthropic Claude
Amazon Nova

Info

Learn more about AWS Bedrock prompt caching strategies and limitations. Cached content must exceed 1,024 tokens for a cache checkpoint to take effect, sometimes more depending on model. See supported models, regions, and limits.

API reference: BedrockPromptCachingMiddleware

import { createAgent, bedrockPromptCachingMiddleware } from "langchain";

const agent = createAgent({
  model: "bedrock:us.anthropic.claude-sonnet-4-5-20250929-v1:0",
  systemPrompt: "<Your long system prompt here>",
  middleware: [bedrockPromptCachingMiddleware({ ttl: "1h" })], // [!code highlight]
});

import { createAgent, bedrockPromptCachingMiddleware } from "langchain";
import { ChatBedrockConverse } from "@langchain/aws";

const agent = createAgent({
  model: new ChatBedrockConverse({ model: "us.anthropic.claude-sonnet-4-5-20250929-v1:0" }),
  systemPrompt: "<Your long system prompt here>",
  middleware: [bedrockPromptCachingMiddleware({ ttl: "5m" })], // [!code highlight]
});

The middleware caches content up to and including the latest message in each request. On subsequent requests within the TTL window (5 minutes or 1 hour), previously seen content is retrieved from cache rather than reprocessed, reducing costs and latency.

How it works:

First request: System prompt, tools, and the user message are sent to the API and cached
Second request: The cached content is retrieved from cache. Only the new message needs to be processed
This pattern continues for each turn, with each request reusing the cached conversation history

Note

Prompt caching reduces API costs by caching tokens, but does not provide conversation memory. To persist conversation history across invocations, use a checkpointer like MemorySaver.

import { createAgent, bedrockPromptCachingMiddleware, AIMessage, HumanMessage, tool } from "langchain";
import { ChatBedrockConverse } from "@langchain/aws";
import { z } from "zod";

const getWeather = tool(
  async ({ city }) => `The weather in ${city} is sunny and 72F.`,
  {
    name: "get_weather",
    description: "Get the current weather for a city.",
    schema: z.object({ city: z.string() }),
  }
);

// System prompt must exceed 1,024 tokens for caching to take effect
const LONG_PROMPT =
  "You are a helpful weather assistant with deep expertise in meteorology, " +
  "climate science, and atmospheric phenomena. When answering questions about " +
  "weather, provide accurate and up-to-date information. " +
  "You should always strive to give the most helpful response possible. ".repeat(85);

const agent = createAgent({
  model: new ChatBedrockConverse({ model: "us.anthropic.claude-sonnet-4-5-20250929-v1:0" }),
  systemPrompt: LONG_PROMPT,
  tools: [getWeather],
  middleware: [bedrockPromptCachingMiddleware({ ttl: "5m" })], // [!code highlight]
});

// First invocation: writes the cache (system prompt, tool definitions, and message)
let response = await agent.invoke({
  messages: [new HumanMessage("What is the weather in Miami?")],
});
const last = response.messages.at(-1);
console.log(last?.content);

// Check cache token usage
if (AIMessage.isInstance(last)) {
  const details = last.usage_metadata?.input_token_details;
  if (details) {
    console.log(`Cache read: ${details.cache_read ?? 0}, Cache write: ${details.cache_creation ?? 0}`);
  }
}

// Second invocation within the TTL: reuses the cached system prompt and tool definitions
response = await agent.invoke({
  messages: [new HumanMessage("How about Seattle?")],
});
console.log(response.messages.at(-1)?.content);

Model-specific behavior

The middleware handles differences between model families automatically:

Feature	ChatBedrockConverse (Anthropic)	ChatBedrockConverse (Nova)
System prompt caching	✅	✅
Tool definition caching	✅	❌
Message caching	✅	✅ (excludes tool result messages)
Extended TTL (`1h`)	✅	❌

Prompt caching

Configuration options

Full example

Model-specific behavior