SourceyRetriever integration

Overview

Sourcey already emits the files this retriever needs.

Use SourceyRetriever to retrieve from a published Sourcey docs site. It reads search-index.json, uses llms-full.txt when present, and returns Document objects with canonical page URLs in metadata.source.

Integration details

Retriever	Source	Package
`SourceyRetriever`	Published Sourcey docs sites.	`langchain-sourcey`

Setup

Installation

npm

yarn

pnpm

npm install langchain-sourcey @langchain/core

yarn add langchain-sourcey @langchain/core

pnpm add langchain-sourcey @langchain/core

No API key is required.

Instantiation

siteUrl should point at the root of a published Sourcey docs build.

import { SourceyRetriever } from "langchain-sourcey";

const retriever = new SourceyRetriever({
  siteUrl: "https://sourcey.com/docs",
  topK: 3,
});

Usage

const docs = await retriever.invoke("mcp integration");

for (const doc of docs) {
  console.log(doc.metadata.title);
  console.log(doc.metadata.source);
}

Use within a chain

Install a chat model package. This example uses OpenAI:

npm

yarn

pnpm

npm install @langchain/openai

yarn add @langchain/openai

pnpm add @langchain/openai

import type { Document } from "@langchain/core/documents";
import { StringOutputParser } from "@langchain/core/output_parsers";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { RunnablePassthrough, RunnableSequence } from "@langchain/core/runnables";
import { ChatOpenAI } from "@langchain/openai";
import { SourceyRetriever } from "langchain-sourcey";

const retriever = new SourceyRetriever({
  siteUrl: "https://sourcey.com/docs",
  topK: 3,
});

const prompt = ChatPromptTemplate.fromTemplate(
  `Answer the question using the documentation context below.

{context}

Question: {question}`
);

const formatDocs = (docs: Document[]) =>
  docs.map((doc) => doc.pageContent).join("\n\n");

const chain = RunnableSequence.from([
  {
    context: retriever.pipe(formatDocs),
    question: new RunnablePassthrough(),
  },
  prompt,
  new ChatOpenAI({ model: "gpt-4.1-mini" }),
  new StringOutputParser(),
]);

await chain.invoke("How does Sourcey document MCP servers?");

Sourcey site requirements

For clean retrieval, the published Sourcey site should:

publish search-index.json
publish llms-full.txt
set siteUrl in sourcey.config.ts so returned citations are canonical

If llms-full.txt is not available, SourceyRetriever falls back to extracting text from the matched HTML page.

Sourcey guide: Using Sourcey with LangChain
Package: langchain-sourcey on npm