RAG (Retrieval-Augmented Generation) lets you ground an LLM's answers in your own data — documents, databases, knowledge bases — instead of relying solely on its training. In this guide I'll walk through building a production-grade RAG chatbot using Node.js, LangChain.js, Pinecone as the vector store, and OpenAI for embeddings and generation.

1. Architecture Overview

The RAG pipeline has two phases:

• Ingestion: Split documents → generate embeddings → upsert to Pinecone.
• Query: Embed the user question → query Pinecone for top-K chunks → pass chunks + question to GPT-4o → stream the response.

2. Project Setup

mkdir rag-chatbot && cd rag-chatbot
npm init -y
npm install express @langchain/openai @langchain/pinecone @pinecone-database/pinecone langchain dotenv

Create a .env file:

OPENAI_API_KEY=sk-...
PINECONE_API_KEY=...
PINECONE_INDEX=rag-index

3. Ingest Documents into Pinecone

This script loads text files, splits them into chunks, generates embeddings, and upserts them into Pinecone.

// ingest.js
import { PineconeStore } from "@langchain/pinecone";
import { OpenAIEmbeddings } from "@langchain/openai";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { Pinecone } from "@pinecone-database/pinecone";
import { readFileSync } from "fs";

const pc = new Pinecone({ apiKey: process.env.PINECONE_API_KEY });
const index = pc.index(process.env.PINECONE_INDEX);

const rawText = readFileSync("./data/knowledge-base.txt", "utf-8");

const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 1000,
  chunkOverlap: 150,
});

const docs = await splitter.createDocuments([rawText]);

const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-small" });

await PineconeStore.fromDocuments(docs, embeddings, {
  pineconeIndex: index,
  namespace: "default",
});

console.log(`Upserted ${docs.length} chunks to Pinecone.`);

4. Build the RAG Query Chain

// rag.js
import { ChatOpenAI } from "@langchain/openai";
import { OpenAIEmbeddings } from "@langchain/openai";
import { PineconeStore } from "@langchain/pinecone";
import { Pinecone } from "@pinecone-database/pinecone";
import { createRetrievalChain } from "langchain/chains/retrieval";
import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { ChatPromptTemplate } from "@langchain/core/prompts";

const pc = new Pinecone({ apiKey: process.env.PINECONE_API_KEY });
const index = pc.index(process.env.PINECONE_INDEX);

const vectorStore = await PineconeStore.fromExistingIndex(
  new OpenAIEmbeddings({ model: "text-embedding-3-small" }),
  { pineconeIndex: index, namespace: "default" }
);

const retriever = vectorStore.asRetriever({ k: 5 });

const llm = new ChatOpenAI({ model: "gpt-4o", streaming: true });

const prompt = ChatPromptTemplate.fromTemplate(`
You are a helpful assistant. Answer the question based only on the context below.
If the answer is not in the context, say "I don't have that information."

Context:
{context}

Question: {input}
`);

const documentChain = await createStuffDocumentsChain({ llm, prompt });
export const ragChain = await createRetrievalChain({ retriever, combineDocsChain: documentChain });

5. Express API with Streaming

// server.js
import express from "express";
import "dotenv/config";
import { ragChain } from "./rag.js";

const app = express();
app.use(express.json());

app.post("/chat", async (req, res) => {
  const { question } = req.body;
  if (!question) return res.status(400).json({ error: "question is required" });

  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.setHeader("Connection", "keep-alive");

  const stream = await ragChain.stream({ input: question });

  for await (const chunk of stream) {
    if (chunk.answer) {
      res.write(`data: ${JSON.stringify({ text: chunk.answer })}\n\n`);
    }
  }

  res.write("data: [DONE]\n\n");
  res.end();
});

app.listen(3000, () => console.log("RAG server running on port 3000"));

6. Testing the Endpoint

curl -X POST http://localhost:3000/chat \
  -H "Content-Type: application/json" \
  -d '{"question": "What are the refund policies?"}'

Production Considerations

• Namespace isolation: Use separate Pinecone namespaces per tenant or document set.
• Re-ranking: Add a Cohere re-ranker step after retrieval to improve relevance before passing to the LLM.
• Metadata filters: Add document metadata (date, source, category) to Pinecone and filter at query time.
• Prompt caching: If using Claude instead of GPT-4o, use Anthropic's prompt caching to cut costs on repeated system prompts by up to 90%.
• Rate limiting: Add express-rate-limit around the /chat endpoint to prevent abuse.

Conclusion

You now have a streaming RAG chatbot backed by Pinecone vector search. The same pattern scales to hundreds of thousands of documents by using Pinecone's serverless tier and batching your ingestion pipeline. For enterprise deployments I recommend adding a Redis cache layer for frequently asked questions to reduce OpenAI API costs significantly.

Need help building a RAG system for your product? Get in touch or hire me on Upwork.