AI RAG Pipeline

The problem

Out-of-the-box LLM answers were confident and often wrong. The model had no access to internal context, so responses drifted, hallucinated, and couldn't cite a source. We needed grounded, traceable answers at interactive latency.

Research

Benchmarked chunking strategies, embedding models, and rerankers against a hand-labelled relevance set. Naive top-k similarity left too much noise in the context window.

• ▹Fixed-size vs. semantic chunking trade-offs
• ▹Embedding model recall vs. cost curves
• ▹Reranking lift measured against a gold set

Strategy

Treat retrieval as a pipeline, not a lookup: hybrid search to widen recall, a reranking pass to sharpen precision, then disciplined context assembly so the model reads the smallest sufficient set.

Development

Built an ingestion path that normalises, chunks, and embeds sources into Pinecone, with an evaluation harness that scores every change against the gold set so quality never regresses silently.

• ▹Hybrid (dense + keyword) retrieval
• ▹Cross-encoder reranking stage
• ▹Context budgeting before generation
• ▹Regression-tested retrieval quality

AI integration

Context injection was tuned per query type, with guardrails that force citations and fall back gracefully when confidence is low — so the system says 'I don't know' instead of inventing an answer.

Results

Answer relevance improved roughly 3x against the evaluation set while staying under interactive latency, with every answer traceable to its source.