Most RAG pipelines fail at the preprocessing stage. I see teams dumping entire PDFs into vector stores and wondering why Claude retrieves irrelevant context for support tickets.
Here's the fix:
Semantic chunking with overlap. Instead of fixed 1000-character cuts, use semantic boundaries (paragraphs/sections) with 20% overlap. For technical docs, I use 512-token chunks with 100-token overlap to prevent context loss across boundaries.
Metadata filtering first. Tag chunks by document type (invoice vs troubleshooting), product version, and date. Your retrieval should filter the namespace before running vector similarity—cuts latency by 60% and eliminates hallucinations from outdated docs.
Hybrid search + reranking. Combine keyword (BM25) + vector similarity, then rerank with Cohere or similar. This catches synonyms like "login issues" and "authentication errors" that pure vector search misses.
I recently rebuilt a support bot that was scoring 0.31 on relevance. After switching to semantic chunking + metadata filtering, it hit 0.89. The bottleneck wasn't the LLM—it was the preprocessing.
What's your chunking strategy? Fixed-size or semantic boundaries?