Start narrow.
Use policies, notes, product docs, FAQs, or manuals where source paragraphs matter. Avoid huge corpora until retrieval is measurable.
Retrieval-augmented generation
Build a Free RAG Stack That Can Explain Its Sources.
A RAG project is not only an LLM call. It is a pipeline for documents, chunks, embeddings, vector search, reranking, answer generation, citations, evaluation, and maintenance.
Last updated May 21, 2026. Free tiers, model access, and dataset terms can change; verify official sources before launch.
Blueprint
The minimum useful RAG architecture.
DocsParseChunkEmbedVector DBRerankGenerateCiteEvaluate
Best beginner scope: 5 to 20 documents, 20 test questions, one embedding model, one vector database, citations in every answer, and a failure log. That is enough to prove the system without hiding behind complexity.
Optimize retrieval first.
Try BGE, E5, Nomic, Jina, or hosted embedding APIs. Evaluate with questions you expect users to ask.
Pick for workflow.
Use Chroma for notebooks, Qdrant for app prototypes, pgvector for Postgres apps, and Milvus or Weaviate for larger systems.
Improve precision.
Rerankers often improve RAG quality more than switching answer models because they improve what the model sees.
Evaluation
Judge RAG by grounded answers, not demo magic.
A RAG system should know when the documents do not contain the answer. Track retrieval hit rate, citation correctness, hallucinations, latency, and user satisfaction.
Simple RAG eval sheet
- Write 20 realistic questions.
- Mark which document chunk should answer each question.
- Check whether retrieval returns that chunk in the top results.
- Check whether the final answer cites the right source.
- Log every failure as parsing, chunking, retrieval, reranking, or generation.
People also ask
RAG stack questions that matter before implementation.
What is the cheapest RAG stack for a student project?
Use local documents, an open embedding model, Chroma or Qdrant, a hosted LLM with a free tier, and a small evaluation set. Keep the document set small until the workflow is correct.
Do I need a vector database for RAG?
For tiny demos, a local vector library can work. For apps with repeated search, metadata filters, persistence, and scaling needs, a vector database is usually easier to maintain.
Why does my RAG app hallucinate?
Common causes are poor parsing, chunks without enough context, weak retrieval, missing reranking, prompts that allow guessing, or questions not answered by the source documents.