Software Engineer · intermediate
AI-Powered RAG Knowledge Base for Developers
A RAG system where users upload technical docs and ask questions — with accurate, cited answers from a vector DB.
Project overview
AI integration is one of the most sought-after skills right now. This project proves you can bridge the gap between traditional software engineering and modern AI capabilities, focusing on data privacy and reducing hallucinations.
Real-world use case
Internal company wikis or customer support bots that need to provide answers based on private, frequently updated technical manuals rather than general internet knowledge.
Tools and technologies
- Python
- LangChain
- Pinecone/Milvus
- FastAPI
- React
Skills you can demonstrate
AI/ML Integration, Vector Databases, Semantic Search
Build roadmap
- Build the ingestion pipeline
Accept PDF and Markdown uploads and split them with RecursiveCharacterTextSplitter at roughly 500 tokens with 50 overlap. Keep filename, page, and section as metadata on every chunk. - Generate embeddings
Use a hosted embedding model or sentence-transformers locally; batch requests and cache by content hash so re-uploads cost nothing. Note the dimension — it fixes your index. - Set up the vector DB
Pinecone serverless for zero ops, Milvus to self-host. Upsert chunks with metadata and confirm a known question retrieves the right chunk before going further. - Build the retrieval API
A FastAPI endpoint that embeds the question, searches top-k, and returns chunks with scores. Add a keyword fallback for exact identifiers that embeddings miss. - Add the LLM with citations
Send retrieved chunks to Claude with a prompt that requires inline citations to chunk ids. Parse and link them, and refuse to answer when nothing relevant was retrieved. - Build the React chat UI
Upload drop zone, chat window, and clickable citations that highlight the source passage. Record a 30-second demo of a question, its answer, and a citation click.
Ways to extend the project
Implement "Agentic RAG" where the system can decide to search the web or call an external API if the local knowledge base doesn't have the answer.
How to showcase your work
- GitHub: Include a "Hallucination Evaluation" report showing how chunking strategies affected quality.
- LinkedIn: Video demo showing the system citing specific lines from a technical manual.
- Resume: Focus on RAG, vector DBs, and accuracy evaluation.