My AI Assistant

Overview

A fully offline, privacy-first AI assistant built on a local RAG (Retrieval-Augmented Generation) pipeline. It ingests private documents — PDFs, web pages, plain text — and answers natural language questions with source context. No API keys, no cloud, no data leaving the machine.

The Problem

Most AI assistants send your data to cloud APIs. For sensitive documents — contracts, internal specs, personal notes — this is a non-starter. This project proves you can run a capable QA system entirely on local hardware.

Architecture

Documents (PDF/Web/Text)
    → Parser (pypdf / BeautifulSoup)
    → Embedder (sentence-transformers)
    → Vector Store (ChromaDB)
         ↓
User Query → Similarity Search → Context Chunks
    → Ollama (local LLM: llama3 / mistral)
    → Answer with cited sources

Technical Stack

Key Features

• Ingest PDFs, web pages, and plain text files
• Local vector embeddings — no external embedding API
• Source citation — every answer shows which document chunk was used
• Both a desktop GUI (PyQt6) and a terminal TUI (Textual)
• Swap models with one config change (llama3, mistral, phi-3, etc.)

Lessons Learned

ChromaDB's persistent mode writes embeddings to disk, so re-ingesting the same document is idempotent — it deduplicates by content hash. This made incremental document updates trivial.