AnythingLLM
FreeFeed your own documents to a local model and get answers from your material
An all-in-one local knowledge base assistant. Import PDFs, Word files, web pages and code repositories into a workspace, and the model answers from that material with source citations. It bundles a vector database and embedding model, and paired with a local runner such as Ollama it works entirely offline. The Windows desktop installer is about 378 MB.
Why we picked it
Getting an AI to read your own material before answering — the pattern called RAG — normally means installing a vector database, choosing an embedding model, writing chunking logic and tuning retrieval. That is far too much for most people. AnythingLLM's value is packaging the whole stack into a desktop app that works on install. 64,707 stars, MIT-licensed.
Usage is direct: create a workspace and drag documents in — PDF, Word, TXT, Markdown, web links, even an entire GitHub repository. It handles parsing, chunking, embedding and indexing. Ask a question inside that workspace and the model retrieves relevant passages before answering, citing which document and section each answer came from, which matters enormously for verification.
Workspaces are isolated, so you can keep separate ones for company policies, project documentation and reading notes without cross-contamination.
Model connectivity is flexible: point it at a local runner such as Ollama or LM Studio (fully offline, with not one word of your material leaving the machine), or paste an API key for OpenAI, Anthropic and others (better quality, but your material goes to the provider and costs money). The choice is yours, and that is the point. The embedding model and vector store default to bundled local implementations requiring no configuration.
Things to note:
- The download is large (about 378 MB on Windows, 483 MB on macOS) because the embedding model and vector database ship inside.
- Answer quality depends on the model you attach. A small local model retrieves accurately but summarizes weakly. For material that matters, use something in the 7B class or above.
- It is not full-text search. RAG retrieves by semantic similarity, so for "where exactly does this word appear" a plain search tool remains better.
A Docker deployment exists for shared team use, alongside a paid cloud-hosted edition — the desktop and self-hosted versions themselves are entirely free with no feature limits.