
localGPT
Private, self-hosted document Q&A over your own files, running on local models
- Pricing
- Open source
- Platforms
- Web, API, CLI
- GitHub stars
- 22.2K
- Last commit
- 3 months ago
- Language
- Python
- License
- MIT
Similar tools
About localGPT
localGPT is an open-source AI tool for files and documents. It is available on Web, API, CLI, self-hosted, Linux, macOS and Windows.
The README says LocalGPT is an on-premise platform for asking questions of your own PDFs, DOCX, HTML, Markdown and text files, with all processing kept on the machine that runs it. Answers come from a local Ollama model, and document parsing is handled by Docling, so indexing and querying do not depend on a hosted API.
Under the hood it combines vector search with LanceDB full-text search, fuses the two result lists with reciprocal rank fusion, and reranks candidates with a cross-encoder. A router decides per query whether to retrieve from documents or answer directly from the model. Several optional stages such as late chunking and answer verification are shipped disabled, and the README cites its own evaluation records for those defaults.
It suits people who want to query internal documents without sending them to a third-party service, and developers who want a REST API on top of a local RAG pipeline. The README notes that the RAG API handles one request at a time and that changing the embedding model requires rebuilding indexes.
What it does
- Hybrid document retrieval
- Runs vector search and full-text search in parallel and merges results with reciprocal rank fusion.
- Local model serving
- Generation and utility models run through Ollama, with embeddings and reranking on CUDA, Apple MPS or CPU.
- Multi-format parsing
- Reads PDF, DOCX, HTML, Markdown and TXT via Docling, with OCR fallback for scanned PDFs.
- Per-query routing
- Chooses between retrieval over documents and a direct LLM answer for each question.
- Source attribution
- Returns the chunks that grounded each answer alongside the response text.
- REST and streaming API
- Exposes sessions, indexes and chat over HTTP, with Server-Sent Events for pipeline progress.
Good for
- Asking questions of confidential internal documents without uploading them to a cloud service
- Summarising and searching a folder of PDFs or manuals on a local machine
- Building a private document chat backend on top of its REST API
- Comparing retrieval settings through its configurable default and fast pipeline profiles
Pricing
Open source
Free and MIT-licensed; self-hosted. A contact form is listed for business deployment and customization.
localGPT FAQ
Does my data leave the machine?
The README says no data leaves the machine, and that generation runs through a local Ollama server. Models downloaded from HuggingFace are fetched once at first use.
What license is it under?
The project is licensed under MIT. The README notes that models used with it have their own licences.
Which file types can be indexed?
PDF, DOCX, HTML/HTM, Markdown and TXT files are supported for parsing and indexing.
Are there hardware requirements?
The README lists Python 3.10+, Node.js 20+, and 8GB+ RAM with 16GB+ recommended, plus Ollama.
Is localGPT free?
Yes. localGPT is open source, so you can use and self-host it for free.
What are the best alternatives to localGPT?
The closest alternatives to localGPT are PageIndex, anything-llm, WeKnora, chat-ollama and ragflow.
Where this comes from
- Description: written from the project's README.



