Overview

PrivateGPT is an open-source API layer that turns local models into production AI applications. Rather than running models itself, it connects to any OpenAI-compatible inference server — Ollama, llama.cpp, vLLM, and others — and provides the higher-level building blocks you need to build private AI products: a standard messages API, document ingestion, retrieval-augmented generation (RAG) with citations, embeddings, custom tools, MCP connectors, and structured access to databases and CSVs.

The project began as a viral 2023 proof-of-concept that let you chat with your own documents fully offline, with no data leaving your machine. It crossed 50,000 GitHub stars and remains one of the most-watched AI repositories. Maintained by the Zylon team and released under the Apache-2.0 licence, PrivateGPT follows the Claude API as its reference model, so the API surface is familiar and complete — with streaming, async processing, token counting, extended thinking, and structured outputs. It ships a built-in workbench UI (at /ui) for testing and demos, while the API itself is the real product for developers.

Key Features

• Standard messages API: streaming, async batch processing, and token counting.
• Document and file ingestion: upload PDFs and other documents and retrieve answers with citations.
• Agentic RAG: grounded question-answering over your private data.
• Built-in tools: web search, web fetch, and code execution mirroring the Claude API.
• Custom tools and MCP connectors: extend the API with your own tools and external servers.
• Database and CSV access: structured querying and text-to-sql over your data.
• Embeddings and orchestration: file all the primitives for private AI apps in one layer.
• Claude-API-compatible: drop-in familiarity for teams already building on that model.

Use Cases

• Building private, on-premise AI applications and agents for an organisation.
• Secure document Q&A where data must never leave your network.
• Running a local OpenAI-compatible backend for coding tools (Claude Code, OpenCode, and others).
• Data analysis against databases and tables without sending them to a cloud API.

Platforms

Linux, macOS, and Windows (pip/uv, Homebrew, or Docker).

Licence

Apache-2.0 (open source)

Website

github.com/zylon-ai/private-gpt · docs.privategpt.dev