LocalAI
Open source · MIT licensed
One runtime.
Every kind of AI.
Your hardware.
LocalAI runs text, vision, speech, sound, images, video, embeddings, reranking, and autonomous agents behind one modular stack—from a CPU laptop to a distributed GPU cluster.
The runtime, not just the endpoint.
Bring the model. Choose the engine. Keep control.
A small core, not a giant bundle.
Backends arrive when the model needs them.
LocalAI keeps the core lean. Each backend wraps a best-in-class engine—llama.cpp, vLLM, SGLang, MLX, whisper.cpp, diffusion engines, and many more—as an isolated service pulled on demand.
- Install, update, or remove engines independently.
- Mix CPU, NVIDIA, AMD, Intel, Apple Silicon, Vulkan, and Jetson.
- Build your own backend in any language through an open gRPC contract.
We integrate the best engines. We build new ones, too.
Inference work that moves the open ecosystem forward.
The LocalAI team develops native C, C++, Rust, and GGML engines when the available stack is too heavy, too closed, or simply does not exist yet.
See the engines we maintain ↗Start on one machine. Keep going.
The same runtime from workstation to private AI fabric.
Run useful models locally, including CPU-only setups.
Add authentication, API keys, roles, quotas, and usage visibility.
Route across workers, fit models across devices, and scale with demand.
More than inference
A complete local AI control plane.
Create agents with MCP tools, skills, memory, RAG, citations, and streamed execution from the UI or API.
Build interruptible voice experiences with WebRTC, streaming STT, LLM output, and TTS.
Keep data on your infrastructure and add PII analysis, redaction, policy middleware, and audit visibility.
Discover capabilities, import models, fine-tune, quantize, route, and monitor them in one place.
One command to begin
Run your first local AI stack.
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest