https://localai.io/index.htmlThe open, modular AI runtime. Run text, vision, voice, image, video, agents, and more on hardware you control.Hugoenhttps://localai.io/docs/overview/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/overview/index.htmlWhat is LocalAI?https://localai.io/getting-started/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/getting-started/index.htmlWelcome to LocalAI! This section takes you from a fresh install to a working chat, a working API call, and your first customized model. Follow the pages in order the first time through; each one builds on the previous. Tip Haven’t installed LocalAI yet? See the Installation guide first. Docker is the recommended installation method for most users. The path Install LocalAI (see the Installation guide). Quickstart - start LocalAI, install your first model, and get your first chat reply from both the web interface and the API. Install and run models - the different ways to install models (gallery, Hugging Face, OCI, local files) and run them. Try it out - copy-paste examples for every OpenAI-compatible endpoint (chat, vision, functions, audio, images, embeddings). Customize models - edit model configuration and prompt templates to tune behavior. Troubleshooting - fixes for the most common install and startup problems. Once you have a model answering, the Build your first agent walkthrough shows how to turn that model into an autonomous agent that can use tools.https://localai.io/features/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/features/index.htmlLocalAI provides a comprehensive set of features for running AI models locally. The pages in this section are grouped by capability, and the left navigation is ordered to match these groups. Text Text Generation - Generate text with GPT-compatible models using various backends. OpenAI Functions and tools - Use function calling and the tools API with local models. Constrained Grammars - Control model output format with BNF grammars. Interleaved thinking - Reasoning models that interleave thought and output. Model aliases - Expose one model under several names. API discovery - How LocalAI advertises its capability surface. Agents Agents - Autonomous AI agents with tools, knowledge base, and skills. Agent Actions - The catalog of actions an agent can run. Model Context Protocol (MCP) - Give a model external tools over MCP. LocalAI Assistant - Chat to administer your LocalAI instance. Audio Audio to text - Transcribe audio to text. Text to audio - Generate speech, music, and sound effects from text. Audio classification - Classify sounds and audio events. Audio diarization - Separate speakers in an audio stream. Audio transform - Transform and process audio. Voice Activity Detection - Detect speech segments in audio data. Voice recognition - Identify speakers by voice. Realtime API - Low-latency multi-modal conversations (voice and text) over WebSocket. Vision GPT Vision - Analyze and understand images with vision-language models. Object detection - Detect and locate objects in images. Face recognition - Recognize faces in images. Image and Video Image Generation - Create images with Stable Diffusion and other diffusion models. Video Generation - Generate videos from text, image, or audio conditioning, including LongCat and avatar workflows. Retrieval Embeddings - Generate vector embeddings for semantic search and RAG applications. Reranker - Improve retrieval accuracy with cross-encoder models. Stores - Vector similarity search for embeddings. Distributed and acceleration Distributed inference - Scale inference across multiple nodes (P2P federation or production distributed mode). Distributed mode - The production distributed deployment. MLX distributed - Distributed inference on Apple Silicon with MLX. GPU Acceleration - Optimize performance with GPU support. Platform and model management Backends - Available backends and how to manage them. Model Gallery - Browse and install pre-configured models. Runtime Settings - Configure application settings via the web UI without restarting. Quantization - Quantize models for smaller memory footprint. Fine-tuning - Fine-tune models on your own data. Authentication - Protect the API with keys. For operator-facing runtime, proxy, and monitoring concerns (middleware, cloud and MITM proxies, backend monitor), see the Operations section.https://localai.io/advanced/index.htmlTue, 06 Oct 2020 08:49:15 +0000https://localai.io/advanced/index.htmlAdvanced usagehttps://localai.io/operations/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/operations/index.htmlOperator-facing runtime, proxy, and monitoring concernshttps://localai.io/reference/index.htmlTue, 06 Oct 2020 08:49:15 +0000https://localai.io/reference/index.htmlReferencehttps://localai.io/integrations/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/integrations/index.htmlCommunity Integrations The lists below cover software and community projects that integrate with LocalAI. Feel free to open up a Pull request (by clicking at the “Edit page” below) to get your project added! Build & Deploy aikit - Build and deploy custom LocalAI containers Helm chart - Deploy LocalAI on Kubernetes GitHub Actions - Use LocalAI in CI/CD workflows Web UIs localai-admin LocalAI-frontend QA-Pilot - Interactive chat for navigating GitHub code repositories Big AGI - Powerful web interface running entirely in the browser Agentic Libraries & Assistants cogito - Agentic library for Go LocalAGI - Local smart assistant with autonomous agents MCP Servers MCPs - Model Context Protocol servers OS Assistants Keygeist - AI-powered keyboard operator for Linux Voice VoxInput - Use voice to control your desktop IDE & Editor Plugins VSCode extension GPTLocalhost (Word Add-in) - Run LocalAI in Microsoft Word locally Framework Integrations Langchain (Python) - pypi langchain4j - Java LangChain lingoose - Go framework for LLM apps LLPhant - PHP library for LLMs and vector databases FlowiseAI - Low-code LLM app builder LLMStack Midori AI Subsystem Manager Terminal Tools ShellOracle - Terminal utility Shell-Pilot - Interact with LLMs via pure shell scripts Mods - AI on the command line Chat Bots Discord bot Slack bot Telegram bot Hellper (Telegram) Home Automation Extended OpenAI Conversation - Conversation agent for Home Assistant that supports a custom OpenAI endpoint LLM Vision - Image & video feed analysis for Home Assistant OpenAI TTS Speech Service - OpenAI TTS custom component for Home Assistant LocalAI Monitor - Monitor & control of LocalAI from Home Assistant Nextcloud integration plugin and AI assistant Automation & DevOps Reflexia - Auto-documentation GitHelper - GitHub bot for issues with code and documentation context kairos - Immutable Linux OS Other Integrations AnythingLLM Logseq GPT3 OpenAI plugin CodeGPT (JetBrains) - Custom OpenAI-compatible endpoints Wave Terminal - Native LocalAI support Obsidian BMO Chatbot spark openops (Mattermost) Model Gallery Examples Configuration Guides This section provides step-by-step instructions for configuring specific software to work with LocalAI.https://localai.io/faq/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/faq/index.htmlFrequently asked questions Here are answers to some of the most common questions. Do I need to install all the backends? No. You install only the backends your models use. LocalAI’s core is a single binary (or container) that provides the OpenAI-compatible API, request routing, the web UI, and agents. Each inference backend (llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX, and others) is a separate artifact, installed only when a model needs it.https://localai.io/basics/news/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/basics/news/index.htmlRelease notes have been now moved completely over Github releases. You can see the release notes here. 2026 Highlights July 2026: LongCat video and avatar generation - dedicated CUDA backend for LongCat-Video text/image-to-video and LongCat-Video-Avatar-1.5 speech-driven avatars. Includes multi-segment continuation, portrait and recorded-audio inputs in Studio, and an SDPA CUDA 13 ARM64 build for DGX Spark. April 2026: Audio Transform - generic audio-in / audio-out endpoint with optional reference signal. First implementation: LocalVQE C++ backend (joint AEC + noise suppression + dereverberation, DeepVQE-style). Both batch (POST /audio/transformations) and bidirectional WebSocket streaming (/audio/transformations/stream). Studio “Transform” tab with synchronized waveform players for input / reference / output. April 2026: Face recognition backend - insightface-powered 1:1 verification, 1:N identification, face embedding, face detection, and demographic analysis. Ships both a non-commercial buffalo_l model and an Apache 2.0 OpenCV Zoo alternative. May 2026: Speaker diarization - new /v1/audio/diarization endpoint returning “who spoke when” segments. Backed by sherpa-onnx (pyannote-3.0 + speaker embeddings + clustering) for pure diarization, and vibevoice-cpp for diarization bundled with long-form ASR. Supports json / verbose_json / rttm response formats. June 2026: Sound classification - new /v1/audio/classification endpoint for audio tagging / sound-event classification, returning scored AudioSet labels (baby cry, glass breaking, alarms, …). Backed by ced.cpp, a 527-class AudioSet tagger ported to ggml. June 2026: PII analyze / redact API - the PII detection pipeline (NER + restricted-regex pattern tiers) is now a standalone service: POST /api/pii/analyze returns detected entity spans and POST /api/pii/redact returns the sanitised text (or 400 pii_blocked), without routing a chat request through the middleware. Events gain an origin (middleware / proxy / pii_analyze / pii_redact) so /api/pii/events can be filtered by source. July 2026: Model capabilities endpoint - GET /v1/models/capabilities, an additive superset of /v1/models that reports each model’s capabilities plus its input_modalities / output_modalities (text / image / audio / video). Lets clients route attachments using inferred or explicitly declared model modalities instead of backend-name checks. June 2026: Concurrent scoring and PII NER on llama.cpp - the Score (router classifier) and TokenClassify (PII NER) primitives now ride llama.cpp’s server task queue instead of locking the context, so they run concurrently with chat/completion/embedding traffic and with each other. The known_usecases restriction that forced dedicated scorer/NER model configs on llama-cpp is lifted, repeated scoring calls reuse the prompt KV cache across candidates, and scoring inputs are no longer capped by the physical batch size. 2024 Highlights April 2024: Reranker API May 2024: Distributed inferencing, Decentralized P2P llama.cpp - Docs July/August 2024: P2P Dashboard, Federated mode and AI Swarms, P2P Global community pools, FLUX-1 support, P2P Explorer October 2024: Examples moved to LocalAI-examples November 2024: Voice Activity Detection (VAD), Bark.cpp backend December 2024: stablediffusion.cpp backend (ggml)