https://localai.io/docs/features/index.htmlLocalAI provides a comprehensive set of features for running AI models locally. The pages in this section are grouped by capability, and the left navigation is ordered to match these groups. Text Text Generation - Generate text with GPT-compatible models using various backends. OpenAI Functions and tools - Use function calling and the tools API with local models. Constrained Grammars - Control model output format with BNF grammars. Interleaved thinking - Reasoning models that interleave thought and output. Model aliases - Expose one model under several names. API discovery - How LocalAI advertises its capability surface. Agents Agents - Autonomous AI agents with tools, knowledge base, and skills. Agent Actions - The catalog of actions an agent can run. Model Context Protocol (MCP) - Give a model external tools over MCP. LocalAI Assistant - Chat to administer your LocalAI instance. Audio Audio to text - Transcribe audio to text. Text to audio - Generate speech, music, and sound effects from text. Audio classification - Classify sounds and audio events. Audio diarization - Separate speakers in an audio stream. Audio transform - Transform and process audio. Voice Activity Detection - Detect speech segments in audio data. Voice recognition - Identify speakers by voice. Realtime API - Low-latency multi-modal conversations (voice and text) over WebSocket. Vision GPT Vision - Analyze and understand images with vision-language models. Object detection - Detect and locate objects in images. Face recognition - Recognize faces in images. Image and Video Image Generation - Create images with Stable Diffusion and other diffusion models. Video Generation - Generate videos from text, image, or audio conditioning, including LongCat and avatar workflows. Retrieval Embeddings - Generate vector embeddings for semantic search and RAG applications. Reranker - Improve retrieval accuracy with cross-encoder models. Stores - Vector similarity search for embeddings. Distributed and acceleration Distributed inference - Scale inference across multiple nodes (P2P federation or production distributed mode). Distributed mode - The production distributed deployment. MLX distributed - Distributed inference on Apple Silicon with MLX. GPU Acceleration - Optimize performance with GPU support. Platform and model management Backends - Available backends and how to manage them. Model Gallery - Browse and install pre-configured models. Runtime Settings - Configure application settings via the web UI without restarting. Quantization - Quantize models for smaller memory footprint. Fine-tuning - Fine-tune models on your own data. Authentication - Protect the API with keys. For operator-facing runtime, proxy, and monitoring concerns (middleware, cloud and MITM proxies, backend monitor), see the Operations section.Hugoenhttps://localai.io/docs/features/text-generation/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/text-generation/index.htmlLocalAI supports generating text with GPT with llama.cpp and other backends (such as rwkv.cpp as ) see also the Model compatibility for an up-to-date list of the supported model families. Note: You can also specify the model name as part of the OpenAI token. If only one model is available, the API will use it for all the requests. API Reference Chat completions https://platform.openai.com/docs/api-reference/chathttps://localai.io/docs/features/openai-functions/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/openai-functions/index.htmlLocalAI supports running the OpenAI functions and tools API across multiple backends. The OpenAI request shape is the same regardless of which backend runs your model - LocalAI is responsible for extracting structured tool calls from the model’s output before returning the response.https://localai.io/docs/features/constrained_grammars/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/constrained_grammars/index.htmlOverview The chat endpoint supports the grammar parameter, which allows users to specify a grammar in Backus-Naur Form (BNF). This feature enables the Large Language Model (LLM) to generate outputs adhering to a user-defined schema, such as JSON, YAML, or any other format that can be defined using BNF. For more details about BNF, see Backus-Naur Form on Wikipedia.https://localai.io/docs/features/interleaved-thinking/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/interleaved-thinking/index.htmlReasoning models can “think” before they answer. When such a model also calls a tool, the useful behaviour is for the thinking and the tool call to travel together, and for that thinking to survive the tool-result round trip. LocalAI calls this interleaved thinking: a single assistant turn carries both reasoning and tool_calls, and the client hands the reasoning back on the next turn so the model’s chain of thought is not lost when the tool result is appended.https://localai.io/docs/features/model-aliases/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/model-aliases/index.htmlA model alias is a model name that redirects all traffic to another configured model. Declare gpt-4 as an alias of my-llama-3 and every client calling gpt-4 is served by my-llama-3 with no client reconfiguration: the clients keep their existing model name while you control what answers them on the server side. Declaring an alias Create a minimal config file in your models directory:https://localai.io/docs/features/api-discovery/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/api-discovery/index.htmlProgrammatic API discovery for agents, tools, and automationhttps://localai.io/docs/features/3d-generation/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/3d-generation/index.htmlLocalAI can generate textured 3D meshes from a single conditioning image via the /3d/generations endpoint, powered by the trellis2cpp backend — a C++/GGML port of Microsoft TRELLIS.2 (trellis2.cpp). The output is a binary glTF (.glb) asset with PBR materials. Generation is image-conditioned only — there is no text-prompt path. Provide a photo or rendering of a single object (ideally on a plain background) and TRELLIS.2 reconstructs a full 3D mesh from it.https://localai.io/docs/features/agents/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/agents/index.htmlLocalAI includes a built-in agent platform powered by LocalAGI. Agents are autonomous AI entities that can reason, use tools, maintain memory, and interact with external services, all running locally as part of the LocalAI process.https://localai.io/docs/features/agent-actions/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/agent-actions/index.htmlActions are the tools an agent can call while it reasons: search the web, open a browser, read or write a GitHub repository, generate an image, remember a fact, and so on. This page lists the actions that ship with LocalAI (through the embedded LocalAGI library) and explains how to enable one on an agent. For an end-to-end first run, see /docs/getting-started/first-agent/. For the agent platform overview, see /docs/features/agents/. Enabling an action on an agent Actions are attached per agent, not globally.https://localai.io/docs/features/mcp/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/mcp/index.htmlAgentic capabilities with Model Context Protocol integrationhttps://localai.io/docs/features/localai-assistant/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/localai-assistant/index.htmlInfo Looking for something else? LocalAI has three related agentic features that are easy to confuse: LocalAI Assistant (this page): an admin chat modality for administering LocalAI itself (install models, manage backends) by chatting. Agents (/docs/features/agents/): autonomous agents you build that reason, use tools, and act on their own. MCP (/docs/features/mcp/): a way to give a model external tools through the Model Context Protocol. LocalAI Assistant is an admin-only chat modality. When enabled on a chat session, the conversation is wired to an in-process MCP server that exposes LocalAI’s own admin/management surface as tools. You can install models, manage backends, edit model configs and check system status by chatting, with no REST calls or YAML edits.https://localai.io/docs/features/audio-to-text/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/audio-to-text/index.htmlAudio to text models are models that can generate text from an audio file. The transcription endpoint allows to convert audio files to text. The endpoint supports multiple backends: whisper.cpp: A C++ library for audio transcription (default) moonshine: Ultra-fast transcription engine optimized for low-end devices faster-whisper: Fast Whisper implementation with CTranslate2 parakeet-cpp: A C++/ggml port of NVIDIA NeMo Parakeet (FastConformer TDT/CTC/RNNT/hybrid). Runs quantized GGUFs on CPU or GPU, emits word-level timestamps, and supports cache-aware streaming (the realtime_eou model surfaces end-of-utterance events). llama-cpp: Route transcription to any multimodal-audio GGUF model served by the llama-cpp backend (e.g. Qwen3-ASR, Voxtral, Qwen2-Audio). Under the hood the request is converted into a chat completion with the audio attached via the model’s audio encoder - the same path the upstream llama.cpp server uses. Set backend: llama-cpp in the model YAML and point mmproj at the matching audio encoder. voxtral: Voxtral-family models served by a dedicated backend audio.cpp: Multi-family GGML audio engine. Serves transcription and forced alignment from families such as nemotron_asr, qwen3_asr, citrinet_asr, higgs_audio_stt and voxtral_realtime, and covers diarization, VAD, TTS and source separation from the same backend. See the audio.cpp backend page for the model options. The endpoint input supports all the audio formats supported by ffmpeg.https://localai.io/docs/features/text-to-audio/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/text-to-audio/index.htmlAPI Compatibility The LocalAI TTS API is compatible with the OpenAI TTS API and the Elevenlabs API. LocalAI API The /tts endpoint can also be used to generate speech from text. Usage Input: input, modelhttps://localai.io/docs/features/audio-classification/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/audio-classification/index.htmlSound-event classification (audio tagging) answers the question “what am I hearing?” - given an audio clip, it returns a list of scored AudioSet labels (e.g. Baby cry, infant cry, Glass breaking, Dog bark, Alarm). LocalAI exposes this through the /v1/audio/classification endpoint, modelled after /v1/audio/transcriptions. The reference backend is ced.cpp (CED, a 527-class AudioSet tagger), a small ViT over a log-mel spectrogram ported to ggml with full PyTorch parity. Apache-2.0 weights are redistributable as GGUF.https://localai.io/docs/features/audio-diarization/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/audio-diarization/index.htmlSpeaker diarization answers the question “who spoke when?” - given an audio clip with multiple speakers, it returns time-stamped segments labelled with a stable speaker ID (SPEAKER_00, SPEAKER_01, …). LocalAI exposes this through the /v1/audio/diarization endpoint, modelled after /v1/audio/transcriptions. Three backends are supported today:https://localai.io/docs/features/audio-transform/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/audio-transform/index.htmlThe audio-transform endpoints take audio in and emit audio out, optionally conditioned on a second reference audio signal. The category is generic by design - concrete operations include joint acoustic echo cancellation + noise suppression + dereverberation (LocalVQE), voice conversion (reference = target speaker), pitch shifting, audio super-resolution, and so on.https://localai.io/docs/features/voice-activity-detection/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/voice-activity-detection/index.htmlVoice Activity Detection (VAD) identifies segments of speech in audio data. LocalAI provides a /v1/vad endpoint powered by the Silero VAD backend. The audio.cpp backend also serves this endpoint, and ships the silero_vad and marblenet_vad assets inside its own package, so VAD works there with nothing to download (model: bundled:silero_vad plus the family:silero_vad option). API Method: POST Endpoints: /v1/vad, /vad Request The request body is JSON with the following fields:https://localai.io/docs/features/voice-recognition/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/voice-recognition/index.htmlLocalAI supports voice (speaker) recognition: speaker verification (1:1), speaker identification (1:N) against a built-in vector store, speaker embedding, and demographic analysis (age / gender / emotion from voice). The audio analog to Face Recognition, served over the same /v1/voice/* HTTP API by two backends:https://localai.io/docs/features/openai-realtime/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/openai-realtime/index.htmlLocalAI supports the OpenAI Realtime API which enables low-latency, multi-modal conversations (voice and text) over WebSocket. To use the Realtime API, you need to configure a pipeline model that defines the components for Voice Activity Detection (VAD), Transcription (STT), Language Model (LLM), and Text-to-Speech (TTS).https://localai.io/docs/features/audio-cpp/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/audio-cpp/index.htmlaudio.cpp is a GGML audio inference framework covering roughly thirty model families in one engine. LocalAI exposes it through the native C++ audio-cpp backend, so a single installed backend can serve text to speech, voice cloning, transcription, forced alignment, voice activity detection, speaker diarization, source separation, voice conversion and music generation, depending only on which model you point it at. audio.cpp is developed by ShugoAI LLC, not by the LocalAI project. Model packages live in huggingface.co/audio-cpp/audio.cpp-gguf.https://localai.io/docs/features/gpt-vision/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/gpt-vision/index.htmlLocalAI supports understanding images by using LLaVA, and implements the GPT Vision API from OpenAI. Usage OpenAI docs: https://platform.openai.com/docs/guides/vision First install a vision-capable model from the gallery (the examples below use moondream2-20250414, a small vision model): local-ai run moondream2-20250414 To let LocalAI understand and reply with what sees in the image, use the /v1/chat/completions endpoint, for example with curl:https://localai.io/docs/features/object-detection/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/object-detection/index.htmlLocalAI supports object detection and image segmentation through various backends. This feature allows you to identify and locate objects within images with high accuracy and real-time performance. Available backends include RF-DETR (Python) and rf-detr.cpp (native C++/ggml) for object detection and segmentation, and sam3.cpp for image segmentation (SAM 3/2/EdgeTAM). For detecting faces specifically, see the dedicated Face Recognition feature - its /v1/detection support is tuned for face bounding boxes and ships with commercially-safe model options.https://localai.io/docs/features/face-recognition/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/face-recognition/index.htmlLocalAI supports face recognition: face verification (1:1), face identification (1:N) against a built-in vector store, face embedding, face detection, demographic analysis (age / gender), and antispoofing / liveness detection.https://localai.io/docs/features/image-generation/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/image-generation/index.html(Generated with AnimagineXL) LocalAI supports generating images with Stable diffusion, running on CPU using C++ and Python implementations. Usage OpenAI docs: https://platform.openai.com/docs/api-reference/images/create To generate an image you can send a POST request to the /v1/images/generations endpoint with the instruction as the request body: curl http://localhost:8080/v1/images/generations -H "Content-Type: application/json" -d '{ "prompt": "A cute baby sea otter", "size": "256x256" }' Available additional parameters: mode, step.https://localai.io/docs/features/video-generation/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/video-generation/index.htmlLocalAI can generate videos from text prompts and optional image or audio conditioning via the /video endpoint. Supported backends include diffusers, stablediffusion, vllm-omni, and the dedicated longcat-video backend. API Method: POST Endpoint: /video Request The request body is JSON with the following fields:https://localai.io/docs/features/embeddings/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/embeddings/index.htmlLocalAI supports generating embeddings for text or list of tokens. For face embeddings specifically, see the Face Recognition feature - it produces 512-d L2-normalized vectors tuned for face similarity. For the API documentation you can refer to the OpenAI docs: https://platform.openai.com/docs/api-reference/embeddings Model compatibility The embedding endpoint is compatible with llama.cpp models, bert.cpp models and sentence-transformers models available in huggingface.https://localai.io/docs/features/reranker/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/reranker/index.htmlA reranking model, often referred to as a cross-encoder, is a core component in the two-stage retrieval systems used in information retrieval and natural language processing tasks. Given a query and a set of documents, it will output similarity scores.https://localai.io/docs/stores/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/stores/index.htmlStores are an experimental feature to help with querying data using similarity search. It is a low level API that consists of only get, set, delete and find. Tip Face recognition uses this store. The 1:N face identification flow (/v1/face/register, /v1/face/identify, /v1/face/forget) is built on top of the generic store - see Face Recognition for the face-oriented API. For example if you have an embedding of some text and want to find text with similar embeddings. You can create embeddings for chunks of all your text then compare them against the embedding of the text you are searching on.https://localai.io/docs/features/distribute/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/distribute/index.htmlTip Looking for production-grade horizontal scaling with PostgreSQL and NATS? See Distributed Mode. Choosing a distributed mode LocalAI can spread inference across multiple machines in three ways. Pick the one that matches your setup:https://localai.io/docs/features/distributed-mode/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/distributed-mode/index.htmlDistributed mode enables horizontal scaling of LocalAI across multiple machines using PostgreSQL for state and node registry, and NATS for real-time coordination. Unlike the P2P/federation approach, distributed mode is designed for production deployments and Kubernetes environments where you need centralized management, health monitoring, and deterministic routing. Note Distributed mode requires authentication enabled with a PostgreSQL database - SQLite is not supported. This is because the node registry, job store, and other distributed state are stored in PostgreSQL tables.https://localai.io/docs/features/mlx-distributed/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/mlx-distributed/index.htmlMLX distributed inference allows you to split large language models across multiple Apple Silicon Macs (or other devices) for joint inference. Unlike federation (which distributes whole requests), MLX distributed splits a single model’s layers across machines so they all participate in every forward pass.https://localai.io/docs/features/gpu-acceleration/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/gpu-acceleration/index.htmlThis page covers how to use LocalAI with GPU acceleration across different hardware vendors. For container image tags and registry details, see Container Images. For memory management with multiple GPU-accelerated models, see VRAM Management. Automatic Backend Detection When you install a model from the gallery (or a YAML file), LocalAI intelligently detects the required backend and your system’s capabilities, then downloads the correct version for you. Whether you’re running on a standard CPU, an NVIDIA GPU, an AMD GPU, or an Intel GPU, LocalAI handles it automatically.https://localai.io/docs/backends/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/backends/index.htmlLearn how to use, manage, and develop backends in LocalAIhttps://localai.io/docs/models/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/index.htmlThe model gallery is a curated collection of models configurations for LocalAI that enables one-click install of models directly from the LocalAI Web interface. A list of the models available can also be browsed at the Public LocalAI Gallery. LocalAI to ease out installations of models provide a way to preload models on start and downloading and installing them in runtime. You can install models manually by copying them over the models directory, or use the API or the Web interface to configure, download and verify the model assets for you.https://localai.io/docs/features/runtime-settings/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/runtime-settings/index.htmlLocalAI provides a web-based interface for managing application settings at runtime. These settings can be configured through the web UI and are automatically persisted to a configuration file, allowing changes to take effect immediately without requiring a restart. Accessing Runtime Settings Navigate to the Settings page from the management interface at http://localhost:8080/manage. The settings page provides a comprehensive interface for configuring various aspects of LocalAI.https://localai.io/docs/features/quantization/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/quantization/index.htmlLocalAI supports model quantization directly through the API and Web UI. Quantization converts HuggingFace models to GGUF format and compresses them to smaller sizes for efficient inference with llama.cpp. Note This feature is experimental and may change in future releases.https://localai.io/docs/features/tracing/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/tracing/index.htmlLocalAI can retain recent API exchanges and backend operations for inspection on the Traces page in the management interface. Enable tracing in runtime settings or with the existing tracing configuration. API and backend trace histories are persisted in separate directories below the configured data path. They are restored after a clean service restart, whether or not authentication is enabled. Each history remains independently bounded by tracing_max_items. When a history reaches that limit, LocalAI removes its oldest records from memory and disk. The existing clear actions on the Traces page remove both the in-memory history and its persisted records.https://localai.io/docs/features/fine-tuning/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/fine-tuning/index.htmlLocalAI supports fine-tuning LLMs directly through the API and Web UI. Fine-tuning is powered by pluggable backends that implement a generic gRPC interface, allowing support for different training frameworks and model types. Supported Backends Backend Domain GPU Required Training Methods Adapter Types trl LLM fine-tuning No (CPU or GPU) SFT, DPO, GRPO, RLOO, Reward, KTO, ORPO LoRA, Full Availability Fine-tuning is always enabled. When authentication is enabled, fine-tuning is a per-user feature (default OFF). Admins can enable it for specific users via the user management API.https://localai.io/docs/features/authentication/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/features/authentication/index.htmlLocalAI supports two authentication modes: legacy API key authentication (simple shared keys) and a full user authentication system with roles, sessions, OAuth, and per-user usage tracking. Legacy API Key Authentication The simplest way to protect your LocalAI instance is with API keys. Set one or more keys via environment variable or CLI flag: # Single key LOCALAI_API_KEY=sk-my-secret-key localai run # Multiple keys (comma-separated) LOCALAI_API_KEY=key1,key2,key3 localai run Clients provide the key via any of these methods: