Features

LocalAI provides a comprehensive set of features for running AI models locally. The pages in this section are grouped by capability, and the left navigation is ordered to match these groups.

Text

• Text Generation - Generate text with GPT-compatible models using various backends.
• OpenAI Functions and tools - Use function calling and the tools API with local models.
• Constrained Grammars - Control model output format with BNF grammars.
• Interleaved thinking - Reasoning models that interleave thought and output.
• Model aliases - Expose one model under several names.
• API discovery - How LocalAI advertises its capability surface.

Agents

• Agents - Autonomous AI agents with tools, knowledge base, and skills.
• Agent Actions - The catalog of actions an agent can run.
• Model Context Protocol (MCP) - Give a model external tools over MCP.
• LocalAI Assistant - Chat to administer your LocalAI instance.

Audio

• Audio to text - Transcribe audio to text.
• Text to audio - Generate speech, music, and sound effects from text.
• Audio classification - Classify sounds and audio events.
• Audio diarization - Separate speakers in an audio stream.
• Audio transform - Transform and process audio.
• Voice Activity Detection - Detect speech segments in audio data.
• Voice recognition - Identify speakers by voice.
• Realtime API - Low-latency multi-modal conversations (voice and text) over WebSocket.

Vision

• GPT Vision - Analyze and understand images with vision-language models.
• Object detection - Detect and locate objects in images.
• Face recognition - Recognize faces in images.

Image and Video

• Image Generation - Create images with Stable Diffusion and other diffusion models.
• Video Generation - Generate videos from text, image, or audio conditioning, including LongCat and avatar workflows.

Retrieval

• Embeddings - Generate vector embeddings for semantic search and RAG applications.
• Reranker - Improve retrieval accuracy with cross-encoder models.
• Stores - Vector similarity search for embeddings.

Distributed and acceleration

• Distributed inference - Scale inference across multiple nodes (P2P federation or production distributed mode).
• Distributed mode - The production distributed deployment.
• MLX distributed - Distributed inference on Apple Silicon with MLX.
• GPU Acceleration - Optimize performance with GPU support.

Platform and model management

• Backends - Available backends and how to manage them.
• Model Gallery - Browse and install pre-configured models.
• Runtime Settings - Configure application settings via the web UI without restarting.
• Quantization - Quantize models for smaller memory footprint.
• Fine-tuning - Fine-tune models on your own data.
• Authentication - Protect the API with keys.

For operator-facing runtime, proxy, and monitoring concerns (middleware, cloud and MITM proxies, backend monitor), see the Operations section.

Getting Started

To start using these features, make sure you have LocalAI installed and have downloaded some models. Then explore the feature pages above to learn how to use each capability.