LocalAI documentation

LocalAI is the open source AI runtime: a small core that speaks the OpenAI and Anthropic APIs, with each inference backend added only when a model needs it. It runs text, vision, speech, sound, images, video, embeddings, reranking, and autonomous agents on hardware you control, from a CPU laptop to a distributed GPU cluster.

New here? Read the Overview for what LocalAI is and how the pieces fit together, then follow the Quickstart.

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest

Sections

  • Getting started - install LocalAI, run your first model, call the API, and fix the common startup problems.
  • Features - every capability, grouped by modality: text, agents, audio, vision, image and video, retrieval, distributed inference, and model management.
  • Advanced - model configuration, VRAM management, reverse proxies and TLS, and the rest of the fine-grained control surface.
  • Operations - running and governing an instance: middleware, cloud and MITM proxies, backend monitoring.
  • Reference - architecture, CLI flags, the compatibility table, API and runtime errors, system info, and binaries.
  • FAQ - short answers to the questions that come up most often.

Also useful

  • Integrations - projects and tools built on top of LocalAI.
  • News - where release notes live.
  • Model gallery - browse the models you can install with one click.
  • GitHub and Discord - report an issue or ask a question.