API Discovery & Instructions

LocalAI exposes a set of discovery endpoints that let external agents, coding assistants, and automation tools programmatically learn what the instance can do and how to control it - without reading documentation ahead of time.

Quick start

# 1. Discover what's available
curl http://localhost:8080/.well-known/localai.json

# 2. Browse instruction areas
curl http://localhost:8080/api/instructions

# 3. Get an API guide for a specific instruction
curl http://localhost:8080/api/instructions/config-management

Authentication and discovery

You can read these discovery surfaces without authentication. This remains true when LocalAI uses database authentication or legacy API keys:

• GET /.well-known/localai.json
• GET /api/instructions
• GET /api/instructions/{name}
• Swagger GET requests at /swagger and under /swagger/

Discovery does not grant access to the endpoints it advertises. Those endpoints require credentials unless the authentication guide lists them as a public discovery or bootstrap route. For example, GET /version requires credentials even though the well-known response includes the instance version.

Well-Known Discovery Endpoint

GET /.well-known/localai.json

Returns the instance version, all available endpoint URLs (flat and categorized), and runtime capabilities.

Example response (abbreviated):

{
  "version": "v2.28.0",
  "endpoints": {
    "chat_completions": "/v1/chat/completions",
    "models": "/v1/models",
    "models_capabilities": "/v1/models/capabilities",
    "config_metadata": "/api/models/config-metadata",
    "instructions": "/api/instructions",
    "swagger": "/swagger/index.html"
  },
  "endpoint_groups": {
    "openai_compatible": { "chat_completions": "/v1/chat/completions", "..." : "..." },
    "config_management": { "config_metadata": "/api/models/config-metadata", "..." : "..." },
    "model_management": { "..." : "..." },
    "monitoring": { "..." : "..." }
  },
  "capabilities": {
    "config_metadata": true,
    "config_patch": true,
    "vram_estimate": true,
    "model_storage": true,
    "backend_install": true,
    "mcp": true,
    "chats": true,
    "agent_jobs": true,
    "collections": true,
    "p2p": false
  }
}

The capabilities object reflects the current runtime configuration - for example, mcp is only true if MCP is enabled, chats and agent_jobs are false with --disable-chats, collections is false with --disable-collections, and backend_install is false when the server runs with --disable-backend-install (backend install and upgrade endpoints then answer 403).

Instructions API

Instructions are curated groups of related API endpoints. Each instruction maps to one or more Swagger tags and provides a focused, LLM-readable guide.

List all instructions

GET /api/instructions

curl http://localhost:8080/api/instructions

Returns a compact list of instruction areas:

{
  "instructions": [
    {
      "name": "chat-inference",
      "description": "OpenAI-compatible chat completions, text completions, and embeddings",
      "tags": ["inference", "embeddings"],
      "url": "/api/instructions/chat-inference"
    },
    {
      "name": "config-management",
      "description": "Discover, read, and modify model configuration fields with VRAM estimation",
      "tags": ["config"],
      "url": "/api/instructions/config-management"
    }
  ],
  "hint": "Fetch GET {url} for a markdown API guide. Add ?format=json for a raw OpenAPI fragment."
}

Available instructions:

Get an instruction guide

GET /api/instructions/:name

By default, returns a markdown guide suitable for LLMs and humans:

curl http://localhost:8080/api/instructions/config-management

Add ?format=json to get a raw OpenAPI fragment (filtered Swagger spec with only the relevant paths and definitions):

curl http://localhost:8080/api/instructions/config-management?format=json

Model Capabilities

GET /v1/models/capabilities

An additive, LocalAI-specific superset of /v1/models. It returns the same set of models but enriches each entry with the capabilities the model supports and the input/output modalities it accepts and produces. Use it to decide, before sending a request, whether a given model can take an image, audio, or video attachment directly - or whether the input needs converting/transcribing first.

The reported context_size uses a positive model-level value first. If the model does not set context_size, it uses Settings → Performance → Default Context Size when positive. Otherwise, it uses the backend fallback of 4096 tokens. For llama.cpp with separate KV caches, the reported value accounts for the number of parallel slots.

An entry has a description when the model configuration sets one. With authentication on, a user whose account has a model allow-list sees only the models on it.

Because it is purely additive, clients that only understand /v1/models keep working unchanged; they simply never call this route.

curl http://localhost:8080/v1/models/capabilities

{
  "object": "list",
  "data": [
    {
      "id": "qwen2.5-omni",
      "object": "model",
      "capabilities": ["chat", "vision", "tools"],
      "description": "A model that reads images and audio.",
      "input_modalities": ["text", "image", "audio"],
      "output_modalities": ["text"]
    },
    {
      "id": "parakeet",
      "object": "model",
      "capabilities": ["transcript"],
      "input_modalities": ["audio"],
      "output_modalities": ["text"]
    }
  ]
}

• capabilities - canonical usecase strings (e.g. chat, vision, transcript, tts, embeddings, image, video) plus the modifiers tools and thinking.
• input_modalities / output_modalities - the subsets of {text, image, audio, video} the model accepts and produces. LocalAI combines usecase-based inference, backend settings such as vLLM limit_mm_per_prompt, and explicit model-level known_input_modalities / known_output_modalities. The explicit fields cover distinctions a usecase cannot express, such as a video model that also accepts speech.

The same query parameters as /v1/models are honored (filter, excludeConfigured), and the same per-user model allowlist is applied when authentication is enabled.

Configuration Management APIs

These endpoints let agents discover model configuration fields, read current settings, modify them, and estimate VRAM usage.

Config metadata

GET /api/models/config-metadata

Returns structured metadata for all model configuration fields, organized by section. Each field includes its YAML path, Go type, UI type, label, description, default value, validation constraints, and available options.

# All fields
curl http://localhost:8080/api/models/config-metadata

# Filter by section
curl http://localhost:8080/api/models/config-metadata?section=parameters

Autocomplete values

GET /api/models/config-metadata/autocomplete/:provider

Returns runtime values for dynamic fields. Providers include backends, models, models:chat, models:tts, models:transcript, models:vad.

# List available backends
curl http://localhost:8080/api/models/config-metadata/autocomplete/backends

# List chat-capable models
curl http://localhost:8080/api/models/config-metadata/autocomplete/models:chat

Read model config

GET /api/models/config-json/:name

Returns the full model configuration as JSON:

curl http://localhost:8080/api/models/config-json/my-model

Update model config

PATCH /api/models/config-json/:name

Deep-merges a JSON patch into the existing model configuration. Only include the fields you want to change:

curl -X PATCH http://localhost:8080/api/models/config-json/my-model \
  -H "Content-Type: application/json" \
  -d '{"context_size": 16384, "gpu_layers": 40}'

The endpoint validates the merged config and writes it to disk as YAML.

Model storage report

GET /api/models/storage (admin)

Reports the on-disk footprint of the installed model configurations: per-model sizes with the portion shared with other models split out, every referenced file with the models referencing it, and references whose files are missing from disk.

curl http://localhost:8080/api/models/storage

VRAM estimation

POST /api/models/vram-estimate

Estimates VRAM usage for an installed model based on its weight files, context size, and GPU layer offloading:

curl -X POST http://localhost:8080/api/models/vram-estimate \
  -H "Content-Type: application/json" \
  -d '{"model": "my-model", "context_size": 8192}'

{
  "sizeBytes": 4368438272,
  "sizeDisplay": "4.4 GB",
  "vramBytes": 6123456789,
  "vramDisplay": "6.1 GB",
  "context_note": "Estimate used default context_size=8192. The model's trained maximum context is 131072; VRAM usage will be higher at larger context sizes.",
  "model_max_context": 131072
}

Optional parameters: gpu_layers (number of layers to offload, 0 = all), kv_quant_bits (KV cache quantization, 0 = fp16).

Integration guide

A recommended workflow for agent/tool builders:

1. Discover: Fetch /.well-known/localai.json to learn available endpoints and capabilities.
2. Browse instructions: Fetch /api/instructions for an overview of instruction areas.
3. Deep dive: Fetch /api/instructions/{name} for a markdown API guide on a specific area.
4. Authenticate: Obtain credentials before you call an advertised protected endpoint.
5. Explore config: Use admin credentials with /api/models/config-metadata to understand configuration fields.
6. Interact: Use credentials with inference endpoints and configuration APIs.

Swagger UI

The full interactive API documentation is available without authentication at /swagger/index.html. Requests sent to protected endpoints from Swagger still require credentials.