Skip to main content

LocalAI Assistant

Info

Looking for something else? LocalAI has four agentic surfaces that are easy to confuse:

  • LocalAI Assistant (this page): an admin chat modality for administering LocalAI itself: install models, manage backends and edit configs by chatting.
  • Terminal agent: an agent in your own shell. local-ai chat reads your files and runs commands on your machine, behind an approval gate you control.
  • Agents: autonomous agents you build and run inside LocalAI, that reason, use tools, and act on their own.
  • MCP: the Model Context Protocol itself, the way a model is handed external tools. It sits underneath the other three rather than beside them.

LocalAI Assistant is an admin-only chat modality. When enabled on a chat session, the conversation is wired to an in-process MCP server that exposes LocalAI’s own admin/management surface as tools. You can install models, manage backends, edit model configs and check system status by chatting, with no REST calls or YAML edits.

The same MCP server is published as a Go package and can also be served over stdio to control a remote LocalAI instance from outside (e.g. from a desktop MCP host, Cursor, or mcphost).

Enabling the assistant in chat

Open the chat UI as an admin user and pick a chat-capable model with the model chip above the message box. Open Chat settings (the sliders button in the header) and turn on Manage mode, or type /assistant in the message box. A shield icon appears next to the chat title while the mode is on. In a new chat in Manage mode, starter chips (“What is installed?”, “Install a chat model”, “Show system status”, “Update a backend”) help you get going.

The home page shows a one-line Manage LocalAI by chatting prompt that opens a fresh chat already in Manage mode. You can dismiss it. The same action stays available as /assistant in the command bar and in the Library row.

Once on, try:

Install Qwen 3 chat

The assistant searches the gallery, lists candidates, asks you to pick, summarises the install, and waits for your confirmation before calling install_model. While the install runs, it polls progress and reports the outcome.

Disabling the feature

Either toggle it off in Settings → LocalAI Assistant (takes effect without restart), or hard-disable at startup:

LOCALAI_DISABLE_ASSISTANT=true local-ai run

When disabled, the chat handler refuses requests with metadata.localai_assistant=true and returns 503. The Manage toggle is hidden in the UI.

Security model

  • The chat toggle is hidden for non-admin users.
  • The chat handler re-checks admin role at request time even when auth is configured to skip the assistant feature gate (defense in depth).
  • The MCP server itself is in-process - there is no localhost loopback, no synthetic API key, and no extra TCP socket.
  • Mutating tools (install_model, delete_model, edit_model_config, upgrade_backend, …) are guarded by a system-prompt rule that requires the LLM to confirm the action with the user before calling them. There is no separate code-side preview/apply step.

Standalone stdio MCP server

You can run the same admin tool surface as a stdio MCP server pointed at any LocalAI HTTP API:

local-ai mcp-server --target http://remote.localai:8080 --api-key <admin-key>
# read-only mode - skips registration of every mutating tool
local-ai mcp-server --target http://remote.localai:8080 --read-only

Useful for hooking LocalAI admin into Claude Desktop, Cursor, or any MCP host. The tool catalog is identical to the in-process variant.

Tool catalog

Read-only: gallery_search, list_installed_models, list_galleries, list_backends, list_known_backends, get_job_status, get_model_config, vram_estimate, system_info, list_nodes.

Mutating (require user confirmation per the assistant’s safety prompt): install_model, import_model_uri, delete_model, install_backend, upgrade_backend, edit_model_config, reload_models, toggle_model_state, toggle_model_pinned.

Adding new tools or skills

The MCP server lives at pkg/mcp/localaitools/. Tools are registered in tools_*.go; skill prompts (the markdown the LLM sees) are embedded from prompts/. To add a new admin tool, add a method to the LocalAIClient interface plus implementations in inproc/ and httpapi/, then register the tool and add or update a skill prompt. See .agents/localai-assistant-mcp.md for the full contributor checklist.