Skip to main content
localai.io

Subsections of Concepts

Architecture

Note
The install script, the macOS app and the latest container image give version 4.12.1. The add-ons, Office, Sandbox, Pages and Team in these docs need a newer build.

LocalAI is an API written in Go. It follows the OpenAI API, so software that uses an OpenAI SDK can use LocalAI instead. It runs on consumer hardware. The inference runs in separate backends, many of them in C++. Backends are gRPC servers. You can build your own gRPC server, or point LocalAI at an external one that it then manages.

LocalAI uses a mixture of backends written in various languages (C++, Golang, Python, …). You can check the model compatibility table to learn about all the components of LocalAI.

How LocalAI works: clients call one API on a small core with chats, agents, an add-on host and an MCP client and server, which talks over gRPC to a separate backend process for each model, running on your own hardware.How LocalAI works: clients call one API on a small core with chats, agents, an add-on host and an MCP client and server, which talks over gRPC to a separate backend process for each model, running on your own hardware. How a request flows: the client calls the API, the model config picks a backend, the backend process runs the model over gRPC on your hardware, and tokens stream back.How a request flows: the client calls the API, the model config picks a backend, the backend process runs the model over gRPC on your hardware, and tokens stream back.

For the history of the project, read LocalAI, from March 2023 to now.

Users and sharing

Note
Add-ons need a development build or a release newer than 4.12.1.

LocalAI can run with no accounts or with accounts. This page explains what changes, and how sharing works.

Without accounts

When authentication is off, there is one user. That user can do everything, including the administrator tasks. You can protect the API with a key, but the key gives full access.

With accounts

Turn on user authentication to get accounts. See Authentication. There are two roles.

  • Admin: manages models, backends, settings, users and add-ons. The first user who signs in is an admin.
  • User: uses the inference endpoints and the web interface. A user cannot change server settings.

New people join in one of three ways, set by the registration mode: open (anyone), approval (an admin approves, and this is the default) or invite (an admin gives a single-use invite link, /invite/<code>). An admin creates invite links in Settings > Users and keys.

Features per user

An administrator switches parts of LocalAI on or off for each user. Some features are on by default for a new user. Others are off until an administrator turns them on.

Off by default for a new userOn by default
collections, localai_assistant, agents, triggers, connectors, mcp_jobs, pages, addons, fine_tuning, quantizationThe API features, such as chats, images, audio_speech and embeddings

Administrators always have every feature. To change a user, open Settings > Users and keys and edit the permissions of the user. A feature that is off hides its pages and refuses its routes.

What is yours

Each user has their own files, chats, saved agents, pages and add-on connections. Nobody else sees them. An administrator can read the saved agents, triggers and connectors of every user, switch them off and delete them, but not run them as that user.

Sharing

Nothing is shared by default. A person shares something they own, and they choose who gets it and what they can do.

What you shareRoles
A folder of your homeCan read, Can edit
A saved agentUse, View
An add-on resource, such as a Tables databaseViewer, Editor
A shared folder, made by an administratorCan read, Can edit

You can share with one person or with Everyone. A grant to Everyone also reaches accounts that are made later. A person can hold their own grant and the Everyone grant. Both add up.

The owner is never a grant. Only the owner shares, changes the grants or deletes the resource. Sharing is never a tool: no model, agent, trigger or add-on can share or take back access. Only your own browser session can change a grant.

Add-ons and users

An administrator installs an add-on for the whole instance. A user needs the addons feature to use it. Add-ons that connect to a service, such as a mailbox, keep a separate connection for each user, and an administrator approves the service client first. See the add-on user guide.