CLI Reference

Complete reference for all LocalAI command-line interface (CLI) parameters and environment variables.

For client-side text inference measurements, see Benchmark text models.

These options configure the LocalAI server process. To configure an individual model, see Model Configuration.

Note: All CLI flags can also be set via environment variables. Environment variables take precedence over CLI flags. See .env files for configuration file support.

Global Flags

Diagnostics flags

See Profiling and request diagnostics for sampling semantics, phase boundaries, and protected pod captures.

These options apply only to local-ai run, including distributed frontends. Diagnostics state is per-instance, not shared between replicas. Debug logging enables neither feature. The profiling listener is separate from the public API and accepts only numeric loopback addresses with explicit ports. --preload-backend-only validates these options but does not start profiling or sampling.

Sampling adds overhead. The process applies configured rates once and resets them to zero on shutdown. There is no runtime toggle, continuous capture, or public API route for diagnostics. The private listener has no authentication: other processes in the same network namespace can access profiles. Profiles can contain secrets; protect captured files. The /debug/pprof/cmdline endpoint is unavailable. Shutdown allows five seconds for cleanup, then force-closes profiling connections. Existing application initialization operations may not promptly observe cancellation. The signal callback still returns after its cleanup budget expires. If initialization fails without returning an application handle, this lifecycle cannot shut down resources created inside that failed initialization. Application shutdown has no context parameter; work that ignores cancellation can outlive the cleanup budget.

Storage Flags

| --models-config-file | | YAML file containing a list of model backend configs (alias: --config-file) | $LOCALAI_MODELS_CONFIG_FILE, $CONFIG_FILE | | --addon-host-ops | fine-tuning=finetune.,quantization=quantize.,sandbox=workspace.,sandbox=egress.,office=workspace.,web=workspace.,telegram=connector. | Add-ons that may call host operations, as a comma list of addon-id=operation-prefix. An add-on that is not listed gets no host operation channel; an empty value lists none. See Host operations | $LOCALAI_ADDON_HOST_OPS | | --addons-role | combined | How this process takes part in the add-on owner: combined (the node is its own owner), owner or frontend. See Turning the owner on or off | $LOCALAI_ADDONS_ROLE | | --addons-owner-address | | https://host:port of the add-on owner carrier, with no path. Required for the owner and frontend roles | $LOCALAI_ADDONS_OWNER_ADDRESS | | --addons-owner-listen | | owner role only: the address the carrier listens on. Empty listens on all interfaces, on the port of --addons-owner-address | $LOCALAI_ADDONS_OWNER_LISTEN | | --addons-carrier-dir | | Directory with the carrier key set from local-ai util addons-carrier-keys. Required for the owner and frontend roles | $LOCALAI_ADDONS_CARRIER_DIR | | --addons-enabled | true | Run the add-on owner. --addons-enabled=false turns add-ons off completely. See The addons feature gate | $LOCALAI_ADDONS_ENABLED | | --addons-ownership-mode | local-owner without authentication, named-user with it | named-user gives each account its own access to add-ons; local-owner makes every caller share one owner | $LOCALAI_ADDONS_OWNERSHIP_MODE | | --auto-upgrade-addons | false | Upgrade installed add-ons on their own when a newer version appears in their library | $LOCALAI_AUTO_UPGRADE_ADDONS | | --addons-official-library | true | Register the official add-on library on the first start | $LOCALAI_ADDONS_OFFICIAL_LIBRARY | | --addons-backend-library | true | Register the backend library on the first start | $LOCALAI_ADDONS_BACKEND_LIBRARY | | --addons-backend-max-bytes | 8589934592 | Largest compressed size, in bytes, of a backend image installed from a library | $LOCALAI_ADDONS_BACKEND_MAX_BYTES | | --max-concurrent-backend-operations | 1 | How many backend install, upgrade and delete operations run at once. 1 runs them one after the other. Operations on the same backend always run one at a time. In distributed mode it also bounds how many nodes one operation is sent to at the same time | $LOCALAI_MAX_CONCURRENT_BACKEND_OPERATIONS | | --artifact-download-concurrency | 1 | How many files of a model artifact to download at once. 1 downloads sequentially. Raising it helps artifacts split into many files on a fast link, at the cost of more concurrent load on the models volume. Whole files only — a single file is never split, so resume and per-file checksum verification are unaffected | $LOCALAI_ARTIFACT_DOWNLOAD_CONCURRENCY |

Backend processes receive a private scratch directory through TMPDIR, TMP, and TEMP. LocalAI removes that directory when the backend exits and removes abandoned directories left by a LocalAI crash before starting another backend. Set $LOCALAI_BACKEND_TEMP_DIR to choose their base volume. LocalAI always appends localai-UID/backend-runtime; the default base is TMPDIR.

Backend Flags

For more information on VRAM management, see VRAM and Memory Management.

Models Flags

Note: You can also pass model configuration URLs as positional arguments: local-ai run MODEL_URL1 MODEL_URL2 ...

Performance Flags

Default thread count

When the server thread setting is unset or zero, LocalAI selects the smaller of the physical core count and Go’s current GOMAXPROCS value at startup. The result is at least one thread.

By default, Go accounts for CPU affinity and Linux container CPU quotas. For example, docker run --cpus=2 limits the automatic default to two threads. Go normally applies a minimum of two for CPU quotas, so --cpus=1 can still produce two threads. Set --threads=1 if you need one inference thread.

An explicit GOMAXPROCS value overrides Go’s automatic CPU detection. A nonzero server thread setting from the CLI, environment, or saved runtime settings bypasses this automatic default.

API Flags

Collections and Assistant Flags

The collections service is the knowledge base that the chats search. The old LOCALAI_AGENT_POOL_* names still work and warn once; see Moving from the old chat and agents.

LOCALAI_DISABLE_AGENTS is now a name for LOCALAI_DISABLE_COLLECTIONS; nothing switches the chats themselves off except the chats permission of each user.

Hardening Flags

Authentication Flags

See Authentication & Authorization for full documentation.

Chats Flags

See Chats, Agent tasks and jobs and Chat workers.

Collections Flags

See Collections. The old --agent-pool-* names still work and log a deprecation warning.

Chat Flags

Use local-ai chat to run the built-in terminal agent against a LocalAI server. The agent can run shell commands, delegate to sub-agents, and use MCP tools. Read-only calls run on their own; everything else goes through an approval prompt you answer. See Terminal agent for the full feature page.

The agent’s own subcommands are reached by passing them through. LocalAI’s flags must come first, because everything after the first positional argument is forwarded verbatim:

local-ai chat plugin install https://github.com/user/plugin
local-ai chat skill list
local-ai chat mcp add my-server -- npx -y @modelcontextprotocol/server-filesystem /tmp
local-ai chat --config-dir /srv/agent plugin list   # flags first

Summon the agent from any shell prompt with Ctrl+Space:

echo 'eval "$(local-ai chat --init zsh)"' >> ~/.zshrc

Inside a session:

• /models lists the models the server offers, marking the current one.
• /model <name> switches model, keeping the conversation.
• /compact summarizes the conversation so far to free up context.
• /skill <name>, /agent <name> <task>, /attach <file>, /goal <text>.

If no server is reachable, the agent offers to start one for the session (interactive terminals only) and otherwise points you at local-ai run.

Piped use needs --cli, and exits 0 when it answers:

echo "what is 2+2" | local-ai chat --cli

Read-only tools still run in a piped session. A tool call that is not read-only has nobody to approve it, so it is denied and the session exits 3, a code chosen to be distinct from the 1 a failure reports.

P2P Flags

Library sources

local-ai library manages the sources LocalAI reads add-ons, backends and models from. The sources are stored under the data path (addons/sources.json), the file the server reads, so the commands work with the server stopped.

All local-ai library commands accept --official-libraries (LOCALAI_OFFICIAL_LIBRARIES, default true). With --official-libraries=false the list shows the shipped default sources as off. The same switch is the runtime setting official_libraries and the toggle “Use official libraries” in Settings. See Library sources.

When two enabled sources list the same entry (same kind and id), your own source wins over a shipped default. If two of your own sources list it, the entry is refused and the message names both sources: disable or remove one of them. The --galleries and --backend-galleries flags keep working: each gallery they name is added as a source of your own (kind model or backend, format gallery) at startup, unless it is already a source.

Other Commands

LocalAI supports several subcommands beyond run:

• local-ai models - Manage LocalAI models and definitions
• local-ai backends - Manage LocalAI backends and definitions
• local-ai chat - Run the built-in terminal agent against a LocalAI server
• local-ai tts - Convert text to speech
• local-ai sound-generation - Generate audio files from text or audio
• local-ai transcript - Convert audio to text
• local-ai mcp-server - Run the LocalAI admin and client tool surface as a stdio MCP server (controls a remote LocalAI instance over HTTP). --files-dir (LOCALAI_MCP_FILES_DIR) is the folder where the client tools read and write files; it has no default, and without it the client tools that read or write files are not registered. --read-only also leaves out the client tools that write files or make content (generate_*, speak, embed, voice_profile create and delete)
• local-ai worker - Start a worker for distributed mode (generic, backend-agnostic)
• local-ai p2p-worker - Run workers to distribute workload via p2p (llama.cpp-only)
• local-ai agent-worker - Start a worker for distributed mode (serves MCP tool calls via NATS)
• local-ai util - Utility commands
• local-ai explorer - Run P2P explorer
• local-ai federated - Run LocalAI in federated mode
• local-ai completion - Generate shell completion scripts for bash, zsh, or fish

Use local-ai <command> --help for more information on each command.

Add-on carrier keys

local-ai util addons-carrier-keys makes the key set that a separate add-on owner and its frontends share. See Running the owner as its own workload for how the set is used.

local-ai util addons-carrier-keys --out ./carrier-keys --owner-host addons.owner.example

The command prints which files go to the owner and which go to the frontends. It writes the files with mode 0600 in a directory with mode 0700, and it does not keep the CA key.

Examples

Basic Usage

./local-ai run

./local-ai run --models-path /path/to/models --address :9090

./local-ai run --f16

Environment Variables

export LOCALAI_MODELS_PATH=/path/to/models
export LOCALAI_ADDRESS=:9090
export LOCALAI_F16=true
./local-ai run

Advanced Configuration

./local-ai run \
  --models model1.yaml model2.yaml \
  --enable-watchdog-idle \
  --watchdog-idle-timeout=10m \
  --p2p \
  --federated

Related Documentation

• See Advanced Usage for configuration examples
• See VRAM and Memory Management for memory management options