CLI Reference

Complete reference for all LocalAI command-line interface (CLI) parameters and environment variables.

For client-side text inference measurements, see Benchmark text models.

These options configure the LocalAI server process. To configure an individual model, see Model Configuration.

Note: All CLI flags can also be set via environment variables. Environment variables take precedence over CLI flags. See .env files for configuration file support.

Global Flags

Diagnostics flags

See Profiling and request diagnostics for sampling semantics, phase boundaries, and protected pod captures.

These options apply only to local-ai run, including distributed frontends. Diagnostics state is per-instance, not shared between replicas. Debug logging enables neither feature. The profiling listener is separate from the public API and accepts only numeric loopback addresses with explicit ports. --preload-backend-only validates these options but does not start profiling or sampling.

Sampling adds overhead. The process applies configured rates once and resets them to zero on shutdown. There is no runtime toggle, continuous capture, or public API route for diagnostics. The private listener has no authentication: other processes in the same network namespace can access profiles. Profiles can contain secrets; protect captured files. The /debug/pprof/cmdline endpoint is unavailable. Shutdown allows five seconds for cleanup, then force-closes profiling connections. Existing application initialization operations may not promptly observe cancellation. The signal callback still returns after its cleanup budget expires. If initialization fails without returning an application handle, this lifecycle cannot shut down resources created inside that failed initialization. Application shutdown has no context parameter; work that ignores cancellation can outlive the cleanup budget.

Storage Flags

| --models-config-file | | YAML file containing a list of model backend configs (alias: --config-file) | $LOCALAI_MODELS_CONFIG_FILE, $CONFIG_FILE | | --artifact-download-concurrency | 1 | How many files of a model artifact to download at once. 1 downloads sequentially. Raising it helps artifacts split into many files on a fast link, at the cost of more concurrent load on the models volume. Whole files only — a single file is never split, so resume and per-file checksum verification are unaffected | $LOCALAI_ARTIFACT_DOWNLOAD_CONCURRENCY |

Backend processes receive a private scratch directory through TMPDIR, TMP, and TEMP. LocalAI removes that directory when the backend exits and removes abandoned directories left by a LocalAI crash before starting another backend. Set $LOCALAI_BACKEND_TEMP_DIR to choose their base volume. LocalAI always appends localai-UID/backend-runtime; the default base is TMPDIR.

Backend Flags

For more information on VRAM management, see VRAM and Memory Management.

Models Flags

Note: You can also pass model configuration URLs as positional arguments: local-ai run MODEL_URL1 MODEL_URL2 ...

Performance Flags

API Flags

Hardening Flags

Authentication Flags

See Authentication & Authorization for full documentation.

Chat Flags

Use local-ai chat to run the built-in terminal agent against a LocalAI server. The agent can run shell commands, delegate to sub-agents, and use MCP tools. Read-only calls run on their own; everything else goes through an approval prompt you answer. See Terminal agent for the full feature page.

The agent’s own subcommands are reached by passing them through. LocalAI’s flags must come first, because everything after the first positional argument is forwarded verbatim:

local-ai chat plugin install https://github.com/user/plugin
local-ai chat skill list
local-ai chat mcp add my-server -- npx -y @modelcontextprotocol/server-filesystem /tmp
local-ai chat --config-dir /srv/agent plugin list   # flags first

Summon the agent from any shell prompt with Ctrl+Space:

echo 'eval "$(local-ai chat --init zsh)"' >> ~/.zshrc

Inside a session:

• /models lists the models the server offers, marking the current one.
• /model <name> switches model, keeping the conversation.
• /compact summarizes the conversation so far to free up context.
• /skill <name>, /agent <name> <task>, /attach <file>, /goal <text>.

If no server is reachable, the agent offers to start one for the session (interactive terminals only) and otherwise points you at local-ai run.

Piped use needs --cli, and exits 0 when it answers:

echo "what is 2+2" | local-ai chat --cli

Read-only tools still run in a piped session. A tool call that is not read-only has nobody to approve it, so it is denied and the session exits 3, a code chosen to be distinct from the 1 a failure reports.

P2P Flags

Other Commands

LocalAI supports several subcommands beyond run:

• local-ai models - Manage LocalAI models and definitions
• local-ai backends - Manage LocalAI backends and definitions
• local-ai chat - Run the built-in terminal agent against a LocalAI server
• local-ai tts - Convert text to speech
• local-ai sound-generation - Generate audio files from text or audio
• local-ai transcript - Convert audio to text
• local-ai agent - Run agents standalone without the full LocalAI server
• local-ai mcp-server - Run the LocalAI admin tool surface as a stdio MCP server (controls a remote LocalAI instance over HTTP)
• local-ai worker - Start a worker for distributed mode (generic, backend-agnostic)
• local-ai p2p-worker - Run workers to distribute workload via p2p (llama.cpp-only)
• local-ai agent-worker - Start an agent worker for distributed mode (executes agent chats via NATS)
• local-ai util - Utility commands
• local-ai explorer - Run P2P explorer
• local-ai federated - Run LocalAI in federated mode
• local-ai completion - Generate shell completion scripts for bash, zsh, or fish

Use local-ai <command> --help for more information on each command.

Examples

Basic Usage

./local-ai run

./local-ai run --models-path /path/to/models --address :9090

./local-ai run --f16

Environment Variables

export LOCALAI_MODELS_PATH=/path/to/models
export LOCALAI_ADDRESS=:9090
export LOCALAI_F16=true
./local-ai run

Advanced Configuration

./local-ai run \
  --models model1.yaml model2.yaml \
  --enable-watchdog-idle \
  --watchdog-idle-timeout=10m \
  --p2p \
  --federated

Related Documentation

• See Advanced Usage for configuration examples
• See VRAM and Memory Management for memory management options