Build LocalAI

Build

LocalAI can be built as a container image or as a single, portable binary. Note that some model architectures might require Python libraries, which are not included in the binary.

LocalAI’s extensible architecture allows you to add your own backends, which can be written in any language, and as such the container images contains also the Python dependencies to run all the available backends (for example, in order to run backends like Diffusers that allows to generate images and videos from text).

This section contains instructions on how to build LocalAI from source.

Build LocalAI locally

Requirements

In order to build LocalAI locally, you need the following requirements:

• Golang >= 1.21
• GCC
• GRPC

To install the dependencies follow the instructions below:

brew install go protobuf protoc-gen-go protoc-gen-go-grpc wget

apt install golang make protobuf-compiler-grpc

go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af

make build

Build

To build LocalAI with make:

git clone https://github.com/go-skynet/LocalAI
cd LocalAI
make build

This should produce the binary local-ai

Container image

Requirements:

• Docker or podman, or a container engine

In order to build the LocalAI container image locally you can use docker, for example:

docker build -t localai .
docker run localai

Example: Build on mac

Building on Mac (M1, M2 or M3) works, but you may need to install some prerequisites using brew.

The below has been tested by one mac user and found to work. Note that this doesn’t use Docker to run the server:

Install xcode from the Apps Store (needed for metalkit)

brew install abseil cmake go grpc protobuf wget protoc-gen-go protoc-gen-go-grpc

git clone https://github.com/go-skynet/LocalAI.git

cd LocalAI

make build

wget https://huggingface.co/TheBloke/phi-2-GGUF/resolve/main/phi-2.Q2_K.gguf -O models/phi-2.Q2_K

cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/phi-2.Q2_K.tmpl

./local-ai backends install llama-cpp

./local-ai --models-path=./models/ --debug=true

curl http://localhost:8080/v1/models

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
     "model": "phi-2.Q2_K",
     "messages": [{"role": "user", "content": "How are you?"}],
     "temperature": 0.9 
   }'

Troubleshooting mac

• If you encounter errors regarding a missing utility metal, install Xcode from the App Store.

• After the installation of Xcode, if you receive a xcrun error 'xcrun: error: unable to find utility "metal", not a developer tool or in PATH'. You might have installed the Xcode command line tools before installing Xcode, the former one is pointing to an incomplete SDK.

xcode-select --print-path

sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer

• If completions are slow, ensure that gpu-layers in your model yaml matches the number of layers from the model in use (or simply use a high number such as 256).

• If you get a compile error: error: only virtual member functions can be marked 'final', reinstall all the necessary brew packages, clean the build, and try again.

brew reinstall go grpc protobuf wget

make clean

make build

Build backends

LocalAI have several backends available for installation in the backend gallery. The backends can be also built by source. As backends might vary from language and dependencies that they require, the documentation will provide generic guidance for few of the backends, which can be applied with some slight modifications also to the others.

Manually

Typically each backend include a Makefile which allow to package the backend.

In the LocalAI repository, for instance you can build a backend by doing:

git clone https://github.com/go-skynet/LocalAI.git

make -C LocalAI/backend/python/vllm

With Docker

Building with docker is simpler as abstracts away all the requirement, and focuses on building the final OCI images that are available in the gallery. This allows for instance also to build locally a backend and install it with LocalAI. You can refer to Backends for general guidance on how to install and develop backends.

In the LocalAI repository, you can build a backend by doing:

git clone https://github.com/go-skynet/LocalAI.git

make docker-build-<backend-name>

Note that make is only by convenience, in reality it just runs a simple docker command as:

docker build --build-arg BUILD_TYPE=$(BUILD_TYPE) --build-arg BASE_IMAGE=$(BASE_IMAGE) -t local-ai-backend:<backend-name> -f LocalAI/backend/Dockerfile.golang --build-arg BACKEND=<backend-name> .               

Note:

• BUILD_TYPE can be either: cublas, hipblas, sycl_f16, sycl_f32, metal.
• BASE_IMAGE is tested on ubuntu:24.04 (and defaults to it) and quay.io/go-skynet/intel-oneapi-base:latest for intel/sycl

AI Coding Assistants

This document provides guidance for AI tools and developers using AI assistance when contributing to LocalAI.

LocalAI follows the same guidelines as the Linux kernel project for AI-assisted contributions. See the upstream policy here: https://docs.kernel.org/process/coding-assistants.html. The rules below mirror that policy, adapted to LocalAI’s license and project layout.

AI tools helping with LocalAI development should follow the standard project development process:

• CONTRIBUTING.md - development workflow, commit conventions, and PR guidelines
• AGENTS.md - the agent entry point with links to all detailed topic guides
• .agents/ai-coding-assistants.md - the full policy source of truth

Licensing and Legal Requirements

All contributions must comply with LocalAI’s licensing requirements:

• LocalAI is licensed under the MIT License
• New source files should use the SPDX license identifier MIT where applicable to the file type
• Contributions must be compatible with the MIT License and must not introduce code under incompatible licenses (e.g., GPL) without an explicit discussion with maintainers

Signed-off-by and Developer Certificate of Origin

AI agents MUST NOT add Signed-off-by tags. Only humans can legally certify the Developer Certificate of Origin (DCO). The human submitter is responsible for:

• Reviewing all AI-generated code
• Ensuring compliance with licensing requirements
• Adding their own Signed-off-by tag (when the project requires DCO) to certify the contribution
• Taking full responsibility for the contribution

AI agents MUST NOT add Co-Authored-By trailers for themselves either. A human reviewer owns the contribution; the AI’s involvement is recorded via Assisted-by (see below).

Exception: automation operated by a maintainer

The rule above addresses the common case, an AI assistant helping a human contributor who then signs off. It does not fit automation that a maintainer runs themselves, which opens pull requests with no human submitter to sign. Applied literally there, nothing ever signs and the DCO check blocks the pull request permanently.

A maintainer-operated bot MUST therefore add a Signed-off-by trailer naming the maintainer who operates it, not the bot and not the model:

Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

This is not the AI certifying the DCO. The maintainer is, exactly as they do for a commit they typed by hand: they configured the automation, they own its output, and they take responsibility for it when they merge it. The Assisted-by trailer still records that a model produced the code, so the provenance trail is unchanged.

The exception is narrow and does not widen the rule for anyone else:

• It applies only to automation a LocalAI maintainer operates and whose output that maintainer reviews before merge.
• The sign-off names a real person who accepts DCO responsibility.
• An AI assistant helping an outside contributor still MUST NOT sign off. That contributor adds their own trailer.
• A bot MUST NOT sign off on behalf of anyone other than its operator, and MUST NOT add a trailer for a contributor whose branch it pushes to. If automation contributes to someone else’s branch, it leaves the sign-off to that contributor.

Attribution

When AI tools contribute to LocalAI development, proper attribution helps track the evolving role of AI in the development process. Contributions should include an Assisted-by tag in the commit message trailer in the following format:

Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]

Where:

• AGENT_NAME - name of the AI tool or framework (e.g., Claude, Copilot, Cursor)
• MODEL_VERSION - specific model version used (e.g., claude-opus-4-7, gpt-5)
• [TOOL1] [TOOL2] - optional specialized analysis tools invoked by the agent (e.g., golangci-lint, staticcheck, go vet)

Basic development tools (git, go, make, editors) should not be listed.

Example

fix(llama-cpp): handle empty tool call arguments

Previously the parser panicked when the model returned a tool call with
an empty arguments object. Fall back to an empty JSON object in that
case so downstream consumers receive a valid payload.

Assisted-by: Claude:claude-opus-4-7 golangci-lint
Signed-off-by: Jane Developer <jane@example.com>

Scope and Responsibility

Using an AI assistant does not reduce the contributor’s responsibility. The human submitter must:

• Understand every line that lands in the PR
• Verify that generated code compiles, passes tests, and follows the project style
• Confirm that any referenced APIs, flags, or file paths actually exist in the current tree (AI models may hallucinate identifiers)
• Not submit AI output verbatim without review

Reviewers may ask for clarification on any change regardless of how it was produced. “An AI wrote it” is not an acceptable answer to a design question.

Docs style guide

Write

• Use short sentences. One idea in each sentence. Use the active voice and the present tense.
• Use digits for numbers. Write the product name as “LocalAI”.
• Say what the code does. Do not promise features that are not released. Do not write “coming soon” or “roadmap”.
• Do not use marketing words, such as “powerful”, “seamless” or “unlock”. Do not write “any model”. Name the models that run, or say “open models”. The product line is “Your own AI, on your own hardware. Ready to work for you.”
• Do not claim that data stays on the machine. Text can leave it when you use a cloud model, or an add-on that calls a service. Say which.
• Mark an add-on as an add-on, and say when it is in the development channel of the library.
• Do not write internal host names, addresses, local paths or names of people.

Page template

1. The goal in one or two sentences.
2. What you need.
3. Numbered steps.
4. What you should see.
5. Troubleshooting, as a table of what you see and what to do.
6. Next steps.

A page answers one task. Move a page over 400 lines to Reference, or split it.

Front matter

+++
title = "Page title"
description = "One sentence."
weight = 10
+++

The URL follows the file path. Do not set url. When you move a page, add its old URL to aliases and to docs/redirects.txt.

Links and images

• Link to other pages with {{% relref "/section/page" %}}.
• Give every image alt text. Check that the image shows in the light and the dark variant.
• Images that you link from a page start with /docs/images/.
• Do not commit a screenshot that shows personal data.

Check before you push

cd docs
hugo --minify --baseURL https://localai.io/docs/
cd ..
.github/ci/check-docs-redirects.sh docs/public docs/redirects.txt
.github/ci/check-docs-links.py docs/public --fragments

The pull request build runs the same checks.