From the team
Blog
Release write-ups, benchmark reports and engineering notes from the LocalAI team. Numbers link to the release, commit or benchmark run they came from.
Release
What landed in LocalAI 4.9
Authentication now denies by default, chat can compress its own history, models and backends each got one page, and vllm-cpp builds for cards other than Blackwell. 146 pull requests in thirteen days.
Read →
Release
What landed in LocalAI 4.8
A new inference engine, a terminal agent in the CLI, 3D generation, and a web interface 3.48x lighter. 386 pull requests in twenty-two days.
Read →
History
LocalAI, from March 2023 to now
Three years, 133 releases and 224 contributors later. Here are the four decisions that shaped it: making the core small, adding agents, making it a cluster, and giving it eyes and ears.
Read →
Engineering
Why we write our own C and C++ engines
Eighteen of our backends are C or C++ ports we wrote from scratch instead of wrapping an upstream engine. Here is why we did it and what we measured.
Read →
Benchmarks
parakeet.cpp: the same NeMo transcript, without the Python
The same transcript as NVIDIA NeMo at a median 1.40x on CPU, and about 27x the speed of whisper.cpp, from one binary and one GGUF file.
Read →
Release
LocalAI 4.3: signed backends, and the prompt cache that was off
Keyless cosign verification for backend OCI images, the llama.cpp prompt cache enabled by default, per-API-key usage attribution, and the replica-pinning bug that kept a second node idle.
Read →
Release
LocalAI 4.2: who spoke when, and whose face is that
A /v1/audio/diarization endpoint, voice and face recognition with liveness, a drop-in Ollama API, and eleven new backends.
Read →
Research
APEX: a 35B MoE model at 12.2 GB, and faster than F16
Qwen3.5-35B-A3B goes from 64.6 GB to 12.2 GB and speeds up from 30.4 to 74.4 tokens per second. Perplexity moves from 6.537 to 7.088. Here is the precision assignment that does it, and where the quality drops.
Read →
Release
LocalAI 4.1: more than one box, and more than one user
Distributed cluster mode that places requests by real free VRAM, OIDC with per-user API keys and quotas, and LoRA fine-tuning that exports straight to GGUF.
Read →
Release
LocalAI 4.0: agents in the core, and a React interface
Native agent orchestration with the Agenthub, a rewritten interface with Canvas mode, MCP Apps with tool streaming, and two things removed.
Read →
Release
LocalAI 3.10: the Anthropic and Responses APIs, and one image for every GPU
A /v1/messages endpoint that Claude clients can talk to unchanged, Open Responses compatibility that passes the official acceptance tests, and GPU libraries moved inside the backend containers so one image works on any hardware.
Read →
Next
Install it and check the numbers yourself.
Every figure in these posts comes out of a benchmark suite or a release that you can run on your own hardware.