From the team
Blog
Release write-ups, benchmark reports and engineering notes from the LocalAI team. Every number here comes out of a benchmark suite, a release or a commit, and the source is named so you can check it.
History
LocalAI, from March 2023 to now
Three years, 133 releases and 224 contributors later. The four changes that mattered most were making the core small, adding agents, making it a cluster, and giving it eyes and ears.
Read →
Release
What landed in LocalAI 4.8
The web interface got 3.48x lighter, gallery entries now install the build your hardware can actually run, and there is a new inference engine in the box. 214 pull requests in thirteen days.
Read →
Engineering
Why we write our own C and C++ engines
A 66 MiB binary instead of a 9.1 GiB virtualenv, depth estimation that beats PyTorch on CPU in half the memory, and biometrics that match insightface bit for bit. The method, the measurements, and what it costs us.
Read →
Benchmarks
parakeet.cpp: NeMo transcripts, byte for byte, without the Python
Same transcript as NVIDIA NeMo, character for character, at a median 1.40x on CPU and about 27x the speed of whisper.cpp. One binary, one GGUF file, no Python at inference.
Read →
Research
APEX: a 35B MoE model at 12.2 GB, and faster than F16
Qwen3.5-35B-A3B goes from 64.6 GB to 12.2 GB and speeds up from 30.4 to 74.4 tokens per second. Perplexity moves from 6.537 to 7.088. Here is the precision assignment that does it, and where it costs you.
Read →
Next
Install it and check the numbers yourself.
Every figure in these posts comes out of a benchmark suite or a release that you can run on your own hardware.