https://localai.io/docs/models/engines/index.htmlBackends that have their own page: vllm.cpp, RKLLM, Strata, audio.cpp, NeMo-Speech.cpp and Whisper-Medusa.Hugoen-GBhttps://localai.io/docs/models/engines/vllm-cpp/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/engines/vllm-cpp/index.htmlvllm.cpp is the LocalAI team’s C++20 port of vLLM: the same continuous-batching scheduler, paged KV cache and automatic prefix caching, with no Python at inference time. LocalAI serves it through the native vllm-cpp backend, which loads either a HuggingFace safetensors directory or a .gguf file and applies the chat template, tool-call parsing and reasoning split inside the engine. This page covers installing the backend and the models LocalAI ships ready to run on it. For the full engine_args reference (KV sizing, scheduling policy, speculative decoding, LMCache), see the vllm.cpp section of the text generation guide.https://localai.io/docs/models/engines/rkllm/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/engines/rkllm/index.htmlServe RKLLM models from Rockchip NPUs through LocalAIhttps://localai.io/docs/models/engines/strata/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/engines/strata/index.htmlStrata is an MIT-licensed inference engine for large mixture-of-experts models on consumer hardware. LocalAI exposes it through the experimental strata backend. The backend starts the Strata engine as a child process and talks to it over the engine’s line protocol. Text generation only. Strata is developed by its own authors, not by the LocalAI project. Installing local-ai backends install strata LocalAI picks the image that fits the machine. The image already contains everything the backend needs: the engine, a portable Python (with numpy) for the tokenizer helper and the pack tool, and the Strata Python modules. You do not build or download an engine yourself.https://localai.io/docs/models/engines/audio-cpp/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/engines/audio-cpp/index.htmlaudio.cpp is a GGML audio inference framework covering roughly thirty model families in one engine. LocalAI exposes it through the native C++ audio-cpp backend, so a single installed backend can serve text to speech, voice cloning, transcription, forced alignment, voice activity detection, speaker diarization, source separation, voice conversion and music generation, depending only on which model you point it at. audio.cpp is developed by ShugoAI LLC, not by the LocalAI project. Model packages live in huggingface.co/audio-cpp/audio.cpp-gguf.https://localai.io/docs/models/engines/nemo-speech-cpp/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/engines/nemo-speech-cpp/index.htmlNeMo-Speech.cpp is NVIDIA’s Apache-2.0 C++/ggml runtime for the Nemotron Speech models. LocalAI exposes it through the native nemo-speech-cpp backend, which serves four model families from one installed backend: transcription, speaker diarization, speech synthesis and text translation. NeMo-Speech.cpp is developed by NVIDIA, not by the LocalAI project. Installing local-ai backends install nemo-speech-cpp Or install it from the Backends page in the web UI. nemo-speech-cpp is a preference-only backend: LocalAI never picks it automatically during model import, because the family is decided by the GGUF’s general.architecture key, which cannot be read from a remote repository, and because a translation model carries an ordinary LLM architecture with no NeMo-specific marker at all. Set backend: nemo-speech-cpp in the model YAML, or select it explicitly in the import form.https://localai.io/docs/models/engines/whisper-medusa/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/engines/whisper-medusa/index.htmlRun Whisper-Medusa speech-to-text checkpoints with LocalAI