https://localai.io/docs/models/capabilities/audio/index.htmlSpeech to text, text to speech, sound classification, speakers and voice activity.Hugoen-GBhttps://localai.io/docs/models/capabilities/audio/audio-to-text/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/capabilities/audio/audio-to-text/index.htmlTranscribe audio to text with the OpenAI-compatible transcription API.https://localai.io/docs/models/capabilities/audio/text-to-audio/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/capabilities/audio/text-to-audio/index.htmlAPI Compatibility The LocalAI TTS API is compatible with the OpenAI TTS API and the Elevenlabs API. LocalAI API The /tts endpoint can also be used to generate speech from text. Usage Input: input, modelhttps://localai.io/docs/models/capabilities/audio/audio-classification/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/capabilities/audio/audio-classification/index.htmlSound-event classification (audio tagging) answers the question “what am I hearing?” - given an audio clip, it returns a list of scored AudioSet labels (e.g. Baby cry, infant cry, Glass breaking, Dog bark, Alarm). LocalAI exposes this through the /v1/audio/classification endpoint, modelled after /v1/audio/transcriptions. The reference backend is ced.cpp (CED, a 527-class AudioSet tagger), a small ViT over a log-mel spectrogram ported to ggml with full PyTorch parity. Apache-2.0 weights are redistributable as GGUF.https://localai.io/docs/models/capabilities/audio/audio-diarization/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/capabilities/audio/audio-diarization/index.htmlSpeaker diarization answers the question “who spoke when?” - given an audio clip with multiple speakers, it returns time-stamped segments labelled with a stable speaker ID (SPEAKER_00, SPEAKER_01, …). LocalAI exposes this through the /v1/audio/diarization endpoint, modelled after /v1/audio/transcriptions. Five backends are supported today: sherpa-onnx - pyannote-3.0 segmentation + a speaker-embedding extractor (3D-Speaker, NeMo, WeSpeaker) + fast clustering. Pure diarization - no transcription cost. Recommended when you only need speaker turns. vibevoice.cpp - produces speaker-labelled segments as a by-product of its long-form ASR pass, so you can optionally get a transcript per segment for free. NeMo-Speech.cpp - NVIDIA Sortformer, served standalone by the NeMo-Speech.cpp backend. It is end to end, so the speaker capacity is fixed by the checkpoint and the count hints are ignored. The same backend can instead put speaker tags on a transcript, by attaching a Sortformer model to an ASR one. audio.cpp - the sortformer_diar family, served by the multi-modality audio.cpp backend. parakeet.cpp - NVIDIA Nemotron-3-Diarization (Sortformer, up to 8 speakers), served standalone or paired with a Parakeet ASR model for per-segment text. See the Audio to Text page for the parakeet-cpp option reference. Because diarization is exposed as a regular OpenAI-compatible endpoint, any HTTP client works. There is no Python dependency on pyannote or NeMo on the consumer side.https://localai.io/docs/models/capabilities/audio/audio-transform/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/capabilities/audio/audio-transform/index.htmlThe audio-transform endpoints take audio in and emit audio out, optionally conditioned on a second reference audio signal. The category is generic by design - concrete operations include joint acoustic echo cancellation + noise suppression + dereverberation (LocalVQE), voice conversion (reference = target speaker), pitch shifting, audio super-resolution, and so on.https://localai.io/docs/models/capabilities/audio/voice-activity-detection/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/capabilities/audio/voice-activity-detection/index.htmlVoice Activity Detection (VAD) identifies segments of speech in audio data. LocalAI provides a /v1/vad endpoint powered by the Silero VAD backend. The audio.cpp backend also serves this endpoint, and ships the silero_vad and marblenet_vad assets inside its own package, so VAD works there with nothing to download (model: bundled:silero_vad plus the family:silero_vad option). API Method: POST Endpoints: /v1/vad, /vad Request The request body is JSON with the following fields:https://localai.io/docs/models/capabilities/audio/voice-recognition/index.htmlMon, 01 Jan 0001 00:00:00 +0000https://localai.io/docs/models/capabilities/audio/voice-recognition/index.htmlLocalAI supports voice (speaker) recognition: speaker verification (1:1), speaker identification (1:N) against a built-in vector store, speaker embedding, and demographic analysis (age / gender / emotion from voice). The audio analog to Face Recognition, served over the same /v1/voice/* HTTP API by two backends: