Run with container images
LocalAI provides a variety of images to support different environments. These images are available on quay.io and Docker Hub.
For GPU Acceleration support for Nvidia video graphic cards, use the Nvidia/CUDA images, if you don’t have a GPU, use the CPU images. If you have AMD or Mac Silicon, see the build section.
Tip
Available Images Types:
- Images ending with
-coreare smaller images without predownload python dependencies. Use these images if you plan to usellama.cpp,stablediffusion-ncnorrwkvbackends - if you are not sure which one to use, do not use these images.
Prerequisites
Before you begin, ensure you have a container engine installed if you are not using the binaries. Suitable options include Docker or Podman. For installation instructions, refer to the following guides:
Tip
Hardware Requirements: The hardware requirements for LocalAI vary based on the model size and quantization method used. For performance benchmarks with different backends, such as llama.cpp, visit this link. The rwkv backend is noted for its lower resource consumption.
Standard container images
Standard container images do not have pre-installed models. Use these if you want to configure models manually.
| Description | Quay | Docker Hub |
|---|---|---|
| Latest images from the branch (development) | quay.io/go-skynet/local-ai:master | localai/localai:master |
| Latest tag | quay.io/go-skynet/local-ai:latest | localai/localai:latest |
| Versioned image | quay.io/go-skynet/local-ai:v4.0.0 | localai/localai:v4.0.0 |
| Description | Quay | Docker Hub |
|---|---|---|
| Latest images from the branch (development) | quay.io/go-skynet/local-ai:master-gpu-nvidia-cuda-12 | localai/localai:master-gpu-nvidia-cuda-12 |
| Latest tag | quay.io/go-skynet/local-ai:latest-gpu-nvidia-cuda-12 | localai/localai:latest-gpu-nvidia-cuda-12 |
| Versioned image | quay.io/go-skynet/local-ai:v4.0.0-gpu-nvidia-cuda-12 | localai/localai:v4.0.0-gpu-nvidia-cuda-12 |
| Description | Quay | Docker Hub |
|---|---|---|
| Latest images from the branch (development) | quay.io/go-skynet/local-ai:master-gpu-nvidia-cuda-13 | localai/localai:master-gpu-nvidia-cuda-13 |
| Latest tag | quay.io/go-skynet/local-ai:latest-gpu-nvidia-cuda-13 | localai/localai:latest-gpu-nvidia-cuda-13 |
| Versioned image | quay.io/go-skynet/local-ai:v4.0.0-gpu-nvidia-cuda-13 | localai/localai:v4.0.0-gpu-nvidia-cuda-13 |
| Description | Quay | Docker Hub |
|---|---|---|
| Latest images from the branch (development) | quay.io/go-skynet/local-ai:master-gpu-intel | localai/localai:master-gpu-intel |
| Latest tag | quay.io/go-skynet/local-ai:latest-gpu-intel | localai/localai:latest-gpu-intel |
| Versioned image | quay.io/go-skynet/local-ai:v4.0.0-gpu-intel | localai/localai:v4.0.0-gpu-intel |
| Description | Quay | Docker Hub |
|---|---|---|
| Latest images from the branch (development) | quay.io/go-skynet/local-ai:master-gpu-hipblas | localai/localai:master-gpu-hipblas |
| Latest tag | quay.io/go-skynet/local-ai:latest-gpu-hipblas | localai/localai:latest-gpu-hipblas |
| Versioned image | quay.io/go-skynet/local-ai:v4.0.0-gpu-hipblas | localai/localai:v4.0.0-gpu-hipblas |
| Description | Quay | Docker Hub |
|---|---|---|
| Latest images from the branch (development) | quay.io/go-skynet/local-ai:master-vulkan | localai/localai:master-vulkan |
| Latest tag | quay.io/go-skynet/local-ai:latest-gpu-vulkan | localai/localai:latest-gpu-vulkan |
| Versioned image | quay.io/go-skynet/local-ai:v4.0.0-vulkan | localai/localai:v4.0.0-vulkan |
These images are compatible with Nvidia ARM64 devices with CUDA 12, such as the Jetson Nano, Jetson Xavier NX, and Jetson AGX Orin. For more information, see the Nvidia L4T guide.
| Description | Quay | Docker Hub |
|---|---|---|
| Latest images from the branch (development) | quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64 | localai/localai:master-nvidia-l4t-arm64 |
| Latest tag | quay.io/go-skynet/local-ai:latest-nvidia-l4t-arm64 | localai/localai:latest-nvidia-l4t-arm64 |
| Versioned image | quay.io/go-skynet/local-ai:v4.0.0-nvidia-l4t-arm64 | localai/localai:v4.0.0-nvidia-l4t-arm64 |
These images are compatible with Nvidia ARM64 devices with CUDA 13, such as the Nvidia DGX Spark. For more information, see the Nvidia L4T guide.
| Description | Quay | Docker Hub |
|---|---|---|
| Latest images from the branch (development) | quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-cuda-13 | localai/localai:master-nvidia-l4t-arm64-cuda-13 |
| Latest tag | quay.io/go-skynet/local-ai:latest-nvidia-l4t-arm64-cuda-13 | localai/localai:latest-nvidia-l4t-arm64-cuda-13 |
| Versioned image | quay.io/go-skynet/local-ai:v4.0.0-nvidia-l4t-arm64-cuda-13 | localai/localai:v4.0.0-nvidia-l4t-arm64-cuda-13 |