LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families and architectures. Does not require GPU. It is maintained by mudler.
In a nutshell:
Local, OpenAI drop-in alternative REST API. You own your data.
NO GPU required. NO Internet access is required either
Optional, GPU Acceleration is available. See also the build section.
Supports multiple models
🏃 Once loaded the first time, it keep models loaded in memory for faster inference
⚡ Doesn’t shell-out, but uses bindings for a faster inference and better performance.
LocalAI is focused on making the AI accessible to anyone. Any contribution, feedback and PR is welcome!
Note that this started just as a fun weekend project by mudler in order to try to create the necessary pieces for a full AI assistant like ChatGPT: the community is growing fast and we are working hard to make it better and more stable. If you want to help, please consider contributing (see below)!