This section is for developers and contributors. If you are looking for the user documentation, this is not the right place!
This section will collect how-to, notes and development documentation
We use conventional commits and semantic versioning. Please follow the conventional commits specification when writing commit messages.
Creating a gRPC backend
LocalAI backends are
In order to create a new backend you need:
- If there are changes required to the protobuf code, modify the proto file and re-generate the code with
- Modify the
Makefileto add your new backend and re-generate the client code with
make protogenif necessary.
- Create a new
extra/grpcif it’s not written in go: link, and create the specific implementation.
- Update the Dockerfile: if the backend is written in another language, update the
Dockerfiledefault EXTERNAL_GRPC_BACKENDS variable by listing the new binary link.
Once you are done, you can either re-build
LocalAI with your backend or you can try it out by running the
gRPC server manually and specifying the host and IP to LocalAI with
--external-grpc-backends or using (
EXTERNAL_GRPC_BACKENDS environment variable, comma separated list of
name:host:port tuples, e.g.
./local-ai --debug --external-grpc-backends "my-awesome-backend:host:port" ...