Collections (knowledge base)

A collection is a named set of documents that LocalAI splits into chunks and stores as embeddings. You can search a collection by meaning. A saved agent can list the collections it may search, and a chat with that agent gets the tools list_collections and search_collection.

Collections are on by default. To turn them off, set LOCALAI_DISABLE_COLLECTIONS=true. The name of this variable is old: it now turns off the collections only.

How it works

1. You create a collection and upload files to it, or you add a URL as a source.
2. LocalAI splits each document into chunks no larger than LOCALAI_COLLECTIONS_MAX_CHUNKING_SIZE.
3. It calls the embedding model to turn each chunk into a vector, and stores the vectors in the vector engine.
4. A search embeds the query and returns the closest chunks.

The embedding calls go to the LocalAI API of the same server, inside the process, so they need no API key and work with user accounts. The embedding model must be installed. The default is granite-embedding-107m-multilingual. When it is not, the embeddings endpoint answers 404 and every upload and search fails. The Collections page then says No embedding model installed and names the model; an administrator gets a button that opens the Models page with a search for it. GET /api/collections reports the same in embedding_model and embedding_model_installed.

Who sees a collection

• With authentication off, there is one set of collections.
• With authentication on, each user has their own collections. A user needs the collections feature. Admins always have it.
• A name that is not yours answers 404. Nobody sees another user’s collections, with two exceptions for admins below.

API

All paths are under /api/collections. While the service starts, every path answers 503 with collections are starting, please retry shortly. When collections are off, the paths do not exist.

Example:

curl -X POST http://localhost:8080/api/collections \
  -H "Content-Type: application/json" -d '{"name": "docs"}'
curl -X POST http://localhost:8080/api/collections/docs/upload -F file=@manual.pdf
curl -X POST http://localhost:8080/api/collections/docs/search \
  -H "Content-Type: application/json" -d '{"query": "how do I reset the device?", "max_results": 3}'

Notes:

• update_interval is in minutes. A value below 1 means 60.
• Names and entries with special characters, such as :, must be URL-encoded in the path.
• A 404 means the collection or the entry does not exist for you.
• A search for more passages than the collection holds returns all of them. An empty collection returns none.
• /readyz answers 200 only after the collections are ready, so /api/collections does not answer 503 once the server is in rotation.
• An upload can fail with 409 when the embedding model changed and the vectors have a different size. Restart LocalAI to embed the collection again, or reset the collection.

Admin options

With authentication on, an admin can add these query parameters:

• ?all_users=true on GET / also returns the collections of every other user, in user_groups, keyed by user ID.
• ?user_id=<id> on every path except GET / and POST / works on the collections of that user. Only an administrator may use it.

Other users cannot use either parameter. user_id is ignored for them and they always work on their own collections.

Settings

Set these as environment variables or flags. Most can also be set in the runtime settings file, and a change needs a restart.

PostgreSQL vector engine

The default engine keeps the vectors in files on the local disk. To keep them in PostgreSQL, use a PostgreSQL database and set:

LOCALAI_COLLECTIONS_VECTOR_ENGINE=postgres
LOCALAI_COLLECTIONS_DATABASE_URL=postgresql://user:password@host:5432/localrecall?sslmode=disable

The docker-compose.yaml of the project has a commented example with a PostgreSQL service. With several frontends in distributed mode, use the postgres engine, so that all frontends share the same collections.

Use a collection in an agent

Add the names of your collections to the collections list of a saved agent. A chat with that agent can then search only those collections. If a listed collection is deleted, the chat still starts and the model is told that it is not available.