Decisions API

The Decisions API is a fast, typed decision layer. You send a piece of text (the state) and a set of named questions. A decision model answers each question with a value and a confidence, in one pass. The model does not generate text, so there is nothing to parse and no free-form output to validate.

LocalAI serves it on the /v1/systemone routes. The request and response shapes follow the kev project, and the field names and question types are the same ones Ollama serves on its /v1/systemone endpoint (Ollama 0.35 and later). The wire contract is called SystemOne; the capability a model declares is called decisions. See Compatibility with Ollama for what differs.

LocalAI also serves the OpenAI-compatible /v1/decisions contract with input and an ordered questions array. Both APIs share SystemOne execution, model selection, validation, admission limits, and backend support. They differ in request and response shape, not in the underlying model.

OpenAI-compatible decisions

Use an installed local decision model, such as julia-1-llama-cpp:

curl http://localhost:8080/v1/decisions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "julia-1-llama-cpp",
    "input": "My order arrived broken and I want my money back. This is the second time.",
    "questions": [
      {
        "type": "predicate",
        "name": "refund_requested",
        "instructions": "The customer explicitly asks for a refund"
      },
      {
        "type": "choice",
        "name": "team",
        "instructions": "Which team should handle this ticket?",
        "choices": [
          {"value": "billing", "description": "Payments, invoices, and refunds"},
          {"value": "shipping", "description": "Delivery and damaged goods"},
          {"value": "product", "description": "Questions about how the product works"}
        ]
      },
      {
        "type": "score",
        "name": "urgency",
        "instructions": "How urgent is this ticket?",
        "levels": [
          {"label": "not urgent"},
          {"label": "somewhat urgent"},
          {"label": "urgent"},
          {"label": "critical"}
        ]
      }
    ]
  }'

The response contains model, an ordered answers array, and usage. Answers preserve question order and optional names. Named questions must have unique names. Each question requires instructions.

Choice values retain their JSON types: true and "true" are different choices. Numbers, objects, arrays, and null are not supported choice values. Scores are probability-weighted sums of zero-based level indices, not necessarily integers. Backend refusals appear as answers with type: "refusal".

usage contains input_tokens, output_tokens, their sum as total_tokens, input_tokens_details, and output_tokens_details. Counts reflect available backend accounting, including explicit zero output tokens. The NER path reports zero token counts. Detail objects contain only backend-supplied counts and are empty objects when unavailable. LocalAI does not invent cached, reasoning, text, or image token breakdowns.

Input and compatibility limits

input accepts a text string or an array of user messages. Each message has role: "user", optional type: "message", and content. Content accepts a string or an array of input_text and input_image parts:

{
  "input": [{
    "role": "user",
    "content": [
      {"type": "input_text", "text": "Does this package look damaged?"},
      {"type": "input_image", "image_url": "data:image/png;base64,...", "detail": "auto"}
    ]
  }]
}

This fragment requires a complete PNG or JPEG data URL and an image-capable model, such as OpenJev with its projector. Image-only input is supported. Only omitted or default ("auto") image detail is supported; other values return an error. LocalAI never fetches external image URLs or file paths.

The adapter joins text parts with newlines and collects images in encounter order. It flattens message boundaries and text/image interleaving into SystemOne text and an image list. It does not preserve a conversation layout for inference.

Requests support at most 64 questions. Text-only bodies have a 64 KiB limit; image-bearing bodies have a 16 MiB limit. The shared image limits are 8 images, 12 MiB aggregate encoded data-URL bytes, 8 MiB aggregate decoded bytes, 4096 pixels per dimension, and 16 million aggregate pixels. See Bounded image input for validation and admission behavior. Text-only models and the NER path reject images instead of dropping them. Backend-specific question limits still apply, including llama.cpp’s 2–10 score levels.

safety_identifier is accepted as compatibility metadata. It does not establish an authentication identity or enable a safety service. Normal LocalAI authentication and the decisions permission still apply.

Probabilities and confidence come from the local model and its pipeline. OpenAI-compatible fields do not guarantee OpenAI calibration, model quality, or safety behavior. Evaluate the installed model for your task.

Endpoints

Which route a model can serve depends on its kind:

Question types

Example

curl http://localhost:8080/v1/systemone -H "Content-Type: application/json" -d '{
  "model": "laya-vllm-cpp",
  "state": "My order arrived broken and I want my money back. This is the second time.",
  "questions": {
    "team": {
      "type": "choice",
      "instructions": "Which team should handle this ticket?",
      "criteria": {
        "billing": "Payments, invoices and refunds",
        "shipping": "Delivery and damaged goods",
        "product": "Questions about how the product works"
      }
    },
    "refund_requested": {
      "type": "noul",
      "instructions": "The customer explicitly asks for a refund"
    },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is this ticket?",
      "criteria": ["not urgent", "somewhat urgent", "urgent", "critical"]
    }
  }
}'

Answers from a decision model carry a confidence value, and the response reports token usage and latency_ms. The NER path does not report token usage.

Choosing a model

A model can serve the Decisions API only if it is a decision model. Declare the usecase in the model config:

name: laya
backend: vllm-cpp
known_usecases:
  - decisions
parameters:
  model: convaiinnovations/laya

decisions is never guessed, and a model that declares it is not listed as a chat, completion or embeddings model. A model that declares usecases without decisions or token_classify gets a 400 from these endpoints that names the missing usecase. A model that declares token_classify and not decisions is served by the zero-shot NER path. A vllm-cpp config that declares no usecases is treated as a decision model, so setups that predate the flag keep working, but a config that declares only chat (as an older laya gallery entry did) now gets the 400 and needs known_usecases: [decisions].

Install one from the gallery and filter on the decisions tag:

The engine, vllm.cpp, also supports the xor decision model. That checkpoint needs a conversion step, so it is not a gallery entry yet. The kev, Nimble and CLM entries install checkpoints that were already converted with the vllm.cpp convert-kev.py, convert-nimble.py and convert-clm.py scripts.

Nimble refuses a question with more than 26 choices (the upstream release allows 255). On CPU it needs about 20 GB of free RAM. For CLM, put the question in instructions: the state head reads the state followed by the instructions. On CPU it needs about 19 GB of free RAM.

Tev1 is an autoregressive decision model. The engine answers each question by scoring the option letters, so its confidence is the entropy measure Ollama uses. A Tev1 choice or score question accepts at most 24 options (Ollama allows 26), because the model is trained on the letters A to X, and every option needs a nonempty description. The published checkpoints name another architecture in config.json, so the Tev1 gallery entries set engine_args.hf_overrides to load them as Tev1Model (see Overriding config.json keys). The same model also answers /v1/chat/completions requests.

Request limits

A request is refused with 400 (or 413 for the body size) when:

• a text-only body is larger than 64 KiB (image-bearing bodies have the bounded budget below),
• state is missing or blank,
• there are no questions, or more than 64,
• a question id is blank,
• a choice question has fewer than 2 options or a blank option key,
• a score question has fewer than 2 levels,
• a noul question has criteria with keys other than "false" and "true".

A noul question may carry criteria with a description for each outcome, for example {"false": "No refund is requested", "true": "The customer requests a refund"}. Some models cap the number of options for a choice or score question. Models that answer with a letter accept at most 26, and Tev1 accepts at most 24. The engine refuses more options than the model supports and the error names the limit.

Compatibility with Ollama

The field names, question types and answer fields are the same as Ollama’s /v1/systemone, so a client written for one works against the other for the common case. These behaviors differ:

Access control

When authentication is on, all four routes need the decisions feature. It is on by default for every user, like the other API features, and an administrator can turn it off per user.

Native llama.cpp decisions

The stock llama-cpp backend supports text-only decision GGUFs carrying upstream SystemOne metadata. Declare known_usecases: [decisions]; ordinary score need not be enabled. Requests use the existing internal Score RPC, not a backend HTTP server. Choice, score, and noul questions may be combined in one request. Structured state and questions are forwarded without NER rendering. llama.cpp score questions accept 2–10 levels; this backend-specific limit does not constrain vllm-cpp. Older forks without native decision support return 501. Missing decision metadata also returns 501, while backend invalid requests return 400.

A llama-cpp backend image built before native decisions cannot answer these requests, even when the gallery reports it as up to date. LocalAI detects this from the failed request: it returns 501 with a message that the backend is older than the server, and GET /backends/upgrades lists the backend with a reason field. Upgrade the backend, or install a current build such as the -development variant. Adding score to known_usecases does not help. The flag clears when the backend is reinstalled; if the reinstalled image is still old, the next decision request flags it again.

Bounded image input

The public routes and internal decision validator share an image contract. Supply PNG or JPEG base64 data URLs in images, OpenAI image_url message content, or Anthropic image content with a base64 source and media_type. Remote URLs and file paths are never fetched. MIME must match the decoded image header; malformed base64, unsupported formats and invalid headers return 400.

Limits per request are 8 images, 12 MiB aggregate encoded data-URL bytes, 8 MiB aggregate decoded bytes, 4096 pixels per dimension, and 16 million aggregate pixels. Exceeding these limits returns 413. Headers are checked before full pixel decoding, which rejects truncated or corrupt images; native decoders must independently protect direct RPC inputs.

Image-bearing request bodies may use up to 16 MiB. Text-only requests retain the 64 KiB raw-wire limit, including whitespace; JSON escaping during internal serialization does not impose a second HTTP limit. Absent, null, or empty images do not enable the larger budget. Native decision responses retain a separate 64 KiB limit, independent of the request budget. These limits do not raise any global HTTP limit.

A shared 8-request admission ceiling covers public decision handlers and internal decision runners before body buffering, image decoding or serialization. Saturation fails promptly (HTTP 503); cancellation before admission does not take a slot. A slot remains held through inference/response handling, and an internal cancelled call retains its slot until its underlying worker actually ends. This bounds concurrent decision-owned allocation and retained request bodies, not total process memory: caller-owned inputs, upstream middleware buffers and model/backend memory are outside this budget. Full decoding is sequential per admitted request, with each pixel buffer limited by the checked dimensions and aggregate pixel budget (up to 16 million pixels; decoded byte storage depends on pixel format). Garbage collection timing is not an RSS guarantee. The separate 8-operation backend ceiling still bounds abandoned native operations. Validation helpers do not acquire nested slots.

For image-only input, provide explicit structured state such as "state": {} alongside images, or a message containing image content. Missing, null or blank string state remains invalid. Arbitrary domain JSON is preserved, not interpreted as image content outside message content parts. Images are never replaced by invented text.

Admission is not a promise of model image capability: the NER path and text-only native decision models reject images with 501 rather than silently dropping them. OpenJev requires its vision projector (see below). Failed requests are not billed. Router image probes preserve the canonical request; full public API image validation across installed models is separate from native RPC validation.

Native responses report backend input/output usage, including zero generated tokens. LocalAI records supplied usage once; explicit zero counts are distinct from missing usage. Missing counts are not estimated, and invalid negative counts are rejected rather than billed.

Julia-1 CPU example

Install the separate stock llama.cpp entry (existing vllm-cpp entries are unchanged):

local-ai models install julia-1-llama-cpp

Julia-1 is a 144.3M-parameter multilingual text decision model. The gallery pins ggml-org/Julia-1-GGUF revision 16fee17949206fbf58da9347daea44d792a81211, file Julia-1-Q8_0.gguf (168,166,496 bytes, about 160.4 MiB), SHA-256 1ea6a7e87156eeeda88cb7a36a61265b37ba7b993897b7289b99aea5b5e47069. The source model and GGUF publisher declare Apache-2.0. Source provenance: SupersonicLabs/Julia-1 revision a85b127321d580d65176c89ced8273f305745d85, based on jhu-clsp/mmBERT-small. This is a real model, not the upstream tiny test fixture; assess its accuracy for your own tasks.

curl http://localhost:8080/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"model":"julia-1-llama-cpp","state":"I was charged twice and need a refund.","questions":{"route":{"type":"choice","instructions":"Which team should handle this?","criteria":{"billing":"payments and refunds","shipping":"delivery problems","technical":"software issues"}},"refund":{"type":"noul","instructions":"Does the customer request a refund?"}}}'

The pinned artifact was installed through the gallery installer, checksum-verified, and tested on CPU with the native Score RPC using choice, score, and noul in one request. That smoke returned 97 input tokens and zero output tokens; token counts vary with the request. This does not establish broad model accuracy or image support.

Native family defaults

Each family has a separately named default; these do not replace vllm-cpp entries. All entries except OpenJev are text-only and omit projectors. OpenJev includes its Q8 projector. Download size is not a RAM estimate.

OpenJev and Nimble are noncommercial models. OpenJev’s upstream multimodal capability does not imply LocalAI decision-image support. Nimble requires the native Nimble integration included in this source tree’s llama.cpp pin bed0a856606ee4a24a164066f73d2379447033f5; older installed backends must be updated before serving it. This source prerequisite is integrated, but the OpenJev and Nimble installation/runtime checks remain pending as listed above. The published entries pin revisions and SHA-256 checksums, but metadata verification alone is not a runtime test. No model-quality guarantee follows from these smoke tests. Laya, Kev-4B, and lev were also retested against the newer native backend with 1- and 11-level score requests correctly rejected.

OpenJev and Nimble validation used the native backend at llama.cpp revision bed0a856606ee4a24a164066f73d2379447033f5, CPU-only with two threads, a 2048-token context, and batch size 512. Each artifact was installed through the gallery, SHA-256 verified, and checked for its decision metadata and SystemOne template. Each request included choice, score, and noul questions together, including the full question set required by Nimble. Response-shape, probability-normalization, and noul-bound assertions passed; 1- and 11-level score requests were rejected. The test requests reported 224 input tokens for OpenJev and 933 for Nimble, with explicit zero output tokens for both. No projector was installed or tested.

These are bounded text contract smoke tests, not accuracy benchmarks or performance guarantees. Floating-point probabilities can vary with hardware and build settings; tests do not require exact answer probabilities or token counts. Neither image support nor interruption during active evaluation is established by these tests. CC-BY-NC-4.0’s noncommercial restriction still applies.

Multimodal router probes

The decisions router classifier preserves ordered OpenAI message content and Anthropic base64 image sources as structured state, including image-only turns. It uses the internal decision runner, not a loopback HTTP request. The shared image limits above are validated before model loading; no URL is fetched by the classifier. Original message content is not rewritten when selecting a candidate or the configured fallback.

Score, rerank and KNN classifiers are text-only: image input produces an explicit classifier error and follows the existing configured fallback policy, rather than classifying an image-stripped prompt. Without a fallback, routing fails. Parent cancellation remains terminal and does not select a fallback. Image probes bypass text embedding caches and are not trimmed to text-only turns. Native context overflow is reported by the backend rather than silently dropping images. These transport guarantees do not establish installed projector capability or real-model image accuracy; those require separate native and end-to-end validation.

OpenAI chat routing classifies the original structured message before preparing media for the selected model. Remote image URLs are not downloaded as decision inputs. After selection (including a configured fallback), the served model’s normal media preparation runs without replacing the original content blocks. Invalid classifier configuration fails closed even with a configured fallback. Runtime classification and input errors follow the configured fallback policy. Cancellation never selects a fallback.

Router probe extraction checks the shared 16 MiB state budget before copying text or serializing messages, including JSON escaping expansion. This applies to typed and untyped internal requests as well as parsed API requests; it does not add a limit to non-router inference. Direct internal probes containing custom structs, JSON/text marshalers, or excessively nested values fail extraction rather than executing their serialization. Supported internal values are the chat schema message/content/tool types and plain JSON values (including json.RawMessage, conservatively budgeted for escaping). Direct prompt-only probes also check serialized escaping before allocation. The separate 64 KiB text-only Decisions request limit is unchanged. Anthropic conversion preserves typed content blocks through both native selection and fallback, including ordered text and images.

OpenJev image decisions

The openjev-llama-cpp gallery entry installs OpenJev Q4_K_M and its pinned Q8 vision projector (mmproj-OpenJev-Q8_0.gguf). Both artifacts come from ggml-org/OpenJev-GGUF revision 10840f375658dea7afc5ff4711127bca8218b560. The weights occupy 18,973,872,288 bytes and the projector 629,247,232 bytes: 19,603,119,520 bytes total (about 19.61 GB decimal), excluding runtime memory, KV cache and backend files. The model is CC-BY-NC-4.0, noncommercial only; LocalAI’s software license does not override the model license.

For a manually installed model, set mmproj: mmproj-OpenJev-Q8_0.gguf alongside parameters.model, not inside parameters. LocalAI resolves that filename relative to the model directory and forwards it to llama.cpp’s projector loader. The gallery uses an 8192-token context to leave room for image tokens and question framing. This is not a guarantee that all eight maximum-sized images fit; context overflow remains an error. Size the context for the actual workload.

Native image decisions require both a decision format that accepts images and a loaded projector that supports vision input. A missing projector, an audio-only projector, or a text-only decision model does not silently fall back to a text decision. Other decision gallery entries remain text-only.

Direct native RPC callers receive the same image count, encoded/decoded byte, dimension and aggregate pixel bounds as public callers. Validation precedes llama.cpp’s permissive media parsing and full pixel decode; PNG decompression is independently bounded before stb decodes pixels. This validation applies only to native decision tasks, not ordinary chat or legacy scoring.

Native decision image validation rejects PNG streams with invalid checksums and incomplete JPEG scans, including truncated scans with an appended end marker. Source builds with native decision support require zlib and libjpeg development packages (zlib1g-dev libjpeg-dev on Ubuntu; zlib jpeg-turbo on Homebrew). Packaged backends include the required runtime libraries.