When it helps
Tuning paid on a few labels, with real text, when OpenDecision-Small was weak. It did not pay with many labels, or when the labels did not follow from the text.
Macro F1 out of 100 on 200 rows the model never trained on, measured on 2026-11-03. Each
job had 4 examples and up to 2,300 rows of text, and each row above is one run. Seeds move
these numbers: on the same rows across 2 seeds and our batching variants, Spanish read 64 to
66 and Financial sentiment 73 to 77. None of this is customer data, so test on your own.
Give it answers you already know
Answer 20 rows you know. After you add a file, the form shows its rows one at a time. You can also type answers under Add details, or label 20 rows in a CSV. Rows the form shows count for more: they also tell us how common each answer is in your file, which typed rows cannot. On web logs with no labels (CSIC 2010), scored on 1,500 hidden true answers, a model with no answers scored 63.91 and 65.23 out of 100 on our earlier training. With 20 answered rows on the current training, 8 runs scored 69.79 to 94.71, and half scored above 89.56. We picked those 8 to hold the 5 worst of 20 runs. In one real job through the form, a model with 20 answers caught 85% of attacks and blocked 1.8% of normal traffic. Since then we also check how often the AI’s rules give each answer against how often yours do, and the weakest of those 8 runs scored 86.68 and 87.89 in 2 of 3 new runs. Answer carefully. With 3 of 20 answers wrong, two models scored 3.5 and 11.6 points lower than the same runs with true answers. Results still swing from run to run, even with the same rows, so check a model on your own rows before it blocks anything. A model that gives the same answer to almost every row is never marked better, and you are not charged for it. Emotion and Financial sentiment barely moved.Create one
In the dashboard, open Fine Tune, then New fine tune. Owners and admins can start one. This step has no API.
A job took 3 to 10 minutes in our tests (13 jobs on public datasets, 8 running at once).
It can take longer when GPUs are busy. You can leave the page: we email the person who
started it when it finishes. Until then you can cancel it, and nothing is charged.
The result
Each of these leaves a model you can use or delete. A win means the whole 90% range of the
gain sits above 0, after taking off anything lost by shrinking the model to the file we
host. A job can also fail or come back as Needs input. Neither is charged.
Your text
Your text will be sent to RunInfra, Freestyle and Modal to build this model. It will train only your model. RunInfra runs the AI model that labels rows, Freestyle hosts a temporary machine for the job, and Modal runs the GPU. We hold your files and examples in memory until that machine has them. They do not reach our disk, our database or our logs. If our server restarts during a job, the job fails with no charge and you upload them again. We store the name, the question, the description, the labels and the scores.Run it on our servers
The model’s page has a Try it box, and this code with your model’s name filled in.decision, one per
text, in the same fields as /decide. Do not send decisions: the API
refuses it for a fine tune. A multi-label model has type: "multilabel", so read
selected, which can be empty. best names a label even then.
The model runs on our API server’s own CPU. All hosted fine tunes share one process, which
runs one text at a time, so for steady high volume run the model on your machine. A model
unloads after 10 idle minutes, and its next call takes a moment longer. When we are busy or
a model is still loading, you get a 503 with Retry-After in seconds. The SDKs carry that
wait in retryAfter and retry_after, and never retry for you, because a retry could bill
twice.
Median time through the API on one laptop (Ryzen 5 7535HS, Linux container, 4 CPUs, 2
threads), the slower of two runs, not on our servers, and with no network or queue. The
first request to a model took about 280 ms there, or about 580 ms if the process had to
start too.
The model reads at most 512 tokens of a text, and the whole text still bills. From 32 labels
up, a long text needs more memory than hosted allows, so the API answers
413 input_too_large with the longest text that fits. Running on your machine has no such
limit.
Run it on your machine
One word changes, and the result has the same shape. The call is free and the text stays on your machine.~/.cache/tokz and verifies
its hashes, so it needs the network once. Set TOKZ_CACHE_DIR to put the cache elsewhere.
After that it works offline, and it asks the server about once an hour whether the model
still exists, waiting at most about a second. usage reads 0 and id starts with local_.
The Node packages take about 320 MB on disk (Linux x64). A loaded model took about 300 MB of
memory after one short text, and a long text at 28 labels added about 1.3 GB (Node on the
same laptop).
Local runs the same model file as hosted, the file the score describes. On 3 real models and
185 test texts each, every score was within 0.000001 of our reference script and no answer
changed. That covers Node 20 and 22 on Windows, Node 20 on Linux, Python 3.9, 3.10, 3.12 on
Windows and Python 3.12 on Linux, all on one AVX2 laptop, and our server on 2 AMD AVX2 machines.
On an Intel CPU with AVX-512 VNNI, scores moved by up to 0.1 from that laptop’s, in our server
and our reference script alike, and 14 of 185 answers changed on a 28-label model, 1 on a
10-label model and none on a 2-label model. So a model can answer a few texts differently from
one machine to another, most often with many labels. macOS and arm64 have not been tested. decideLocal runs only on Node.js, and onnxruntime-node has
no build for Alpine or Intel Macs, so use decide there. Python needs onnxruntime at 1.26.0
or older, because newer versions change the scores. tokz[local] keeps it there.
Download
On the model’s page, Download model gives you a.tar of every model file: about 47 MB
on the 3 models we measured, and more for a model trained on long text. Any member of the
organization can download it.
predict.py runs hosted.onnx. Add download after the text to run download.onnx, which
is smaller and is not the file we scored: the two answered differently on 3, 31 and 100 of
185 mixed test texts across 3 models, and at 77 labels the small one scored 61 against 69 on
the same rows, in one run.
Delete
On the model’s page, choose Delete. Owners and admins can delete. A model stays until you do.- Hosted calls stop right away, and the files leave our servers.
decideLocalanddecide_localstop, and remove their cached copy, the next time they check with us, about once an hour, once the machine can reach us. A machine that never reaches us keeps running its copy.- A
.taryou downloaded keeps working. - We erase the question, labels and description. We keep the name, the scores and any charge, which stays in your spend. You can use the name again.
Prices
A hosted call whose client disconnects first is billed for the texts that already ran, with the
same minimum. Texts that had not run are not billed.
Starting any fine tune needs $2.00 in your balance. A job with a file holds $2.00 while it
runs. The hold counts as spent, so a low balance can get a
402 on /decide until the hold
ends. See Billing.
A hosted call bills the text’s tokens plus the question’s, once per 16 labels, plus the label
tokens. Above 16 labels all of that bills at 3x, so at 77 labels a text bills about 15 times
its tokens.
Limits
The dashboard says which limit you hit and what to do next. There is no API to list models:
GET /v1/decide/models shows only the catalog, and GET /v1/decide/usage shows a hosted fine
tune under an id starting ft_.
Errors
Nothing is billed for any of these. Local mode’s errors are on the
Troubleshooting page.