Laya vs Jev: Accuracy, Latency and Cost, Compared
Both models answer the same three question types. The choice between them is about where the decision runs, what it costs at volume, and which one is better calibrated on your data.
Head to head
| Jev (hosted) | Laya (local) | |
|---|---|---|
| Access | HTTP API, model jev-latest / jev-1.13.0 | Open weights, pip / ONNX / MLX / Core ML |
| Size | Frontier-scale, operated by the vendor | ~421M parameters, 1–2 GB RAM |
| Answer types | choice, score, noul | choice, score, noul |
| Public benchmark | Reference implementation in the comparison | 0.766 routing accuracy vs 0.727 |
| Calibration | Calibrated by training (RLCD) | Calibrated per checkpoint; Brier ~0.062 reported |
| Latency | Network + inference; demoed at sub-second decisions | ~35 ms GPU, 7–14 ms MLX, ~5 ms Core ML |
| Marginal cost | ~$0.042 per million input tokens; output unmetered | Zero per call; you pay for hardware |
| Data residency | Requests leave your infrastructure | Nothing leaves the device |
| Benchmarking rights | Restricted by the hosted model's terms | Unrestricted |
| Fine-tuning | Not available | Yes, and quantisation is straightforward |
The accuracy row, read honestly
On the shared public decision benchmark, Laya scores 0.766 against 0.727 for the hosted model — a real gap, and one the maintainers publish themselves. Two caveats before you redesign anything: the eval set is small, shared and public, and a few hundredths of accuracy on a mixed benchmark often disappears once you condition on your own distribution. The numbers that should move your decision are accuracy and calibration on your labelled sample, next to p95 latency and the cost of the review queue your thresholds create.
Cost, in the units that matter
Community reports from launch week put ~5,000 mixed decision calls at about $2 with the hosted model. That is already cheap enough that cost alone rarely justifies a migration. The stronger arguments for local are structural: no request leaves the machine, no rate limit sits between you and a user action, and no per-call bill grows with traffic. The stronger arguments for hosted are operational: no inference cluster, no quantisation work, no model updates to babysit, and a vendor accountable for the answer.
The decision, in one paragraph
Prototype on the hosted model if you value speed to a working pipeline; move to Laya when the decision must be private, offline, free at the margin, or tunable. Keep both behind one internal function — the question set is the stable part, the model is a parameter you can change later. Projects doing exactly that are listed on the GitHub page.
Last updated: 2026-09-21 · sources & corrections