Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up

openJev-verdict-2.0

Verdict: 151M ModernBERT/GLiClass non-autoregressive decision models with calibration heads, reporting wins over Jev and Laya.

Modelresearch
View on GitHub ↗

Overview

openJev-verdict-2.0 contains two models. Verdict is a 151M general-purpose decision model based on ModernBERT-base and GLiClass, evaluated on JevBench and published on Hugging Face, now at v1.4 through inference-engine fixes: automatic calibrator loading across all candidate counts, NLI-style hypothesis templating for labels, and a context budget cut from 1,024 to 512 tokens, with byte-identical weights. Verdict 2.0 is a specialised architecture for typed software workflows evaluated on LocalLLaMA/typed-decisions.

Per the list summary, Verdict 2.0 reports 77.10% accuracy, 0.0636 Brier and 0.0144 ECE on typed-decisions, beating Jev and Laya, though results are project-local. It has separate distribution and confidence heads, saved evaluation artifacts and an in-browser WebGPU demo. One list summary notes the LICENSE file is not recognised as the Apache 2.0 its badge claims. It suits builders who want a tiny, calibrated decision model.

What the decision model decides

TypeQuestionAnswers
ChoiceWhich candidate label fits the state?—
Yes / noIs the statement true of the state?yes · no

How it works

  1. Load Verdict (151M) or Verdict 2.0.
  2. Format candidates as hypothesis sentences.
  3. Run one non-autoregressive pass with calibrated temperatures.
  4. Read distribution and confidence heads.

Strengths

  • Very small model; runs in the browser via WebGPU.
  • Separate confidence head.
  • Saved evaluation receipts.

Caveats

  • Results are project-local.
  • Verdict 2.0 weights tracked via Git LFS pointers.
  • Licence file mismatch reported.

Getting started

Download heman10x/rlcd-modernbert-151m from Hugging Face, or try the in-browser WebGPU demo.

At a glance

Type
Model
Status
research
Language
Python
Runtime
PyTorch, WebGPU browser
Has code
yes
Has tests
yes

Use cases and integrations

How curated lists describe it

  • Open decision engine: a calibrated 151M non-autoregressive model that reports beating both TypeSafe Jev and Laya on typed-decision benchmarks, shipped with its own test suite.

    yibie/awesome-jev

  • Calibrated 151M Non-Autoregressive Decision Engine beating TypeSafe Jev & Laya…

    wh000wh000/awesome-jev-live

  • ModernBERT-based decision model with separate distribution and confidence heads, saved evaluation artifacts, public weights, and an in-browser WebGPU demo; reported results are project-local.

    cobanov/awesome-jev

  • Calibrated 151M Non-Autoregressive Decision Engine beating TypeSafe Jev & Laya on LocalLLaMA/typed-decisions (77.10% acc, 0.0636 Brier, 0.0144 ECE).

    hellogumbo/awesome-jev

  • A 151M non-autoregressive decision model on ModernBERT with calibrated uncertainty and an in-browser WebGPU playground. Its LICENSE is not recognised as the Apache 2.0 its badge claims.

    MrJev/awesome-jev

Frequently asked questions

How big is Verdict?

151M parameters on ModernBERT-base and GLiClass.

What changed in v1.4?

Inference fixes only: calibrator loading, hypothesis templating and a 512-token budget.

Does it beat Jev?

The project reports so on typed-decisions, but results are project-local.

Related projects

  • decision-model-audit

    Pre-registered field test of Jev, GLiNER2.5-Decide, Laya and other decision models against small LLMs on fresh September 2026 data.

    ProjectresearchPython
  • open_system_one

    Open System-1 (OpenJev): Apache-2.0 research models giving calibrated per-option probabilities, best is a 4.9B exp9a.

    ProjectresearchPython
  • typed-decisions

    Lightweight reproduction of the Jev shape on a ModernBERT encoder and a diffusion LLM, comparing speed, accuracy, calibration and cost.

    ProjectresearchPython
  • Intern-Decision

    InternLM's open multimodal decision models (0.8B-4B) trained on Qwen3.5, with calibration tooling and a 96-case calibration benchmark.

    ModelresearchPython
  • qwen-rlcd-jev-repro

    Reproducible MLX and Colab recipes that turn Qwen decoders into Jev-like typed decision models, from 0.5B on a laptop to a noncausal 27B.

    ModelresearchPython
  • Verdict-open-jev★ 110

    OpenJev: 151M ModernBERT decision model with RLCD-style calibration, a JevBench audit and an in-browser WebGPU playground.

    BenchmarkresearchPython