Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up

typed-decisions

Lightweight reproduction of the Jev shape on a ModernBERT encoder and a diffusion LLM, comparing speed, accuracy, calibration and cost.

Projectresearch
View on GitHub ↗

Overview

typed-decisions reproduces the shape of Jev: one state, many typed questions (Choice, Score, Noul), and calibrated probability distributions returned in a single parallel pass. It does not implement a TypeSafe spec, since none is public; the API shape is reconstructed from TypeSafe's site and a guide. Every question type is a softmax over an option list, with score read as an expected level and noul as p(yes).

It trains the same task on two backbones so they can be compared side by side: a ModernBERT encoder (149M or 395M) with a matching head over option tokens, and LLaDA-MoE, a 7B diffusion LLM with about 1B active parameters, using LoRA and reading logits at mask slots. Both use cross-entropy plus Brier loss and have zero structured-output errors by construction. Labels come from public datasets such as Banking77, with no synthetic answers or teacher model. It is aimed at researchers.

What the decision model decides

TypeQuestionAnswers
ChoiceWhich intent does this banking query express (Banking77 labels)?—
ScoreScore question read out as an expected level—
Yes / noYes/no question read out as p(yes)yes · no

How it works

  1. Build a corpus from public labeled datasets.
  2. Train an encoder head and a LoRA diffusion LLM on the same typed questions.
  3. Answer all questions for a state in one forward pass.
  4. Fit temperature on validation and compare speed, accuracy and calibration.

Strengths

  • Side-by-side comparison of encoder and diffusion backbones.
  • No synthetic labels or teacher model.
  • Zero structured-output errors by construction.

Caveats

  • Research code tied to the author's Modal setup.
  • Not a drop-in Jev replacement.

At a glance

Type
Project
Status
research
Language
Python
Runtime
PyTorch training on Modal
Has code
yes

Use cases and integrations

How curated lists describe it

Frequently asked questions

Is this a Jev clone?

It reproduces Jev's input and output shape on open backbones, without claiming to match its internals.

Which models are compared?

ModernBERT base and large encoders versus the LLaDA-MoE diffusion LLM.

Where do labels come from?

Public datasets such as Banking77, using their own labels.

Related projects

  • decision-model-audit

    Pre-registered field test of Jev, GLiNER2.5-Decide, Laya and other decision models against small LLMs on fresh September 2026 data.

    ProjectresearchPython
  • open_system_one

    Open System-1 (OpenJev): Apache-2.0 research models giving calibrated per-option probabilities, best is a 4.9B exp9a.

    ProjectresearchPython
  • Intern-Decision

    InternLM's open multimodal decision models (0.8B-4B) trained on Qwen3.5, with calibration tooling and a 96-case calibration benchmark.

    ModelresearchPython
  • openJev-verdict-2.0

    Verdict: 151M ModernBERT/GLiClass non-autoregressive decision models with calibration heads, reporting wins over Jev and Laya.

    ModelresearchPython
  • qwen-rlcd-jev-repro

    Reproducible MLX and Colab recipes that turn Qwen decoders into Jev-like typed decision models, from 0.5B on a laptop to a noncausal 27B.

    ModelresearchPython
  • Verdict-open-jev★ 110

    OpenJev: 151M ModernBERT decision model with RLCD-style calibration, a JevBench audit and an in-browser WebGPU playground.

    BenchmarkresearchPython