I benchmarked TypeSafe's JEV against LLMs, BERT and Laya
Reddit post benchmarking Jev against generative LLMs, a BERT classifier and LAYA on the same task.
This summary is based on how curated lists describe the project; its README was not available to review.
Overview
This is a Reddit post in r/AI_India in which the author compares TypeSafe's Jev with generative LLMs, a classical BERT classifier and LAYA, an open Jev-style decision model, on a shared task. The yibie/awesome-jev list notes it had 49 upvotes at the time of listing.
It is useful to readers deciding between a decision model, a prompted LLM and a fine-tuned classifier, because it puts all three families side by side. The list summary does not report the task, metrics or results, and the post itself was not fetched, so findings should be read at the source.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Other | Compares accuracy and cost of Jev, LLMs, BERT and LAYA on one classification task(inferred) | — |
How it works
- Pick a shared classification task
- Run Jev, several LLMs, a BERT model and LAYA on it
- Compare results across model families
Strengths
- Places a decision model next to both generative and classical baselines
Caveats
- Informal community benchmark; methodology not reviewed here
- Results not summarised in the source list
At a glance
- Type
- Discussion
- Status
- discussion
Use cases and integrations
How curated lists describe it
Reddit (49 upvotes): pits Jev against both generative models and a classical classifier on the same task.
yibie/awesome-jev
Frequently asked questions
What does the post compare?
Jev against generative LLMs, a BERT classifier and LAYA on the same task.
Where are the results?
In the Reddit thread; the directory lists do not summarise them.
Is it a rigorous benchmark?
It is a community post, so treat it as an informal comparison.
Related projects
Open-source BYOK arena where Jev faces another model as a judge on your question, with blind voting before costs are revealed.
ProjectbetaTypeScriptNovelty demo that turns Jev answers to 22 Boolean gate questions into lookup tables for a simulated RISC-V CPU running compiled C.
ProjectprototypeAn X post comparing Jev with a private fine-tuned classifier at matched precision.
DiscussiondiscussionAnthony Maio's Substack essay on what a model that cannot generate text is for, with a critical look at Jev's claims.
ArticlediscussionHacker News thread on Decision Index 0.2.1, which compared about 70 open reproductions of Jev's decision model.
DiscussiondiscussionVideo post on X comparing Jev with a structured-output Qwen baseline running on Cerebras.
Discussiondiscussion