Jev vs Fable on 520 real social posts
Blog review of a scheduler's pre-publish check where Jev answers four yes/no questions per social caption, compared with blind Fable labels.
This summary is based on how curated lists describe the project; its README was not available to review.
Overview
This SeenPaid blog post reviews Jev as an advisory pre-publish check in a social media scheduler. Each caption gets four Noul questions: is it spam, does it have a clear opening, does it stand alone, and is it promotional. The answers are surfaced as advisory signals and never block publishing.
According to the listing, on 100 posts labelled blind by Fable, the two agreed on 94 of 100 for promotion and 85 of 100 for spam at a 0.65 threshold, with Jev the stricter one in 12 of the disagreements versus 3. Scoring all 520 posts cost $0.011 at a 341 ms median latency. The article text was not available for review beyond the listing summary.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this caption spam? | — |
| Yes / no | Does this caption have a clear opening? | — |
| Yes / no | Does this caption stand alone? | — |
| Yes / no | Is this caption promotional? | — |
How it works
- Scheduler sends each caption to Jev with four Noul questions
- Signals shown to the user as advice, never a gate
- Compare with blind labels from another model
Strengths
- Listing reports $0.011 to score 520 posts at 341 ms median
- High agreement on promotional content
Caveats
- Reference labels are from another model, not humans
- Advisory only
- Article not reviewed in full
At a glance
- Type
- Article
- Status
- production
Use cases and integrations
How curated lists describe it
Social media: a scheduler's pre-publish check asks Jev four Noul questions per caption (spam, clear opening, stands alone, promotional) as advisory signals, never a gate; on 100 posts labelled blind by Fable the two agreed 94/100 on promotion and 85/100 at a 0.65 spam threshold (Jev the stricter one 12 times to 3), and scoring all 520 posts cost $0.011 at a 341 ms median.
yibie/awesome-jev
Frequently asked questions
Does the check block posts?
No, the answers are advisory signals only.
How well did it agree with Fable?
94/100 on promotion and 85/100 on spam at a 0.65 threshold, per the listing.
What did it cost?
$0.011 for 520 posts, per the listing.
Related projects
Comparative judgment of ASAP essays by Jev and two LLM judges, turning pairwise decisions into scores compared with human grades.
ServiceresearchPythonIndependent benchmark of Jev, constrained-output LLMs and baselines on intent and spam tasks: accuracy, latency, cost, calibration.
BenchmarkresearchPythonReproducible benchmark of Jev vs Claude Haiku 4.5 on 2,000 emails deciding whether an agent should click a link, with calibration audit.
BenchmarkresearchPythonChrome extension that scores each X timeline post with one batched Jev Noul request for engagement value, virality and tags such as spam.
Browser extensionprototypeJavaScriptExploratory study of zero-shot ham, spam and phishing email classification with Jev, compared with trained TF-IDF baselines.
BenchmarkresearchPythonBenchmark asking Jev and a local LLM whether each collusion.wiki page was agent-written; neither beats the majority baseline.
BenchmarkresearchPython