jsort
sort where the key is a description: Jev judges pairs of texts and a Bradley-Terry fit turns them into a scored ranking.
Overview
jsort sorts lines or whole documents along a dimension you describe in plain English, such as 'more hawkish about inflation'. It shows Jev two texts at a time and asks which ranks higher; Jev answers with a probability in about 200 ms. Rather than comparing every pair, jsort asks about a few pairs per text over several rounds, fits a Bradley-Terry scale locally, and prints texts from the top of the scale down. With -o, each line carries a score and standard error, so the order can be used as a measurement rather than just a ranking.
The README's example ranks 51 sentences from an FOMC press conference opening statement using 255 comparisons in 7.2 seconds for $0.0037, with a reliability of 0.96 and a reported first-position lean, and then ranks 95 FOMC statements from 2011 onward as whole documents. Answers are cached, so reversing or re-running is free. It suits researchers measuring text along a construct, and anyone who wants to order items by a fuzzy criterion.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Does text A rank higher than text B on the described dimension? | — |
How it works
- Provide texts and a dimension description
- jsort schedules pairwise comparisons over several rounds
- Jev returns a probability for each pair
- A Bradley-Terry model is fitted locally
- Texts print in order with scores, errors and run statistics
Strengths
- Scores with standard errors, not just ranks
- Far fewer comparisons than all pairs
- Reports reliability and position bias
- Caching makes reruns free
Caveats
- Pairwise judgments can carry order bias, which is reported
- Long documents need truncation limits
- Requires an API key
Getting started
jsort -o "more hawkish about inflation" presser.txtAt a glance
- Type
- Project
- Status
- beta
- Language
- Python
- Runtime
- CLI
- Has code
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
Text measurement: ranks text along a plain-English criterion using pairwise Jev Noul comparisons and a locally fitted Bradley-Terry scale.
yibie/awesome-jevRanks text along a plain-English criterion using pairwise Jev Noul comparisons and a locally fitted Bradley-Terry scale.
logicrw/awesome-jev-projectsSort by meaning: order lines along a plain-English dimension, from pairwise comparisons judged by TypeSafe's Jev model.
hellogumbo/awesome-jevSorts lines along a plain-English dimension by judging them in pairs, so the sort key is a description rather than a field.
MrJev/awesome-jev
Frequently asked questions
How does jsort rank?
It asks Jev which of two texts ranks higher on your dimension and fits a Bradley-Terry scale to the answers.
How much does it cost?
The README's 51-sentence example used 255 comparisons for $0.0037.
Can I trust the scores?
Each run reports reliability, standard errors and first-position lean so you can judge.
Related projects
Checks whether each cited paper supports its citing sentence: code fetches papers, Claude proves a quote, Jev scores it, a human decides.
ServiceproductionClaude Code plugin that logs your prompts locally and later has Jev score how well you prompt the coding agent over time.
PluginbetaTypeScriptIndependent, source-linked research on Jev with about 948 rubric-scored public repositories, recurring patterns and verified findings.
ProjectresearchWorking paper placing a text judge beside a live trading system as a read-only signal, comparing Jev and Claude as blind headline judges.
PaperresearchSecure, persistent VM sandboxes for AI agents, with a PR-review example that compares a general model with Jev on candidate findings.
ProjectproductionPythonReal-time global intelligence dashboard; per the lists, it batches Jev questions to score headline severity and classify event domains.
AppproductionTypeScript