jev-paper-judge
Upload a paper PDF and get a fast Jev-based judgment of understandability and completeness, with low-confidence answers flagged.
Overview
Jev Paper Judge gives fast feedback on a paper's presentation. You upload a PDF and get a structured judgment of understandability (sentence clarity, defined terms and symbols, signposting, logical flow) and completeness (problem, motivation, contributions, background, reproducible method, setup, interpreted results, limitations, plus mechanical checks such as undefined acronyms and unreferenced figures). It explicitly measures presentation, not correctness.
The browser extracts text with pdf.js and splits it into sections and passages; a Cloudflare Worker sends a paper-level state with 13 questions and each passage with its own questions, including one per term and two per paragraph, all in parallel. Scores are visible arithmetic over the answers, and low-confidence answers are marked rather than smoothed over. The README says a 16-page paper returns in two to three seconds. The rubric and API key stay on the server, and the endpoint only accepts documents, so it cannot be used as a generic proxy.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Are the symbols in this passage defined when they first appear? | yes · no |
| Yes / no | Does the paper state its limitations?(inferred) | yes · no |
| Score | How clear are the sentences in this paragraph?(inferred) | — |
How it works
- Upload a PDF; the browser extracts and segments text
- The Worker sends paper-level and per-passage questions in parallel
- Jev returns probabilities for each question
- Code computes scores, applies confidence routing and builds a report
Strengths
- Seconds per paper, enabling quick edit-and-recheck loops
- Transparent scoring arithmetic
- Uncertain answers are flagged
- Server-side rubric prevents misuse as a proxy
Caveats
- Judges presentation only, not correctness
- Hosted dependency on TypeSafe
- PDF extraction quality affects results
At a glance
- Type
- Demo
- Status
- prototype
- Language
- TypeScript
- Runtime
- Cloudflare Worker plus browser front end
- Has code
- yes
Use cases and integrations
How curated lists describe it
Feedback on your paper in seconds.
hellogumbo/awesome-jev
Frequently asked questions
Does it check whether my paper is right?
No. It measures understandability and completeness only.
How fast is it?
The README says a 16-page paper comes back in two to three seconds.
How are scores computed?
As visible arithmetic over Jev's per-question probabilities in the rubric file.
Related projects
Analytics engine that rates published posts on pre-registered rubrics with Jev and tests ratings against real engagement.
ProjectbetaTypeScriptDocument evaluation CLI and CI gate using Jev or OpenAI Decisions, with pre-registered validation and documented reliability limits.
CLIbetaPythonRussian-language Letta agent skill that uses Jev as a fast external judge, with rules for treating mid-range probabilities as 'unsure'.
ProjectprototypePythonWriting checker running separate named Jev checks such as hedging, filler, tone and passive voice, each with its own verdict.
DemobetaTypeScriptCommunity tool that fetches a SaaS homepage as Markdown and asks Jev ten questions about first-visit clarity, with a leaderboard.
ServicebetaTypeScriptComparative judgment of ASAP essays by Jev and two LLM judges, turning pairwise decisions into scores compared with human grades.
ServiceresearchPython