jev-arena
Side-by-side tool comparing Jev and DeepSeek on labelling the same comments for relevance, sentiment and intent, with replay and reports.
Overview
Jev Arena by NanmiCoder is a Chinese-language web tool that runs the same batch of comments through two models and compares speed, cost and labels. The default pairing is Jev versus DeepSeek, and any OpenAI-compatible chat model can be configured. It supports CSV and Excel import, a live duel view, real-speed replay of saved runs without calling models, and offline HTML reports that need no keys.
The README reports a 10,000-comment test labelling relevance, sentiment and intent, audited by GPT-6 Astra with a 400-item re-label sample. Jev 1.13 took 203.2 s and $0.84 versus 823.5 s and an estimated $1.50 for DeepSeek Flash, with accuracies of 94.70% vs 96.26% (relevance), 82.91% vs 84.54% (sentiment) and 77.90% vs 80.33% (intent); all three correct at once was 62.69% vs 67.26%. The author stresses this is an AI-referenced audit, not a human gold standard, and applies only to that data and configuration. It has tests and is MIT licensed.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this comment relevant? | yes · no |
| Choice | What is the sentiment of this comment? | — |
| Choice | What is the commenter's intent? | — |
How it works
- Import comments from CSV or Excel
- Run Jev and a chat model side by side
- Compare speed, cost and labels live
- Replay saved runs and generate offline reports
Strengths
- About 4x faster and cheaper than DeepSeek in the README's test
- Transparent audit files and per-item checks
- Replay without new model calls
Caveats
- Slightly lower accuracy than DeepSeek in the reported test
- Audit used an AI reference, not human labels
- Documentation in Chinese
Getting started
`git clone https://github.com/NanmiCoder/jev-arena.git && cd jev-arena && npm ci && npm start`, then open localhost:5173 and enter OpenRouter and DeepSeek keys.At a glance
- Type
- Project
- Status
- beta
- Language
- TypeScript
- Runtime
- Node web app
- Has code
- yes
- Has tests
- yes
- License
- MIT
- Stars
- 122
Use cases and integrations
How curated lists describe it
Jev 模型介绍与实测：通过 Choice / Score / Noul 将自然语言转为带类型的判断与概率，用于分类、评分和路由；支持与 DeepSeek 等模型对比评论打标、速度与结果，含 CSV/Excel 导入、原速回放与离线报告。
wh000wh000/awesome-jev-live
Frequently asked questions
What does the tool compare?
Jev and another chat model labelling the same comments for relevance, sentiment and intent.
Which was more accurate?
DeepSeek Flash slightly, but Jev was about 4x faster and cheaper in the README's 10,000-comment test.
Do replays cost money?
No, replays do not call models.
Related projects
Live race labelling 1,000 app reviews with four typed Jev questions versus Gemini 3.8 Flash JSON output.
AppprototypeJavaScriptGoodWatch movie and TV recommendation monorepo, listed for an experiment comparing Jev question designs for attribute scoring.
ProjectresearchTypeScriptJev applied to SemEval-2026 DimABSA dimensional sentiment tasks using only typed primitives, no generation or fine-tuning.
ProjectresearchPythonBun harness that runs Jev over datasets via OpenRouter, streams JSONL answers, and checks calibration, stability and regressions.
LibrarybetaTypeScriptJapanese demo judging the same utterance with Jev and an LLM to compare emotion-shift outputs and speed side by side.
ServiceprototypeTypeScriptChinese 'emotion lie detector' demo comparing Jev's Choice, Score and Noul judgments on a message with a general LLM.
CLIprototypePython