jev-dimabsa
Jev applied to SemEval-2026 DimABSA dimensional sentiment tasks using only typed primitives, no generation or fine-tuning.
Overview
Jev × DimABSA applies Jev to SemEval-2026 Task 3 Track A, dimensional aspect-based sentiment analysis, which replaces polarity with continuous valence and arousal on a 1 to 9 scale. Every system is built from Jev's three primitives (Score for scale positions, Choice and Noul for options and yes/no), a few training-split statistics and the unchanged official scorer, with no text generation, fine-tuning or GPU.
The README reports results from single frozen runs on the official test split after choices were fixed on train and dev. For Task 1 (valence and arousal for a given aspect) it reports 1.0645 RMSE across 10 corpora, the lowest micro aggregate among the 14 teams covering all corpora. For Task 2 (aspect, opinion and VA triplets) it reports 52.09 cF1, between 6th and 7th of 12 teams, and for Task 3 (adding category) 44.06 cF1, between 5th and 6th of 9 teams and above a fine-tuned Llama-3.3-70B baseline. Experiment logs document the protocols.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Score | What valence and arousal does this aspect carry on the 1 to 9 scale? | — |
| Choice | Which category does this aspect belong to?(inferred) | — |
| Yes / no | Is this span an aspect or opinion term?(inferred) | — |
How it works
- Load DimABSA reviews and aspects
- Ask Jev Score questions for valence and arousal, plus Choice and Noul for extraction tasks
- Adjust with train-split statistics
- Score with the official scorer on a frozen test run
Strengths
- Competitive results without training or GPUs
- Single frozen runs with documented protocols
- Uses the official scorer unchanged
Caveats
- Extraction tasks rank mid-table
- Results specific to this shared task
Getting started
See the Quick start section of the README.At a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- Scripts
- Has code
- yes
Use cases and integrations
How curated lists describe it
TypeSafe Jev (System One) on DimABSA, SemEval-2026 Task 3 Track A, all three…
wh000wh000/awesome-jev-liveTypeSafe Jev baseline for DimABSA (SemEval-2026 Task 3) subtask 1: zero-shot and 3-shot valence-arousal regression.
hellogumbo/awesome-jev
Frequently asked questions
What does Jev decide?
Valence and arousal scores for aspects, plus extraction and category decisions for the triplet tasks.
How did it rank?
The README reports the best 10-corpus micro RMSE on Task 1 and mid-table F1 on Tasks 2 and 3.
Was any model fine-tuned?
No, it uses Jev zero-shot or few-shot with train-split statistics.
Related projects
- jev-arena★ 122
Side-by-side tool comparing Jev and DeepSeek on labelling the same comments for relevance, sentiment and intent, with replay and reports.
ProjectbetaTypeScript FGV Direito SP experiment comparing Jev with Gemini 3.8 Flash and GPT-5.6 Luna for annotating 12 variables in 120 TJSP court rulings.
ProjectresearchPythonJapanese demo judging the same utterance with Jev and an LLM to compare emotion-shift outputs and speed side by side.
ServiceprototypeTypeScriptChinese 'emotion lie detector' demo comparing Jev's Choice, Score and Noul judgments on a message with a general LLM.
CLIprototypePythonBenchmark comparing Jev, other decision models and LLMs on four judgments over 60 synthetic candidate-experience reviews.
BenchmarkresearchBenchmark of Jev 1.13.0 against 11 classical classification pipelines on eight datasets under a published protocol.
BenchmarkresearchPython