pijev
pijev: drop-in wrapper for the TypeSafe Python SDK that averages Jev answers across option orders in one request.
Overview
pijev (Permutation Invariant Jev) addresses the fact that Jev's probabilities can change when the same options are reordered. The README shows a live jev-1.13.0 run where billing was chosen either way but its probability moved from 0.43 to 0.63 between two orderings. pijev treats option order as a nuisance variable: it asks each question in several option orders within a single system_one request and averages the answers into one prediction.
Because averaging is a convex combination, Brier score and log loss are guaranteed to be no worse than the average across the included orderings. It is a drop-in wrapper for the official Python SDK with a one-line import change, keeping the same calls and official response types. It is Apache-2.0 licensed and includes experiment details.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which team should handle this support ticket? | billing · technical · account |
How it works
- Import pijev instead of the SDK client.
- Each Choice is sent in several option orders in one request.
- Probabilities are averaged across orderings.
- Return official response types with the averaged result.
Strengths
- Reduces sensitivity to option order.
- Guaranteed no worse Brier and log loss than the average ordering.
- One-line change, one request.
Caveats
- More questions per request increases input tokens.
- Python only.
Getting started
pip install pijev and change the import from typesafe_sdk to pijev.At a glance
- Type
- Library
- Status
- beta
- Language
- Python
- Runtime
- library
- Has code
- yes
- License
- Apache-2.0
Use cases and integrations
How curated lists describe it
pijev turns Jev's option-order variations into a single averaged prediction, with Brier score and log loss guaranteed no worse than the average across the included orderings. All permutations run in one \`system\_one\` request. Keep the same calls and official response types; pijev handles the rest.
logicrw/awesome-jev-projectsDrop-in wrapper for the official Python SDK that asks each question in several option orders and averages the answers, so a decision does not move when the options are shuffled.
MrJev/awesome-jev
Frequently asked questions
Why does option order matter?
The README shows Jev's probability for the same option shifting by 20 points when options were reordered.
Does it add requests?
No, all permutations run in one system_one request.
What guarantee does it give?
Brier score and log loss no worse than the average across included orderings.
Related projects
Decision-forensics library that explains a Jev answer by ablating parts of the input and measuring how far the probability moves.
ProjectbetaPythonAdapter that turns existing LLMs into Jev-style decision functions with option scores, optional calibration and review thresholds.
ProjectprototypePythonDecision sidecar and Copilot CLI pre-turn hook that mimics Jev choice, noul and score using Azure Foundry Cohere Rerank.
HookprototypePythonExact-target audit finding Jev's Choice probabilities are a step function on stated base rates while per-option Noul tracks them closely.
ProjectresearchPythonResearch replicating Jev-style typed decisions with open LLMs via logprobs; Qwen3.5-9B reached 94% with good calibration in ~1 s.
ProjectresearchPythonRunnable yardstick that tests Jev's three headline claims on a small local model: typed output, no hallucination, calibration.
ProjectresearchPython