jev-trust
Python trust middleware for the Jev API that logs decisions, measures calibration from your recorded outcomes and signs the evidence.
This summary is based on how curated lists describe the project; its README was not available to review.
Overview
jev-trust is a Python package on PyPI described as trust middleware for the Jev API. According to the listing, it logs every typed decision, lets you record real outcomes, and measures calibration in your own domain using accuracy, Brier score, top-label ECE and a derived C = 1 - ECE metric.
It then annotates each answer with its measured effective confidence, fires alerts when the model appears overconfident relative to observed outcomes, and signs the evidence with ed25519 so others can recompute the figures independently. It targets teams who want to verify that Jev's probabilities hold up on their own data rather than relying on vendor calibration claims. No README was available, so installation details and API shape are unknown.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Other | Wraps any Jev decision and measures how well its confidence matches recorded outcomes | — |
How it works
- Log each typed Jev decision
- Record the real outcome later
- Compute accuracy, Brier, ECE and effective confidence
- Alert on overconfidence and sign the evidence
Strengths
- Domain-specific calibration measurement
- Signed, independently recomputable evidence
Caveats
- No README available to verify details
- Calibration needs enough recorded outcomes
Getting started
pip install jev-trustAt a glance
- Type
- Service
- Language
- Python
- Runtime
- Library
- Has code
- yes
Use cases and integrations
How curated lists describe it
Python ecosystem: trust middleware for the Jev API that logs every typed decision, measures calibration in your own domain from outcomes you record (accuracy, Brier, top-label ECE, C = 1 − ECE), annotates each answer with its measured effective confidence, fires overconfidence alerts, and signs the evidence (ed25519) for independent recomputation.
yibie/awesome-jev
Frequently asked questions
What does jev-trust measure?
Calibration of Jev answers against your recorded outcomes: accuracy, Brier, top-label ECE and C = 1 - ECE.
Why sign the evidence?
So others can independently recompute the calibration figures, per the listing.
Where is it published?
On PyPI as jev-trust.
Related projects
Monitoring proxy and client that records Jev answer probabilities and alerts on version changes, drift, edge crowding or accuracy drops.
ProxybetaPythonNautilus Compass agent memory layer whose jev-trust middleware logs Jev decisions and measures calibration in your domain.
ServicebetaPythonDecision-forensics library that explains a Jev answer by ablating parts of the input and measuring how far the probability moves.
ProjectbetaPythonTool that recomputes Jev's confidence identities on outputs published in public repositories to spot answers Jev never actually produced.
ServiceresearchPythonOffline Python toolkit that measures sensitivity, Fisher information and information loss in the probabilities of Jev-style agent systems.
ProjectresearchPythonpijev: drop-in wrapper for the TypeSafe Python SDK that averages Jev answers across option orders in one request.
LibrarybetaPython