Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up

citation-verifier

Checks whether each cited paper supports its citing sentence: code fetches papers, Claude proves a quote, Jev scores it, a human decides.

Serviceproduction
View on GitHub ↗Live demo ↗

Overview

Paper Trellis Citation Verifier is a free tool for peer reviewers and authors. Upload a manuscript and it extracts the reference list, pairs each citing sentence with its reference, fetches the cited paper and checks the pair two ways, after which you record your own accept, reject or other verdict and export the table.

Code parses the .docx or PDF, matches citation markers and references (DOI, PMID, arXiv id or title via PubMed, Crossref and OpenAlex), and fetches open full text or the abstract, always saying which was checked. Claude proposes a verdict and a verbatim passage, and the app proves the passage exists in the fetched text. Jev then reads the sentence and passage and returns the probability that the passage supports the sentence, plus its confidence. Either model works alone; together, Jev scores the quote Claude proved. A hosted version is available.

What the decision model decides

TypeQuestionAnswers
Yes / noDoes this passage support the citing sentence?yes · no

How it works

  1. Parse the manuscript and match citations to references
  2. Fetch full text or abstract
  3. Claude proposes a verdict and quote; code proves the quote exists
  4. Jev scores support
  5. Human records the final verdict

Strengths

  • Quotes verified against fetched text
  • States whether full text or abstract was checked
  • Human makes the final call

Caveats

  • Abstract-only checks are weaker
  • Needs model keys for self-hosting

Getting started

Try https://verify.papertrellis.com

At a glance

Type
Service
Status
production
Runtime
web app
Has code
yes

Use cases and integrations

How curated lists describe it

  • Academic publishing: checks whether each cited paper actually supports the sentence citing it, with Claude locating the quote, Jev scoring the support, and a human making the final call.

    yibie/awesome-jev

  • Check whether each cited paper supports the sentence citing it. Claude proves the quote, TypeSafe's Jev scores it, a human decides.

    hellogumbo/awesome-jev

  • Checks whether each cited paper supports the sentence citing it: Claude finds the quote, Jev judges support with a probability you can read as a reliability mark. Hosted version available.

    MrJev/awesome-jev

Frequently asked questions

What does Jev decide?

How likely it is that the passage supports the citing sentence.

Who makes the final verdict?

You do; the tool records accept, reject or other.

Which papers can it fetch?

PubMed, Crossref and OpenAlex matches, with full text from PMC or arXiv when open.

Related projects

  • jsort

    sort where the key is a description: Jev judges pairs of texts and a Bradley-Terry fit turns them into a scored ranking.

    ProjectbetaPython
  • fake-real-jev

    Published Jev client and claim check from the Fake/Real evidence-review site, where Jev filters sources and judges each claim.

    ProjectproductionTypeScript
  • latitude-llm

    Agent observability platform that finds and fixes agent failures, with an optional Jev preclassifier for conversation checks.

    ProjectproductionTypeScript
  • jevtest

    Semantic test matchers for Vitest and Jest that ask Jev whether output satisfies a plain-English expectation and threshold the probability.

    ProjectbetaTypeScript
  • pytest-jev

    pytest plugin for semantic assertions on LLM output, asking Jev whether plain-English claims hold with a 0.8 probability bar.

    PluginbetaPython
  • jev-judge

    'Vitest for LLM outputs': CLI running YAML/JSON eval suites on Jev for RAG faithfulness, hallucination and agent safety in CI.

    CLIprototypeTypeScript