- 1
Does it exist?
We look the paper up in Crossref and OpenAlex.
- 2
Are the details right?
Title, year and first author must match the real record.
- 3
Does it say that?
An AI model reads the abstract and the sentence that cites it.
Check a text
Paste a text with citations like [1] and a References list, or pick an example.
What we found
Loading results...
How we measured it, and the limits
We build a test set from real papers (a random sample from OpenAlex), so the right answer is known for every case. Each case is a short text with one citation.
| Case type | What is wrong |
|---|---|
| correct | Nothing. Used to count false alarms. |
| fabricated | An invented reference: made-up title, and a made-up DOI in half the cases. |
| wrong_details | A real paper with a wrong year, a wrong first author, or a real DOI with another paper's title. |
| misused | A real paper with right details, but about something else than the sentence. |
| contradicted | A real paper with right details, but the sentence says the opposite of its abstract. |
Every case is checked once. We then ask what a simpler checker would have flagged from the same run: level 1 only, levels 1 and 2, or all three.
Limits
- The judge reads the abstract, not the full paper. When a publisher shares no abstract, the citation shows as "cannot verify" instead of a guess.
- The test cases are generated from real papers, not collected from real AI answers.
- The set is small. Read the confidence intervals, not only the percentages.
- Free-tier models can change. Lookup errors are counted separately and never hidden.