Asks AI models to summarize short documents and uses an automated checker to count how often the summary adds things that are not in the original text. It shows how likely a model is to make things up when it should just be faithful to a source.
Leaderboard
Top 3 right now
1stantgroup/finix_s1_32b1.8 %
2ndopenai/gpt-5.4-nano-2026-03-173.1 %
3rdgoogle/gemini-2.5-flash-lite3.3 %
What it measures
Hallucination rate when models summarize short documents, judged by Vectara's HHEM classifier.
How to read the numbers
Share of summaries containing information not supported by the source document (hallucination rate, %) — lower is better.
Note: Top 25 by lowest hallucination rate, in README order. Judged by Vectara's own HHEM classifier on its current (new) document set; covers summarization faithfulness only. Answer rate = share of documents the model agreed to summarize.
Source: Vectara · github.com · results as of 2026-10-22 · retrieved 2026-11-17 · license: Apache-2.0 (repository license)