# Vectara Hallucination Leaderboard

Asks AI models to summarize short documents and uses an automated checker to count how often the summary adds things that are not in the original text. It shows how likely a model is to make things up when it should just be faithful to a source.

Category: factuality. Type: leaderboard. Status: active.
Metric: Share of summaries containing information not supported by the source document (hallucination rate, %) (lower is better).
Source: Vectara — https://github.com/vectara/hallucination-leaderboard/blob/main/README.md, results as of 2026-10-22, retrieved 2026-11-17, license: Apache-2.0 (repository license).

| Rank | Model | Hallucination Rate | Factual Consistency Rate | Answer Rate | Average Summary Length (Words) |
|---|---|---|---|---|---|
| 1 | antgroup/finix_s1_32b | 1.8 % | 98.2 % | 99.5 % | 172.4 |
| 2 | openai/gpt-5.4-nano-2026-03-17 | 3.1 % | 96.9 % | 100.0 % | 144.4 |
| 3 | google/gemini-2.5-flash-lite | 3.3 % | 96.7 % | 99.5 % | 95.7 |
| 4 | microsoft/Phi-4 | 3.7 % | 96.3 % | 80.7 % | 120.9 |
| 5 | meta-llama/Llama-3.3-70B-Instruct-Turbo | 4.1 % | 95.9 % | 99.5 % | 64.6 |
| 6 | deepseek-ai/DeepSeek-V4.1-Flash | 4.2 % | 95.8 % | 99.9 % | 108.2 |
| 7 | snowflake/snowflake-arctic-instruct | 4.3 % | 95.7 % | 62.7 % | 81.4 |
| 8 | google/gemma-3-12b-it | 4.4 % | 95.6 % | 97.4 % | 89.7 |
| 9 | mistralai/mistral-large-2411 | 4.5 % | 95.5 % | 99.9 % | 85.0 |
| 10 | qwen/qwen3-8b | 4.8 % | 95.2 % | 99.9 % | 83.6 |
| 11 | amazon/nova-pro-v1:0 | 5.1 % | 94.9 % | 99.3 % | 66.2 |
| 12 | amazon/nova-2-lite-v1:0 | 5.1 % | 94.9 % | 99.6 % | 94.1 |
| 13 | mistralai/mistral-small-2501 | 5.1 % | 94.9 % | 97.9 % | 98.8 |
| 14 | ibm-granite/granite-4.0-h-small | 5.2 % | 94.8 % | 100.0 % | 107.4 |
| 15 | google/gemma-4-26b-a4b-it | 5.2 % | 94.8 % | 99.8 % | 67.1 |
| 16 | ai21labs/jamba-mini-2 | 5.3 % | 94.7 % | 99.6 % | 109.4 |
| 17 | deepseek-ai/DeepSeek-V3.2-Exp | 5.3 % | 94.7 % | 96.6 % | 64.6 |
| 18 | qwen/qwen3-14b | 5.4 % | 94.6 % | 99.9 % | 111.1 |
| 19 | amazon/nova-micro-v1:0 | 5.5 % | 94.5 % | 100.0 % | 100.0 |
| 20 | deepseek-ai/DeepSeek-V3.1 | 5.5 % | 94.5 % | 94.5 % | 63.7 |
| 21 | openai/gpt-5.4-mini-2026-03-17 | 5.5 % | 94.5 % | 100.0 % | 54.7 |
| 22 | openai/gpt-4.1-2025-04-14 | 5.6 % | 94.4 % | 99.9 % | 91.7 |
| 23 | qwen/qwen3-4b | 5.7 % | 94.3 % | 99.9 % | 104.7 |
| 24 | xai-org/grok-3 | 5.8 % | 94.2 % | 93.0 % | 95.9 |
| 25 | qwen/qwen3-32b | 5.9 % | 94.1 % | 99.9 % | 115.8 |

Notes: Top 25 by lowest hallucination rate, in README order. Judged by Vectara's own HHEM classifier on its current (new) document set; covers summarization faithfulness only. Answer rate = share of documents the model agreed to summarize.
