multimodal benchmark
Vibe-Eval
VIBE-Eval is a hard evaluation suite for measuring progress of multimodal language models, consisting of 269 visual understanding prompts with gold-standard responses authored by experts. The benchmark has dual objectives: vibe checking multimodal chat models for day-to-day tasks and rigorously testing frontier models, with the hard set containing >50% questions that all frontier models answer incorrectly.
Updated Aug 7, 2026
Published models8
Registry coverage8
MetricScore
EvidenceB
Vibe-Eval leaderboard
Sorted by the source-provided rank. Higher score is better according to the registry.
| 1 | GO | 67.2% | 100.0% | 8 | C | |
| 2 | GO | 65.6% | 85.7% | 8 | C | |
| 3 | GO | 65.4% | 71.4% | 8 | C | |
| 4 | GO | 56.3% | 57.1% | 8 | C | |
| 5 | GO | 53.9% | 42.9% | 8 | C | |
| 6 | GO | 51.3% | 28.6% | 8 | C | |
| 7 | GO | 48.9% | 14.3% | 8 | C | |
| 8 | GO | 40.9% | 0.0% | 8 | C |
Score distribution
Top published rows on the benchmark's original scale.
Vibe-Eval highlights
The top published results on this benchmark's own scale.
What is Vibe-Eval?
Definition and scoring fields from the benchmark registry.
VIBE-Eval is a hard evaluation suite for measuring progress of multimodal language models, consisting of 269 visual understanding prompts with gold-standard responses authored by experts. The benchmark has dual objectives: vibe checking multimodal chat models for day-to-day tasks and rigorously testing frontier models, with the hard set containing >50% questions that all frontier models answer incorrectly.
Scores are shown in ratio. The current registry marks this benchmark as not independently verified with evidence level B.
- Family
- Vibe-Eval
- Modality
- multimodal
- Primary category
- multimodal
- Score direction
- higher
- LLMBoard eligible
- No
- Evaluation key
- vibe-eval|llm-stats-current
Source-native results are preserved. Eligibility for the overall LLMBoard score is a separate policy decision.
FAQ
Common questions about Vibe-Eval.
Which model scores highest on Vibe-Eval?
Gemini 2.5 Pro Preview 06-05 is currently ranked first with 67.2%.
What does Vibe-Eval measure?
VIBE-Eval is a hard evaluation suite for measuring progress of multimodal language models, consisting of 269 visual understanding prompts with gold-standard responses authored by experts. The benchmark has dual objectives: vibe checking multimodal chat models for day-to-day tasks and rigorously testing frontier models, with the hard set containing >50% questions that all frontier models answer incorrectly.
Is a higher score better?
Yes. Higher values rank better for this benchmark.
How many models are compared?
8 unique published model results are currently shown.
Does this benchmark affect the overall score?
This benchmark is preserved as source-native evidence but is not eligible for the current overall score.