multimodal benchmark
Vibe-Eval
VIBE-Eval is a hard evaluation suite for measuring progress of multimodal language models, consisting of 269 visual understanding prompts with gold-standard responses authored by experts. The benchmark has dual objectives: vibe checking multimodal chat models for day-to-day tasks and rigorously testing frontier models, with the hard set containing >50% questions that all frontier models answer incorrectly.
Updated Aug 11, 2026
Models8
Model coverage8
MetricScore
EvidenceB
Vibe-Eval Ranking
Higher score ranks better on this benchmark.
| 1 | GO | 67.2% | 100.0% | 8 | C | |
| 2 | GO | 65.6% | 85.7% | 8 | C | |
| 3 | GO | 65.4% | 71.4% | 8 | C | |
| 4 | GO | 56.3% | 57.1% | 8 | C | |
| 5 | GO | 53.9% | 42.9% | 8 | C | |
| 6 | GO | 51.3% | 28.6% | 8 | C | |
| 7 | GO | 48.9% | 14.3% | 8 | C | |
| 8 | GO | 40.9% | 0.0% | 8 | C |
Vibe-Eval Score Distribution
A closer view of the leading scores on this benchmark.
Vibe-Eval Highlights
The leading models and scores on this benchmark.
What is Vibe-Eval?
What Vibe-Eval measures and how its scores work.
VIBE-Eval is a hard evaluation suite for measuring progress of multimodal language models, consisting of 269 visual understanding prompts with gold-standard responses authored by experts. The benchmark has dual objectives: vibe checking multimodal chat models for day-to-day tasks and rigorously testing frontier models, with the hard set containing >50% questions that all frontier models answer incorrectly.
Scores are shown in ratio. This benchmark is not independently verified and has an evidence level of B.
- Family
- Vibe-Eval
- Modality
- multimodal
- Primary category
- multimodal
- Score direction
- higher
- LLMBoard eligible
- Yes
- Evaluation key
- vibe-eval|llm-stats-current
Benchmark scores retain their original unit. Overall score eligibility is shown separately.
FAQ
Common questions about Vibe-Eval.
Which model scores highest on Vibe-Eval?
Gemini 2.5 Pro Preview 06-05 is currently ranked first with 67.2%.
What does Vibe-Eval measure?
VIBE-Eval is a hard evaluation suite for measuring progress of multimodal language models, consisting of 269 visual understanding prompts with gold-standard responses authored by experts. The benchmark has dual objectives: vibe checking multimodal chat models for day-to-day tasks and rigorously testing frontier models, with the hard set containing >50% questions that all frontier models answer incorrectly.
Is a higher score better?
Yes. Higher values rank better for this benchmark.
How many models are compared?
8 model results are currently shown.
Does this benchmark affect the overall score?
Yes. This benchmark can contribute to the current LLMBoard capability score.