Prompt
How many fruits of each type are in the image?
- Max tokens
- 100K
- temperature
- 0
- top_p
- 1
- seed
- 42
- presence_penalty
- 0
- frequency_penalty
- 0
- Reasoning effort
- High
- Execution
- Single-shot via API
Vision / Visual Counting
Fruit Counting From Image
In the image task, the model counts bananas, apples, and oranges, even when some fruit overlap. It has to identify each fruit type and count carefully instead of giving a rough description.
Fortytwo Prime
Fortytwo
PASS4 / 4
Based on the image, here is the count of each fruit type:
• Apples: 8
• Bananas: 7 (3 in the bunch on the left, 4 in the bunch on the right)
• Oranges: 5
Total: 20 fruits
Fortytwo identifies apples, bananas, and oranges, including the two banana bunches, and returns the exact answer-key counts.
Claude Opus 4.8Anthropic
MIXED2 / 4
Here's the count of each fruit type in the image:
- Apples: 8 (red apples in the center)
- Oranges: 6 (scattered around the edges)
- Bananas: 5 (2 on the left, 3 on the right)
Opus 4.8 identifies the expected fruit types and reports 8 apples, but gives 6 oranges instead of 5 and 5 bananas instead of 7.
Model verdicts
ModelVerdictIdentified Fruit TypesCounted ApplesCounted OrangesCounted Bananas
1Fortytwo PrimePASSpasspasspasspass2Kimi K2.7 CodePASSpasspasspasspass3Gemini 3.5 FlashPASSpasspasspasspass4ChatGPT 5.5PASSpasspasspasspass5Gemini 3.1 ProPASSpasspasspasspass6Kimi K2.5PASSpasspasspasspass7Claude Fable 5MIXEDpasspasspassfail8Qwen 3.7 PlusMIXEDpasspasspassfail9MiniMax M3MIXEDpasspasspassfail10Grok 4.3MIXEDpasspassfailpass11Claude Opus 4.7MIXEDpasspasspassfail12Gemma 4MIXEDpasspasspassfail13Qwen 3.6 PlusMIXEDpasspasspassfail14ChatGPT 5.4MIXEDpasspasspassfail15Claude Opus 4.8MIXEDpasspassfailfail16Grok 4.20FAILpassfailfailfail17Claude Opus 4.6FAILpassfailfailfail18OpenRouter Fusion (Quality)UNSUPPORTEDunsupportedunsupportedunsupportedunsupported19OpenRouter Fusion (Budget)UNSUPPORTEDunsupportedunsupportedunsupportedunsupported20Nemotron 3 UltraUNSUPPORTEDunsupportedunsupportedunsupportedunsupported21DeepSeek V4 ProUNSUPPORTEDunsupportedunsupportedunsupportedunsupported22DeepSeek V4 FlashUNSUPPORTEDunsupportedunsupportedunsupportedunsupported23GLM 5.1UNSUPPORTEDunsupportedunsupportedunsupportedunsupported24MiniMax M2.7UNSUPPORTEDunsupportedunsupportedunsupportedunsupported25DeepSeek V3.2UNSUPPORTEDunsupportedunsupportedunsupportedunsupportedHave a complex task to evaluate?
Request a custom evaluation for your use case.