Prompt
How many fruits of each type are in the image?
- Max tokens
- 100K
- temperature
- 0
- top_p
- 1
- seed
- 42
- presence_penalty
- 0
- frequency_penalty
- 0
- Reasoning effort
- High
- Execution
- Single-shot via API
Vision / Visual Counting
Fruit Counting From Image
In the image task, the model counts bananas, apples, and oranges, even when some fruit overlap. It has to identify each fruit type and count carefully instead of giving a rough description.
Fortytwo Prime
Fortytwo
PASS4 / 4
Based on the image, here is the count of each fruit type:
• Apples: 8
• Bananas: 7 (3 in the bunch on the left, 4 in the bunch on the right)
• Oranges: 5
Total: 20 fruits
Fortytwo identifies apples, bananas, and oranges, including the two banana bunches, and returns the exact answer-key counts.
Claude Fable 5Anthropic
MIXED3 / 4
Here's the fruit count from the image:
- Apples: 8 — red apples arranged in the center of the image, several with stems and leaves still attached
- Oranges: 5 — one in the top right, one near the top center, one in the middle, one in the bottom left (with green leaves), and one in the bottom right
- Bananas: 5 — a bunch of 2 on the left side and a bunch of 3 on the right side
Total: 18 pieces of fruit
Claude Fable 5 identifies all three fruit types and counts 8 apples and 5 oranges, but reports only 5 bananas instead of the required 7.
Model verdicts
ModelVerdictIdentified Fruit TypesCounted ApplesCounted OrangesCounted Bananas
1Fortytwo PrimePASSpasspasspasspass2Kimi K2.7 CodePASSpasspasspasspass3Gemini 3.5 FlashPASSpasspasspasspass4ChatGPT 5.5PASSpasspasspasspass5Gemini 3.1 ProPASSpasspasspasspass6Kimi K2.5PASSpasspasspasspass7Claude Fable 5MIXEDpasspasspassfail8Qwen 3.7 PlusMIXEDpasspasspassfail9MiniMax M3MIXEDpasspasspassfail10Grok 4.3MIXEDpasspassfailpass11Claude Opus 4.7MIXEDpasspasspassfail12Gemma 4MIXEDpasspasspassfail13Qwen 3.6 PlusMIXEDpasspasspassfail14ChatGPT 5.4MIXEDpasspasspassfail15Claude Opus 4.8MIXEDpasspassfailfail16Grok 4.20FAILpassfailfailfail17Claude Opus 4.6FAILpassfailfailfail18OpenRouter Fusion (Quality)UNSUPPORTEDunsupportedunsupportedunsupportedunsupported19OpenRouter Fusion (Budget)UNSUPPORTEDunsupportedunsupportedunsupportedunsupported20Nemotron 3 UltraUNSUPPORTEDunsupportedunsupportedunsupportedunsupported21DeepSeek V4 ProUNSUPPORTEDunsupportedunsupportedunsupportedunsupported22DeepSeek V4 FlashUNSUPPORTEDunsupportedunsupportedunsupportedunsupported23GLM 5.1UNSUPPORTEDunsupportedunsupportedunsupportedunsupported24MiniMax M2.7UNSUPPORTEDunsupportedunsupportedunsupportedunsupported25DeepSeek V3.2UNSUPPORTEDunsupportedunsupportedunsupportedunsupportedHave a complex task to evaluate?
Request a custom evaluation for your use case.