Inference Index
Compare

Claude Haiku 4.5 vs Gemini 2.5 Flash

Anthropic vs Google DeepMind. Specs, benchmarks, and real per-task cost — all in one page.

Verdict

Gemini 2.5 Flash leads on LMSYS ELO (1312 vs 1305). Gemini 2.5 Flash is ~3.0× cheaper on a 3:1 input:output blend.

Anthropic
Claude Haiku 4.5

Fast, cheap, and smart enough for most routing and extraction tasks. Great base model for subagents.

Sub-second latency10x cheaper than OpusStrong structured output
Google DeepMind
Gemini 2.5 Flash

Google’s price/performance darling. 1M context at $0.30/M input — nobody comes close on throughput-per-dollar.

Absurd cost-to-context ratioFast multimodalFree tier via AI Studio
Pricing
Input / 1M$0.80$0.30
Output / 1M$4.00$1.20
Context200K1.0M
Max output16K32K
LicenseProprietaryProprietary
Released2025-10-012026-02-05
Benchmarks
LMSYS ELO1305.01312.0
MMLU Pro81.482.1
HumanEval87.085.8
SWE-Bench49.2
MATH76.884.2
MMMU74.6
IFEval87.386.1
Per-task cost
Summarize a 1-hour meeting transcript$0.014$0.0051
Review a 500-line pull request$0.014$0.0048
Answer a customer support ticket$0.0056$0.0019
Extract structured data from a resume$0.0080$0.0027
Debug a stack trace with context$0.013$0.0042

Per-call cost using published token counts for each task. Real-world prompts vary.