Compare
Gemini 2.5 Ultra vs Llama 4 Behemoth
Google DeepMind vs Meta. Specs, benchmarks, and real per-task cost — all in one page.
Verdict
Gemini 2.5 Ultra leads on LMSYS ELO (1385 vs 1342). Llama 4 Behemoth is ~2.7× cheaper on a 3:1 input:output blend. Llama 4 Behemoth is open-weights, the other is proprietary.
Google DeepMind
Gemini 2.5 Ultra2M-token context, native video understanding, and Google’s deepest multimodal stack. The long-context king.
2M context windowNative videoBest multimodal reasoning
Meta
Llama 4 BehemothMeta’s open-weights flagship. 405B params, fully open license, runs on every major inference provider.
Best open-weights modelRuns anywhereTransparent training
Pricing
Input / 1M$7.00$2.50
Output / 1M$21.00$8.00
Context2.0M256K
Max output64K16K
LicenseProprietaryOpen weights
Released2026-02-052025-12-10
Benchmarks
LMSYS ELO1385.01342.0
MMLU Pro90.487.1
HumanEval90.289.3
SWE-Bench58.352.4
MATH92.081.5
GPQA—56.2
MMMU82.1—
IFEval89.788.0
Per-task cost
Summarize a 1-hour meeting transcript$0.115$0.041
Review a 500-line pull request$0.098$0.036
Answer a customer support ticket$0.038$0.014
Extract structured data from a resume$0.056$0.021
Debug a stack trace with context$0.084$0.031
Per-call cost using published token counts for each task. Real-world prompts vary.