Compare

Llama 4 Behemoth vs Gemini 2.5 Flash

Meta vs Google DeepMind. Specs, benchmarks, and real per-task cost — all in one page.

Verdict

Llama 4 Behemoth leads on LMSYS ELO (1342 vs 1312). Gemini 2.5 Flash is ~7.4× cheaper on a 3:1 input:output blend. Llama 4 Behemoth is open-weights, the other is proprietary.

Meta

Llama 4 Behemoth

Meta’s open-weights flagship. 405B params, fully open license, runs on every major inference provider.

Best open-weights modelRuns anywhereTransparent training

Google DeepMind

Gemini 2.5 Flash

Google’s price/performance darling. 1M context at $0.30/M input — nobody comes close on throughput-per-dollar.

Absurd cost-to-context ratioFast multimodalFree tier via AI Studio

Pricing

Input / 1M$2.50$0.30

Output / 1M$8.00$1.20

Context256K1.0M

Max output16K32K

LicenseOpen weightsProprietary

Released2025-12-102026-02-05

Benchmarks

LMSYS ELO1342.01312.0

MMLU Pro87.182.1

HumanEval89.385.8

SWE-Bench52.4—

MATH81.584.2

GPQA56.2—

MMMU—74.6

IFEval88.086.1

Per-task cost

Summarize a 1-hour meeting transcript$0.041$0.0051

Review a 500-line pull request$0.036$0.0048

Answer a customer support ticket$0.014$0.0019

Extract structured data from a resume$0.021$0.0027

Debug a stack trace with context$0.031$0.0042

Per-call cost using published token counts for each task. Real-world prompts vary.

Tool

Interactive compare →

Add up to four models, tweak tasks live.