Weather Report

Weather Report

What models we're running today, how they're configured, and what role each one plays in the factory.

Subscribe via RSS

Why do we publish The Weather Report? The Weather Report started out as a casual internal summary of how each provider and model was performing on our most important use cases. We update it frequently and have found it essential to our process.

As of September 28th, 2026

Big refresh across the board. CS/Math Hard Problems moves to gpt-6-astra. Opus 5.5 now covers Frontend Aesthetics, Frontend Architecture, Architectural Critique, Security review, and Copy / Writing, and is paired with gpt-5.6-sol for Sprint Planning consensus. UX Ideation is opus-5.5 with gpt-image-2.5-sunburst. QA Orchestration moves to gpt-5.6-sol. DevOps Tasks moves to luna on max; it is good enough, and switching models gives only marginal improvements. Bulk MapReduce moves to luna because it is so cheap, and Bulk classification uses system-one models like jev. Image comprehension and Agentic dialogues move to gemini-3.8-flash. Voice moves to gpt-live-1. Modified: CS/Math Hard Problems, Image comprehension, Frontend Aesthetics, Frontend Architecture, Architectural Critique, Sprint Planning, Devops Tasks, QA Orchestration, Security review, Bulk classification, Bulk MapReduce, UX Ideation, Copy / Writing, Agentic dialogues, Voice (interactive).

UseModels (by preference)ParametersNotes
CS/Math Hard Problems
Sep 28
gpt-6-astradefault
Image comprehension
Sep 28
gemini-3.8-flashdefault
Frontend Aesthetics
Sep 28
opus-5.5max
Frontend Architecture
Sep 28
opus-5.5default
Architectural Critique
Sep 28
opus-5.5max
Sprint Planning
Sep 28
consensus(opus-5.5, gpt-5.6-sol)max / default
Devops Tasks
Sep 28
lunamaxGood enough; only marginal improvements from switching models
QA Orchestration
Sep 28
gpt-5.6-soldefault
Security review
Sep 28
opus-5.5high
Bulk classification
Sep 28
System-one models (e.g. jev)default
Bulk MapReduce
Sep 28
lunadefaultSo cheap
UX Ideation
Sep 28
opus-5.5 / gpt-image-2.5-sunburstdefault
Copy / Writing
Sep 28
opus-5.5default
Agentic dialogues
Sep 28
gemini-3.8-flashdefaultGeneral message handling loops with user interaction and limited tool calling.
Voice (interactive)
Sep 28
gpt-live-1default
Consensus operator refers to an LLM merge of the points from independent plans.

Log

September 28th, 2026

Big refresh across the board. CS/Math Hard Problems moves to gpt-6-astra. Opus 5.5 now covers Frontend Aesthetics, Frontend Architecture, Architectural Critique, Security review, and Copy / Writing, and is paired with gpt-5.6-sol for Sprint Planning consensus. UX Ideation is opus-5.5 with gpt-image-2.5-sunburst. QA Orchestration moves to gpt-5.6-sol. DevOps Tasks moves to luna on max; it is good enough, and switching models gives only marginal improvements. Bulk MapReduce moves to luna because it is so cheap, and Bulk classification uses system-one models like jev. Image comprehension and Agentic dialogues move to gemini-3.8-flash. Voice moves to gpt-live-1. Modified: CS/Math Hard Problems, Image comprehension, Frontend Aesthetics, Frontend Architecture, Architectural Critique, Sprint Planning, Devops Tasks, QA Orchestration, Security review, Bulk classification, Bulk MapReduce, UX Ideation, Copy / Writing, Agentic dialogues, Voice (interactive).

June 4th, 2026

Moved to gpt-5.5 across our OpenAI text/code defaults: CS/Math, Frontend Architecture, Architectural Critique, Security review, and the Sprint Planning consensus pair. Bumped the consensus Opus to opus-4.8. DevOps Tasks and QA Orchestration came off opus-4.7/max onto gpt-5.5 at default params, which is good enough there. Voice bumped gpt-realtime-1.5 to gpt-realtime-2. Heads up: Google updated gemini-3-flash-preview to be Gemini 3.5 Flash under the same model ID, so Image comprehension and Agentic dialogues get the upgrade for free. If you haven't already switched to using codex/gpt-5.5 as your defaults everywhere, this is a good time to do that. Gemini 3.5 Flash, while a solid agentic model and good for small tasks, does still sometimes end up in a tool call loop. Opus 4.8 is good when you need input from another model, but I would not daily drive anything with it right now; it burns too many tokens. The default for everyday tasks should be gpt-5.5 on low/medium before you bump up the reasoning. Modified: CS/Math Hard Problems, Frontend Aesthetics, Frontend Architecture, Architectural Critique, Sprint Planning, Devops Tasks, QA Orchestration, Security review, Image comprehension, Agentic dialogues, Voice (interactive). Added: Copy / Writing.

April 21st, 2026

Switched UX Ideation from gemini-3-pro-image-preview (Nano Banana Pro) to gpt-image-2 (ChatGPT Images 2.0). Modified: UX Ideation.

April 20th, 2026

Bumped opus-4.6 → opus-4.7 across our Anthropic defaults. We're seeing discrepancies with opus-4.7's adaptive thinking, so we're sticking with max thinking for now. gpt-5.3-codex remains our default implementation model — we've evaluated alternatives but haven't found a reason to switch.

March 12th, 2026

gpt-5.4 has been formally adopted for planning and architectural critique. It replaces gpt-5.2 in our Sprint Planning consensus pair and Architectural Critique. Gemini can improve consensus in some cases for sprint planning. We're continuing to evaluate gpt-5.4 for implementation tasks but keeping gpt-5.3-codex as our default implementation model for now.

February 23rd, 2026

No specific changes in defaults, but please note for anyone evaluating Gemini 3.1, the gemini-3.1-pro-preview-customtools may significantly outperform gemini-3.1-pro-preview depending on your harness. We've switched to gpt-realtime-1.5 for our internal use cases but aren't officially defaulting to it yet. Very happy with Sonnet 4.6, it may overtake Opus for some of our everyday use cases.

February 13th, 2026

Happy with gpt-5.3-codex-spark. gpt-5.3-codex continues to be our preferred default implementation model with critiques and suggestions from Opus. Modified: Sprint Planning. Added: UX Ideation, Agentic dialogues, Voice (interactive).

February 6th, 2026

New models this week. We're very happy with gpt-5.3-codex. No problems with Opus 4.6 so far.