Model Selection and Tradeoffs

Model Selection and Tradeoffs carries 2.7% of the CCDV-F blueprint, which works out to one or two of the 53 scored questions. It is the judgement half of its domain rather than the mechanism half: not how a model works, but which one a given workload should run on and what you surrender by choosing it.

Questions here are shaped as constrained optimisation across three axes, quality, latency and unit cost, with the quiet expectation that you notice improving one usually costs another. A scenario hands you a workload and a hard requirement (an interactive product with a response-time budget, a nightly run over a hundred thousand documents, an unattended agent working for an hour) and asks what you would point at it. A defensible answer identifies which requirement actually binds, then takes the cheapest option that clears it.

Two variables get less attention from candidates than they deserve. Effort level belongs in the selection decision, because a strong tier held at reduced effort and a lighter tier at full effort are different points on one curve rather than different rungs of a ladder. And the context window is an eligibility gate, not a preference: a workload whose input will not fit is simply not a candidate for that model, however well the tier otherwise suits the task.

Study this as a decision procedure you can defend out loud in a sentence per workload.

What the exam tests here

• •Identifying which of latency, unit cost, quality floor or context size actually binds a described workload
• •Reading a context-window figure as eligibility rather than as a bonus feature
• •Choosing tier and effort level together, since the two trade against each other
• •Escalation routing, where a cheap first pass handles the common case and hands off the rest
• •Pinned model identifiers versus family aliases, and which one a reproducibility requirement demands
• •Re-running an evaluation set when the model behind an existing prompt changes generation

Find the binding constraint first

Almost every selection scenario states several requirements and only one of them binds. Sort them into four buckets: a quality floor (a clause that must not be missed), a latency ceiling (a suggestion that has to land while the user is still typing), a unit-cost target (millions of short classifications), and a context requirement (a change spanning a whole repository).

Decide which one fails first if you choose wrong, and the rest follows: take the least expensive option that clears the binding requirement, not the strongest option available. Distractors invert this. They reach for maximum capability on a workload whose stated problem is spend, or for minimum price on one whose stated problem is a missed clause. A requirement being mentioned does not make it the constraint.

Tier and effort move on the same curve

Because reasoning depth is steered by an effort setting rather than a fixed allowance, the choice is a grid and not a ladder. A capable tier held at low effort and a lighter tier running at full depth overlap substantially in both price and quality, so when a workload is nearly fast enough or nearly cheap enough you have two distinct moves available.

They are not equivalent. Reducing effort changes how long the model deliberates while leaving its underlying capability intact, so mechanical work (extraction, reformatting, routing) barely notices while multi-step reasoning degrades quickly. Dropping a tier changes capability everywhere at once. Try both against the same evaluation set before committing, because which one hurts less is a property of your task rather than a general rule.

Route when the workload is bimodal

Production traffic is often two workloads wearing one name. Support triage is mostly routine with a hard tail; document processing is mostly clean pages with a handful of scans. A single model choice either overpays across the easy majority or underserves the difficult minority.

The escalation pattern fixes that: a small model attempts the request, and anything it flags as low confidence or explicitly declines to handle is retried on a stronger one. What you pay is two calls on the hard path, a confidence signal you must genuinely be able to compute, and a routing layer that becomes another thing to test. That trade is worth making at volume and is overengineering below it, so look for a stated request rate before selecting this option.

Pinned identifiers, aliases and migration

Model identifiers are exact strings, and an alias tracking the newest member of a family behaves differently from a pinned snapshot that never moves. Pinning buys reproducibility: an evaluation result stays true, a cost profile stays predictable, and an auditor asking why a decision came out that way nine months ago gets a real answer. It costs you manual upgrades and a retirement date you have to watch.

An alias keeps you current with no release, which suits an internal tool and badly suits a regulated workflow. Whichever you use, treat a generation change as a change: prompts tuned against an older model often carry emphatic phrasing or step-by-step scaffolding that a more literal successor now overapplies, so re-run the evaluation set instead of assuming a silent improvement.

Where candidates go wrong

Common questions