Cost and Token Management

At 2.8% of a 53-question paper, Cost and Token Management is worth about one and a half marks, and it is among the most answerable sub-skills on the blueprint because nearly everything in it is arithmetic you can check.

The subject is not how to make an application cheaper in the abstract. It is the accounting: which counters an API response reports, what each of them bills at, and how the shape of an application turns those counters into a monthly figure. Two structural facts do most of the work. Output tokens are several times the price of input tokens, so anything lengthening responses costs more than the same volume of prompt. And a stateless interface means every turn resends the entire history, so a chat session's input cost grows with the square of its length rather than in a straight line.

Once those are internalised the questions become tractable. Whether a proposed change saves money, whether an optimisation is real or cosmetic, whether an invoice that grew faster than traffic points at longer sessions or fatter prompts: each follows from the counters.

Budget about an hour here. The ideas are few and precise, and being able to explain a bill is worth more on this exam than being able to recite a rate card.

What the exam tests here

• •Reading the four token counters on a response and knowing they bill at four different rates
• •Why an uncached multi-turn conversation costs more than the sum of its messages
• •That reasoning tokens are billed on the output side, so effort moves the expensive half
• •Break-even reasoning on a cache write, and how a limited entry lifetime affects it
• •Measuring a prompt before sending it rather than estimating from character count
• •Telling a token-throughput limit apart from a spending problem, and applying the right remedy

Usage is four line items, not two

A response reports consumption in separate buckets: ordinary input, output, tokens written into the cache, and tokens read back from it. They are disjoint. A prompt served largely from cache reports a small ordinary input figure alongside a large cache-read figure, which is why code that adds the two together and multiplies by the input rate produces a number bearing no relation to the invoice.

Each bucket carries its own price. Reads are a fraction of base input, writes carry a premium over it, and output sits several times above both, so costing a feature means four multiplications and a sum. Log all four per request and spend becomes attributable: you can say which endpoint, which prompt version and which customer produced a given slice of the total.

Conversation cost is quadratic, not linear

No conversation state is held for you, so turn twelve of a chat resends turns one through eleven as input. Add up a whole session and the input tokens billed scale with roughly the square of the turn count. This is the most underestimated line in a chat product's spend, and it explains the common report that cost rose faster than the user count: sessions got longer, not more numerous.

Two responses matter. Caching the stable head converts most of that growth back toward linear, since the repeated portion bills at the read rate. And summarising old turns removes them from every future request rather than only the next one. Note the tension: the front of the history is exactly the region a cache needs to stay identical.

Reasoning tokens bill as output

Deliberation a model performs before answering is generated text, and it is billed on the output side at the expensive rate. That single fact wires the effort setting directly to spend: raising effort buys thinking with the costliest tokens available, and lowering it produces an immediate saving rather than a cosmetic one.

It also reframes what a runaway bill looks like. An agent whose per-request cost climbed with no change in prompt size has probably started reasoning more, or taking more tool calls, each of which is another round trip carrying the accumulated context. When a scenario reports rising cost against stable input volume, look at the output side first: response length, reasoning depth, and how many turns the agent takes before it stops.

Measure, and do not confuse throughput with spend

Character-count heuristics are wrong in both directions depending on content, badly so for code, structured data and non-English text. The token-counting endpoint answers the question exactly, runs no inference, and must be called with the same model identifier you intend to use, since tokenizers differ across generations.

Keep that separate from rate limiting, which questions like to blur. Limits are expressed in tokens per minute, input and output counted separately, so a burst of large requests can exhaust capacity while spend sits comfortably inside budget. A 429 is a throughput signal answered with backoff, queueing or asynchronous batching. A large invoice is a spend signal answered with caching, tier choice and shorter outputs. Applying one remedy to the other problem is a recurring wrong answer.

Where candidates go wrong

Common questions