Prompt caching vs batch processing
Two cost levers that are routinely confused because both are described as cheaper. They reduce different costs, have different prerequisites, and compose rather than compete.
Short answer
Prompt caching reduces what you pay to re-send the same prefix across requests, and reduces latency as a side effect. Batch processing reduces the price of work nobody is waiting for, and increases latency as the price of that. If your prompt has a large stable prefix and repeat traffic, cache it. If results can be collected later, batch it. If both are true, do both, because they operate on different parts of the bill.
The confusion is caused by the way both are introduced. Each is presented as a way to spend less, so they land in the same mental slot, and questions exploit that by describing a cost problem and offering both as options. Only one usually fits, and which one depends on facts in the scenario that are easy to skim past.
The distinction is cleaner than it looks. Caching is about repetition within a prompt across requests: it only pays when many requests share a long identical prefix, and it pays back in both money and time to first token. Batching is about patience: it applies to any request at all, needs nothing from the prompt's structure, and pays back only in money, in exchange for results arriving later.
So the diagnostic questions are different. For caching, ask what is identical across requests and whether it is long enough to qualify. For batching, ask whether anybody is waiting. A scenario that says a user is on the page has ruled out batching regardless of how much it would save, and a scenario where every request has a unique prompt has ruled out caching regardless of how much traffic there is.
Prompt caching vs Batch processing, side by side
| Prompt caching | Batch processing | |
|---|---|---|
| What it reduces | The input cost of the repeated part of a prompt, and time to first token | The price of the whole request, in exchange for latency |
| What it needs from the prompt | A stable prefix long enough to meet the model's minimum cacheable length | Nothing. Any request can be submitted as batch work. |
| What it needs from the product | Repeat traffic over the same prefix inside the cache lifetime | A workflow where results are collected later rather than awaited |
| Effect on latency | Lower. A cache read is faster than sending the prefix again. | Much higher. Results arrive when the job finishes, not when you ask. |
| Typical fit | A chat over a fixed document set, a long system prompt, a large tool catalogue | Overnight evaluation runs, bulk classification, backfills, dataset labelling |
| How it fails | The prefix changes on every request, so nothing is ever read from cache | Someone is waiting for a result that will not arrive for hours |
Caching pays for a prefix you are going to send again
You mark a breakpoint in the prompt with cache_control, and everything before it becomes the cached prefix. Later requests that begin with byte-identical content read from the cache instead of being processed again.
Three properties decide whether it is worth it. Writing to the cache costs more than sending the same tokens normally, so a prefix used once is more expensive cached than not. Reading from it costs a fraction of normal input, which is where the saving comes from. And the cached entry has a limited lifetime that is refreshed by use, so traffic has to be frequent enough to keep it warm. The exact multipliers and durations are published per model and do change, so price your own workload rather than trusting a remembered figure.
Order the prompt so the stable part comes first
The cache matches on an exact prefix, which makes prompt layout load-bearing rather than cosmetic. Put the things that never change at the top: tool definitions, the system prompt, the fixed document set. Put the things that change on every request at the bottom: the user's question, the retrieved passages, the timestamp.
One changed character before the breakpoint invalidates everything after it, so a prompt that injects the current date or the user's name near the top will miss on every single request while looking correctly configured. This is the most common reason a team reports that caching did nothing for them, and it is also a convenient alignment: the same layout is what the long-context guidance recommends anyway, with documents above the query.
Batching pays for patience
Batch processing submits many requests as a job and returns the results when it completes, within a published maximum window. In exchange you pay a lower price per request than the same work would cost synchronously.
It asks nothing of the prompt. Every request in the job can be entirely different, which is precisely the case where caching has nothing to work with. What it asks is that no user is blocked on the answer. That makes it the right tool for evaluation runs, bulk classification of a backlog, generating embeddings or labels over a dataset, and any nightly job whose output is read the next morning. It makes it the wrong tool for anything in a request path, no matter how attractive the saving.
They compose, and the composition is the interesting case
Consider an evaluation run: five hundred cases, each with the same long system prompt and rubric, each with a different input. That prompt has both properties at once. The rubric is a long stable prefix, so it is cacheable, and nobody is waiting for the results, so it is batchable.
Using both is not double counting, because they discount different things. Batching lowers the price of the request; caching lowers the volume of input being charged for within it. A question that offers caching and batching as mutually exclusive options is testing whether you know they are orthogonal. The real constraint is whether the batch runs closely enough together for cached entries to stay warm.
Neither is the first thing to try
Both are optimisations on a prompt you have already decided to send, and the cheapest token remains the one you do not send at all. Before reaching for either, two questions usually return more.
First, is everything in this prompt earning its place? Tool definitions for capabilities this workflow never uses, accumulated instructions nobody remembers adding, and documents included in case they help are all charged for on every request. Second, is this the right model tier? Extraction and classification work often runs correctly on a lighter, faster tier, and moving it there beats optimising the delivery of an oversized prompt to an oversized model.
How to choose
Reach for prompt caching when
- Many requests share a long identical prefix, such as a system prompt or a fixed document set
- That prefix clears the model's minimum cacheable length
- Traffic is frequent enough to keep the cached entry warm
- Latency matters, because a cache read also cuts time to first token
Reach for batch processing when
- Nobody is waiting for the result and it can be collected later
- The work is high volume, such as evaluation runs, backfills or bulk classification
- Requests differ from each other, so there is no shared prefix to cache
- The saving matters more than when the answer arrives
Where candidates go wrong
Trap 1: caching makes the repeated part free
It makes it cheaper on reads and more expensive on writes. A prefix that is sent once and never again costs more with caching enabled than without, because you paid the write premium and never collected a read. The practical consequence is that caching needs a traffic argument, not just a prompt-shape argument. A long system prompt used by a handful of requests a day may never keep an entry warm long enough to pay back. Options that recommend caching purely because a prompt is long, with nothing in the scenario about repeat volume, are usually the distractor.
Trap 2: put the user's question first so the model sees it
This defeats caching completely, because the cache matches on an exact prefix and the question changes every time. It also runs against the long-context layout guidance, which puts long documents above the query. The same trap appears in subtler forms: a session identifier, the current date, or a personalised greeting placed near the top of the system prompt. Each looks harmless and each guarantees a miss on every request. When a scenario says caching was enabled and had no effect, look for something varying above the breakpoint before looking at the configuration.
Trap 3: batching is just a queue we could build ourselves
You can certainly queue your own work, and that gets you scheduling and back-pressure. What it does not get you is the price, which is the entire reason batch processing exists as a distinct product surface. The inverse error also appears: treating batching as a way to raise throughput within a request path. It does not make anything faster. It makes work cheaper by removing the promise of a prompt answer, and a scenario describing an interactive feature has already excluded it however large the saving looks.
Trap 4: cache the whole prompt
The breakpoint belongs at the end of the stable region, not at the end of the prompt. Placing it after the user's turn produces a cache entry that is unique to that request and will never be read again, which is the write premium with none of the benefit. A related mistake is scattering breakpoints hopefully through a prompt. Each one has a cost and each defines a boundary that must match exactly, so the useful pattern is deliberate: identify the longest genuinely stable prefix, mark it once, and keep everything that varies below it.
How the exam tests it
Model Selection and Optimization is 16.8% of the CCDV-F blueprint, roughly 9 of the 53 scored questions, and Cost and Token Management is 2.8% of that. Questions in this area are typically posed as a scenario with a cost or latency problem and several plausible remedies, so recognising which lever the scenario's own constraints permit matters more than knowing any individual rate.
Common questions
What is the difference between prompt caching and batch processing?
Prompt caching reduces the input cost of a prefix you send repeatedly, and lowers latency. Batch processing reduces the price of a request in exchange for the result arriving later. Caching needs a stable repeated prefix; batching needs nobody to be waiting.
Can I use both at once?
Yes, and an evaluation run is the classic case: a long shared rubric that caches well, over hundreds of different inputs that nobody is waiting for. They discount different things, so the savings are independent. The practical requirement is that the batch runs closely enough together to keep cached entries warm.
Why is my prompt cache never hitting?
Almost always because something varies before the breakpoint. A timestamp, a session identifier, a user name or freshly retrieved passages near the top of the prompt will invalidate the prefix on every request. Move everything variable below the breakpoint, and check the stable part clears the model's minimum cacheable length.
Does caching ever cost more than not caching?
Yes. Writing to the cache costs more than sending the same tokens normally, so a prefix that is written once and never read back is a net loss. Caching pays only when the same prefix is reused enough times, and often enough, to earn back the write premium.
Can I batch requests for an interactive feature?
No. Batch results arrive when the job completes, which can be hours. If a user is waiting for the answer, batching is excluded no matter how large the saving. Reach for caching, a lighter model tier, or a smaller prompt instead.
What should I try before either of these?
Remove what the prompt does not need, and check the model tier. Unused tool definitions, accumulated instructions and just-in-case documents are charged on every request, and a lot of extraction and classification work runs correctly on a lighter, faster tier. Both usually return more than optimising how an oversized prompt is delivered.
Practise this
Model Selection and Optimization is 16.8% of the CCDV-F blueprint, and the bank is weighted to match. Every option carries a written explanation, not just the correct one, so a distinction like this one is explained where you get it wrong.
See CCDV-FOther CCDV-F distinctions
Not affiliated with or endorsed by Anthropic. Domain names and weightings are taken from the published exam guide; always check the official guide before booking.