Debugging and Error Handling
Debugging and Error Handling is 2.6% of the CCDV-F blueprint, one or two of the 53 scored questions, and the sole sub-skill of its domain. It covers diagnosis where the failure never raises: errors delivered inside a successful response, and exception handling that keeps the retryable and the terminal apart.
Written by Kiran Manne
2.6% of the CCDV-F blueprint, which is roughly 1 question of 53. It sits inside Eval, Testing, and Debugging, worth 2.6% in total.
Debugging and Error Handling holds all 2.6 percent of Eval, Testing, and Debugging, which is one or two questions on a 53-item paper. It is the last thing worth studying and it is genuinely short, so aim for a small set of reflexes rather than coverage.
The reflex that matters most is this: in a Claude-backed system, the majority of interesting failures do not throw. A request can succeed at the transport layer and still contain a decline, a truncation, an error reported inside a tool result block, or an instruction the model wrote out as prose instead of executing. Code holding only an exception handler is watching the wrong channel, and a team alerting only on error rates will see none of it.
The second reflex concerns exception handling itself. Catching one broad class of API failure is worse than catching none, because it flattens the distinction the recovery logic depends on: whether this failure will resolve on its own, or whether it will fail identically forever. The remedy is a chain running from specific to general, so a rate limit, a missing resource and a dropped connection each reach the code written for them.
What the exam tests here
- Recognising failure that arrives inside a successful response rather than as an exception
- Ordering exception handling from the most specific class to the most general
- Reading a server-side tool result whose content is an object on failure and a list on success
- Separating a harness defect from a change in model behaviour before touching the prompt
- Capturing enough of a request to reproduce a failure that will not recur on demand
Handle exceptions as a chain, not a class
The libraries expose one exception type per failure class, all descending from a common base, and the tempting handler catches the base. Doing so loses the only thing the handler needed to know.
A rate limit wants a wait and a retry, ideally for the interval the response suggests. A missing resource means the identifier is wrong and no amount of waiting fixes it. An authentication failure needs a person. A dropped connection is worth retrying straight away.
Write the branches from most specific to least, ending with a general case that logs and surfaces rather than swallows. In languages without exception subclasses the same logic runs on the status code after unwrapping the error, and the ordering matters every bit as much.
Some failures arrive with a success status
Two categories deserve explicit checks, because nothing else will surface them.
Server-side tools, the ones running on the provider's infrastructure, report their own failures inside the result block returned with the response. A search that could not run appears as a result whose content is an error object rather than a list of findings, so code assuming a list and indexing it throws somewhere unhelpful, several frames from the real problem.
The second category is the response that completed with nothing usable in it, where the reason is recorded on the response instead of being raised. Both demand the same discipline: inspect the response structure before consuming it, and branch on what you find.
Separate a harness defect from a model behaviour change
When output degrades, the instinct is to edit the prompt, and roughly half the time the prompt is innocent. Establish the layer first.
Replay a captured request exactly as it was sent, against the same model identifier, and see whether the behaviour reproduces. If it does, the model and the prompt are in scope. If it does not, something in the assembly changed: a truncated document, a tool description edited by somebody else, a retrieval step returning different passages, a configuration value differing between environments.
Log the assembled request rather than the template, because the template is not what was sent. Diffing two assembled requests, one from a good run and one from a bad, resolves a surprising share of these in minutes.
Capture enough to reproduce what will not recur
Intermittent behaviour is normal here, so a report saying what happened is not enough to act on.
The record you need is the exact model identifier used, the full assembled request including every parameter, the identifier the provider returned for that call, how the turn ended, and the usage figures. With those, a failure appearing once in fifty runs can be replayed deliberately and investigated at leisure. Without them the investigation begins by trying to make it happen again, which is the slowest possible start.
Sample the capture if volume forbids keeping all of it, and always capture in full on the paths where a failure is expensive.
Where candidates go wrong
Trap: a single catch block around the API call
It looks tidy and it makes every failure equivalent. A malformed request, an expired key, a rate limit and a dropped connection all arrive at one handler, which then does one thing, usually a retry, helping in one of those four cases and actively harming in two.
Worse, the message logged is generic enough that the on-call engineer learns nothing from it. Write the specific handlers first, keep the general one for genuinely unexpected cases, and never match on message text to tell classes apart, because wording is not a stable interface.
Trap: blaming the model when the harness is at fault
A degradation looks like a model problem, so the first change is usually a prompt edit, and that edit sometimes appears to help, which cements a wrong diagnosis.
Common harness causes include a document silently truncated before it was sent, a retrieval step whose results changed, a tool description edited without review, a configuration value differing by environment, and a client that stopped echoing part of the assistant turn back. Each is invisible in a prompt template and obvious in the assembled request. Look at what was actually sent before changing what you meant to send.
Trap: alerting on error rate alone
A dashboard built on exception counts and non-2xx responses stays green through most of the failure modes that matter, because declines, truncations, failed server tools, unusable answers and runaway loops all occur inside successful responses.
Instrumentation that works counts outcomes rather than errors: how turns ended, how often output hit the cap, how often validation of the result failed, how many iterations a run took, and what it cost. A scenario where everything is green and users are complaining is pointing at exactly this gap.
Common questions
How many CCDV-F questions are on Debugging and Error Handling?
2.6 percent of the blueprint, one or two of the 53 scored questions. It is the only sub-skill in Eval, Testing, and Debugging, so preparing the sub-skill and preparing the domain are the same exercise.
Why is a single catch around the API call a problem?
Because recovery depends on a distinction the single catch destroys: whether the failure is transient or permanent. A rate limit wants a wait, a missing resource wants a corrected identifier, an authentication failure wants a person, and a dropped connection wants an immediate retry. One handler cannot do all four, so it does the wrong thing for most of them.
Which kinds of failure do not raise an exception?
A safety decline, an answer truncated by the output cap, a provider-side tool that failed and said so inside its result block, a tool loop left unfinished, and an answer that is fluent and wrong. Every one of those is delivered on a successful response, so the only way to notice is to inspect the response rather than wait for something to be thrown.
How do I debug something that only happens occasionally?
Capture enough to replay it: the model identifier, the fully assembled request, the provider's request identifier, how the turn ended, and the token usage. Then reproduce deliberately instead of waiting. If the replay does not reproduce it, the fault lies in how the request was assembled rather than in the model, and diffing a good request against a bad one usually finds it.
Practise this
The CCDV-F bank covers every sub-skill in the blueprint, weighted the way the real paper is. Free sample questions need no account.
Free CCDV-F questions