Guardrails and Safe Deployment

Guardrails and Safe Deployment is a 2.3% sub-skill: call it one question, occasionally two. It covers the machinery around a model that keeps a launch from turning into an incident, and it is the operational counterpart to the security material rather than a second helping of it.

A guardrail is a control sitting outside the model that constrains what reaches it, what it produces, or what it is permitted to do about either. That last category matters most. Screening text is cheap and imperfect; declining to execute an irreversible operation without a human is neither cheap nor imperfect, it is simply true. Questions here tend to hinge on whether you put a control somewhere it can actually bind.

Deployment is the other half, and it is ordinary release engineering applied to a component whose failures are probabilistic rather than deterministic. You cannot enumerate the inputs, so you cannot test your way to confidence. What you can do is limit exposure while evidence accumulates, watch the right signals, and keep a way to switch it off that does not require a code change.

Expect scenarios describing a team about to ship to everybody at once, and expect the right answer to involve seeing more while exposing less.

What the exam tests here

• •Placing a control on the input, the output or the action, and what each placement costs and buys
• •Choosing which operations require human approval, judged on reversibility and blast radius
• •Staged rollout with a stop mechanism that is configuration rather than a deployment
• •The signals worth watching on a model-backed feature, and what a shift in each implies
• •Grounding techniques that reduce fabrication, and the price each of them carries

Three placements, three different bargains

An input control inspects the request before the model sees it. It adds latency to every call including the overwhelming majority that were fine, and it is the only placement able to stop a prompt being processed at all. An output control inspects the response, which means either delaying the first token or letting text stream and then retracting it, both visible to the user.

An action control sits in front of the effect: the transfer, the deletion, the outbound message. It costs nothing on the ordinary path because it fires only when something consequential is about to happen, and it has real teeth, because it constrains a capability rather than a string. When a scenario forces a choice, the action boundary is usually the strongest option present.

Approval is scarce, so spend it on irreversibility

Human review works until there is too much of it, and a queue nobody reads is worse than no queue at all because it manufactures the appearance of oversight. Ration it by two properties of the operation: how hard it is to undo, and how far the damage spreads when it is wrong. Saving an internal draft scores low on both. Dropping a production table, moving money and contacting a customer score high.

Scope the approval as well as the trigger. Approving one specific operation differs from approving a category for a session, which differs again from a standing permission, and the right granularity depends on what the operator can realistically evaluate at the moment they are asked. Blanket confirmation on everything trains people to click through.

Roll out by exposure, not by confidence

A model-backed feature cannot be exhaustively tested because its input space is open. Graduated exposure with observation at each step substitutes for that. Run it in shadow first, where it executes and logs and its output is compared against whatever it is meant to replace without reaching anyone. Then a small cohort, then a larger one, with a defined signal under watch and a threshold at which you stop.

The stopping mechanism has to be faster than the release process. A configuration switch that disables the feature without rebuilding and redeploying is the difference between a five-minute incident and an hour of one. Write the rollback condition down before launch, because deciding what counts as bad enough during an incident produces slow answers and usually wrong ones.

Ground the answer, and let it say no

Fabrication is reduced by structure rather than by exhortation. Restrict the model to supplied material and require it to quote the passage it relied on, which makes an ungrounded claim visible instead of merely wrong. Explicitly permit an admission that the answer is not present, because a model asked for an answer will supply one. For anything numeric or checkable, verify it in code afterwards instead of trusting the generation.

Each of these has a price. Quotations lengthen output, verification adds a step, and permitted uncertainty means some users now get nothing where they previously got a confident error. That last trade is nearly always correct in a domain where a mistake is expensive, and scenarios are framed to test whether you will make it.

Where candidates go wrong

Common questions