Systems Life Cycle

Systems Life Cycle is the smallest sub-skill inside the exam's largest domain, at 2.8 percent of the blueprint and one or two scored questions. Its scope is everything after the requirements are agreed and before the system is switched off, and it is examined separately because a model-backed system has a lifecycle shape ordinary software does not.

Two differences drive nearly every question. The first is that a prompt is a deployable artefact. It is versioned, reviewed, promoted between environments and rolled back like code, and a team editing prompts in a console while shipping code through a pipeline is running two lifecycles that will drift apart.

The second is that a core dependency changes underneath you on somebody else's schedule. Models are released, superseded and eventually retired, and each generation can differ in behaviour, in which request parameters it accepts, and even in how text is tokenised. A system with no plan for that has a maintenance phase which arrives as an emergency.

Read scenarios here for the phase being described. The intended answer is usually the control belonging to that phase, not the most thorough control available anywhere.

What the exam tests here

• •Treating prompts, tool definitions and evaluation sets as versioned artefacts that travel with the code
• •Choosing between shadow running, a staged rollout and a straight cutover for a given change
• •What belongs in a continuous integration gate when the output is not deterministic
• •Planning for model retirement: pinned identifiers, migration windows and re-baselining
• •Knowing which lifecycle phase a described activity belongs to and what its exit criterion is

Prompts move through the pipeline with the code

The artefacts determining behaviour are the prompt text, the tool definitions and their descriptions, the model identifier, and the settings around them. If any of those live somewhere the pipeline cannot see, you lose the properties that make software releasable: review before change, a record of who changed what, somewhere to try a change, and a rollback that is one revert rather than an act of memory.

The practical form is unremarkable. Prompts live in the repository, changes go through review like any other diff, and the evaluation set guarding them is versioned in the same commit. A prompt edit shipping without either is an untested production change.

Release strategies, and which change deserves which

Three approaches recur and they are not interchangeable.

Shadow running sends real traffic to the new configuration without showing anyone its output, which is the only way to measure a behavioural change against real inputs at zero user risk, paid for by duplicating every call. A staged rollout exposes a small slice of users and widens as the metrics hold, suiting changes whose effect shows up in aggregate. A straight cutover is fine for a change that is provably equivalent, such as a configuration key moving house.

Match the strategy to the blast radius: a new model generation behind the same prompt deserves shadow traffic, a reworded error message does not.

Integration gates for a non-deterministic component

The usual gate asserts an exact output and cannot survive here, so it changes shape rather than disappearing. Three layers are worth having.

Fast deterministic checks needing no model call: prompts render, tool schemas are valid, the model identifier is one the configuration recognises, no secret has crept into a template. A contract layer making a handful of live calls and asserting structure rather than wording, which catches a broken schema or a rejected parameter immediately. And the graded evaluation set, run on prompt, tool and model changes, reporting a score against a threshold rather than a verdict per case.

The first two run on every commit. The third gates the changes that can move behaviour.

The maintenance phase is dominated by dependency change

Model identifiers are exact strings and should be pinned deliberately, so an upgrade is a change you make rather than one that happens to you. The corollary is that pinning creates a deadline: a pinned model is eventually retired, and the migration window is the time you have to move.

Plan that move as a project rather than a swap. Behaviour differs between generations, parameters that were valid can be rejected outright by a newer model, and token counts can shift because the tokenizer itself changed, which moves both your context budgets and your costs. Re-run the evaluation set against the new identifier, re-measure the counts, and keep the previous pin available until the new one has held in production.

Where candidates go wrong

Common questions