CCAR-P practice questions, answered and explained

Which context content should be evicted first when a long-running agent approaches its limit?

• A The system prompt, which has already shaped everything that was done so far

The system prompt governs behavior throughout and is the worst thing to drop.

• B The current task description, which the agent has been working from

The current task is what the agent is working on.

• C The most recent exchange, since it is the least embedded in the work

Recent exchanges are the immediate working state.

• D Raw tool output whose conclusions have already been drawn and recorded

Correct. The value has already been extracted, so the raw form is redundant.

Raw output that has already been reduced to a conclusion is the cheapest thing to lose: the value has been extracted. The system prompt, the current task and recent exchanges are all load-bearing for what happens next.

A team's eval scores have been flat for months while user complaints rise. What is the most likely explanation?

• A Users are simply mistaken about what they are experiencing

Rising complaints are evidence, not error, and dismissing them keeps the eval misleading.

• B The scoring is broken and is not reflecting quality

Broken scoring is possible but would more likely produce erratic scores than stable ones.

• C The model has not changed, so nothing can have got worse

An unchanged model is consistent with changed traffic, which is the point.

• D The eval set no longer represents what users actually do

Correct. Sustained divergence means the set has drifted away from what users actually do.

An eval measures the cases it contains. Sustained divergence between a flat score and rising complaints means production traffic has moved away from the set, which happens as usage evolves. Refreshing it from recent traffic is the fix. Dismissing users ignores the strongest available signal.

What is the most reliable way to detect gradual quality drift in production?

• A Running a fixed case set on a schedule and tracking the scores over time

Correct. The same cases over time is what makes a gradual trend visible.

• B Monitoring latency and error rates, which move when something is wrong

Latency and errors measure availability, not answer quality.

• C Waiting for user complaints to arrive and then treating them as the signal

Complaints lag drift substantially and under-report it.

• D Reviewing a sample of the output once a month and forming a judgement

Ad-hoc sampling has no stable baseline, so a slow trend is invisible.

Drift is gradual and invisible in any single observation, so it is detected by the same cases scored repeatedly over time. Complaints lag badly, ad-hoc sampling has no stable baseline to compare against, and latency and errors measure availability rather than quality.

A reflection loop is added so the agent critiques and revises its own output. What is the main risk?

• A The loop doubles the cost of every request that runs

Cost roughly doubles, which is a price rather than a structural flaw.

• B The loop makes latency less predictable across requests

Latency variance is a manageable operational consequence.

• C Self-critique shares the blind spots of the generation

Correct. Critique shares the blind spots of generation, so systematic errors survive.

• D Revision may introduce errors the first pass did not have

Revision does introduce some errors, but far fewer than the blind spots hide.

A model critiquing its own output uses the same representation that produced it, so systematic errors survive the review; the loop catches slips and misses exactly the failures that matter most. Cost, latency variance and revision-introduced errors are real and secondary to the correlation of blind spots.

Which failure mode is most likely to be missed in testing an agentic system?

• A A tool timing out under production load

Timeouts are a standard case that load testing exercises directly.

• B A tool returning an unexpected response format

A format surprise fails visibly at the parse step.

• C A tool returning an error the agent does not handle

An unhandled error is loud and shows up the first time it happens.

• D A tool returning plausible but incorrect data

Correct. Well-formed wrong data produces a confident answer with no signal.

Errors, timeouts and format problems all announce themselves and are routinely exercised in tests. A tool that returns well-formed, plausible, wrong data produces a confident downstream answer with no signal anywhere, and nothing in the test suite is watching for it.

A pipeline uses a large model for extraction and a small one for the final summary. What should be questioned?

• A Whether the models are being used the right way round

Correct. Extraction is usually the easier task and summarization the harder one.

• B Whether the small model can handle the summary length

Length is a capacity question rather than an allocation one.

• C Whether the two stages need different models at all

Worth asking, though it does not address the apparent inversion.

• D Whether the extraction output format suits the small model

Format compatibility is a detail within whichever allocation is chosen.

Extraction from structured source material is usually the easier task, while summarization involves selection and judgement, so this allocation may well be inverted. Whether differentiation is needed, output length and format are all worth asking after the more basic question of which stage is actually harder.

An agent's MCP tool returns data that includes text resembling instructions. How should this be handled?

• A The tool should strip anything that resembles an instruction

Pattern stripping is a filter and filters get written around.

• B Tool results should be marked as data everywhere they appear

Correct. Labeling applies to every result, whether or not the payload is noticed.

• C The agent should be prompted to ignore instructions in tool results

A prompt instruction is exactly what a crafted payload is written to overcome.

• D The agent should confirm with the user before acting on it

Confirmation triggered by suspicious text only helps when the payload is noticed; gates belong on consequential actions instead.

The baseline defense is structural: results are labeled as data at every point they enter context, whether or not anyone notices the payload. Stripping and confirmation both depend on first recognizing instruction-like text, which is what a crafted payload is built to evade, and a prompt telling the model to ignore instructions is one more instruction the payload can argue against. Labeling reduces the risk rather than removing it, so consequential actions still need their own gates.

An integration reads from a system whose data model changes quarterly. What design responds best?

• A Translate at the boundary so the change is contained

Correct. A translation boundary confines the churn to one place.

• B Pin to the current schema and update on each change

Pinning without a translation layer spreads the dependency everywhere.

• C Negotiate a stable contract with the system's owners

Worth attempting, and not something the integrator can rely on.

• D Read the data generically without assuming a schema

Reading generically throws away structure the work depends on.

An anti-corruption layer confines schema churn to one translation point, so quarterly change costs one small edit rather than a sweep through the system. Pinning without translation spreads the dependency, generic reading discards the structure the work needs, and a negotiated contract is desirable and rarely available.

Which control limits what a successful injection can reach, however thoroughly it persuades the model?

• A Logging all tool calls so an incident can be reconstructed

Logging supports investigation after the impact has occurred.

• B Limiting what the agent's tools can do regardless of intent

Correct. Bounding the tools shrinks the blast radius whether or not the injection lands.

• C Filtering incoming content for instruction-like patterns

Pattern filters are evaded by rephrasing, which is cheap for an attacker.

• D Instructing the agent to disregard embedded instructions

An instruction is exactly what a crafted payload is designed to override.

Injection defenses that depend on the model can be written around, so the durable control is capability: bound what the tools can do and a successful injection reaches a smaller blast radius. Instructions are bypassable, filters are evadable, and logging documents the incident afterwards.

Under whose authority should an agent acting on a user's behalf reach a system?

• A Its own, so behavior is consistent across all users

A consistent identity lets one user reach what another's permissions exclude.

• B A shared one, so access can be managed centrally

Central management is convenient and detaches access from the requester.

• C An elevated one, so it can complete every request

Elevation guarantees completion by removing the constraint entirely.

• D The user's, so it can reach exactly what they can

Correct. Inheriting the user's permissions keeps the agent from being a way around them.

Acting on someone's behalf means inheriting their permissions, so the agent becomes a channel to what the user could already reach rather than a route around it. Consistent, elevated and shared identities all let the agent access material the requesting user could not.

What is the first thing an incident response for an agentic system needs?

• A The ability to notify all affected users

Notification matters once the situation has stopped changing.

• B A rollback of the change that caused the incident

Rollback addresses the cause after the effect has been contained.

• C A way to disable the agent's ability to act

Correct. While the agent can still act, the damage is still growing.

• D A record of what the agent did and when

The record is essential for what comes next and does not stop the harm.

Stopping the bleeding comes first: while the agent can still act, the damage is still growing and everything else is chasing it. Records, notification and rollback are all essential and all come after the ability to act has been removed.

How should a success criterion for an agentic system be expressed?

• A As a comparison against how people currently perform

A human baseline is a useful comparison rather than the goal.

• B As an outcome the business can observe and measure

Correct. An observable business outcome survives changes in implementation.

• C As a threshold the system must exceed before launch

A launch gate is a decision rule derived from the criterion.

• D As a target accuracy on a defined evaluation set

An accuracy target is an instrument and can be met while the outcome is missed.

The system exists to change something the business cares about, and a criterion expressed that way survives changes in implementation. Evaluation targets, human baselines and launch thresholds are all instruments for reaching the outcome rather than the outcome itself.

What is the risk of iterating on a system using production feedback alone?

• A Feedback is biased toward the most recent changes

Recency bias is a milder version of the same selection problem.

• B Feedback arrives too slowly to guide development

Slow feedback delays the work rather than pointing it in the wrong direction.

• C Users describe symptoms rather than underlying causes

Symptom-level reports are normal and still usable as evidence.

• D You optimize for the cases users bother to report

Correct. Reported cases are a biased sample, so silent failures go untouched.

Reported cases are a biased sample of failures, skewed toward users who complain and problems that are visible, so optimising against them leaves the silent failures untouched. Latency, recency bias and symptom-level reporting are all real and all secondary to the selection bias.

A team finds review of agent-written code takes longer than writing it themselves. What should be examined?

• A Whether the reviewers are experienced enough

Experienced reviewers are also slowed by a large undifferentiated diff.

• B Whether a more capable model would write better code

Better code in one large diff is still expensive to review.

• C Whether review is necessary for every change

Removing review addresses the cost by removing the control.

• D Whether the changes arrive in reviewable units

Correct. Review cost tracks the size and coherence of the unit being reviewed.

Review cost scales with the size and coherence of the change, and a large undifferentiated diff is expensive regardless of who wrote it or how good it is. Reviewer experience, model capability and skipping review all address the symptom rather than the shape of the work arriving.

A sceptical senior engineer is blocking adoption on their team. What is the most productive approach?

• A Demonstrate the tool working on an impressive example

A demonstration rarely speaks to the particular thing they are worried about.

• B Find out what specifically they expect to go wrong

Correct. The specific concern is either a real problem or a correctable misconception.

• C Escalate to their manager to remove the obstacle

Escalation turns a technical objection into a political conflict.

• D Start with the more receptive members of the team

Working around them leaves the objection intact and unexamined.

Experienced scepticism usually encodes a real concern, and surfacing it either yields a problem worth fixing or a misconception worth correcting. Escalation converts a technical objection into a political one, demonstrations rarely address the specific worry, and routing around the person leaves the objection alive and unexamined.

Which is the clearest statement of a component's responsibility?

• A It handles everything related to customer records

Everything related to is a territory rather than a responsibility.

• B It provides the customer service capabilities

Capabilities names an area without saying where it stops.

• C It manages the interaction with the billing system

Manages the interaction describes a relationship rather than a duty.

• D It answers whether a given order may be refunded

Correct. A question the component answers has an edge you can test against.

A responsibility stated as a question the component answers has an edge: you can tell what falls inside and what does not. Everything related to, capabilities and manages the interaction all describe territory without a boundary, so scope creeps without anyone noticing.

Which is the strongest reason to include a citation with each retrieved claim?

• A It helps the model stay grounded in the corpus

Citations report grounding rather than causing it.

• B It satisfies the organization's documentation standard

Compliance is a reason to include them rather than what makes them valuable.

• C It lets a reader check the claim against the source

Correct. A citation converts a claim into something a reader can verify.

• D It makes the answer look more authoritative to readers

Looking authoritative without being checkable is the failure mode.

A citation is worth having because it is checkable: it converts a claim into something a reader can verify. Appearing authoritative is the failure mode rather than the benefit, documentation standards are a reason to comply, and citations do not themselves improve grounding.

Which tool design most reduces the chance of an irreversible mistake?

• A A log entry written before the operation executes

Logging records the action without providing a chance to stop it.

• B A confirmation parameter the caller must set to true

A confirmation flag is set by the same reasoning that chose the call.

• C A detailed description warning about the consequences

A warning is advisory and does not constrain anything.

• D An operation that is staged and separately committed

Correct. Staging creates a real interval in which the change can be inspected.

Staging separates deciding from committing, so there is a real interval in which the intended change can be inspected and abandoned. A confirmation flag is set by the same reasoning that chose the call, warnings are advisory, and logging records the action without delaying it.

A proposed optimization saves fifteen per cent of cost and adds a week of work. What decides it?

• A Whether fifteen per cent exceeds the team's target

A percentage target is meaningless without the base it applies to.

• B Whether the team has a week available to spend

Availability determines when it happens rather than whether it should.

• C Whether the optimization adds ongoing complexity

Ongoing complexity is a real cost to weigh once the saving justifies the work.

• D The absolute saving against the cost of the week

Correct. Percentages have to be converted to absolutes before they can be compared.

Fifteen per cent of a small bill does not repay a week; fifteen per cent of a large one repays it quickly. The comparison has to be in absolute terms. Percentage targets, availability and added complexity all matter once the arithmetic favours doing it at all.

What should a handover include about the system's known failure modes?

• A A list of the failures observed during development

A list without conditions leaves the team unable to recognize them.

• B The changes attempted that did not resolve them

Failed attempts prevent repetition and do not enable handling.

• C The frequency at which each failure occurred

Frequencies help prioritize once the conditions are understood.

• D The conditions that produce them and what to do

Correct. Recognition and response are what the receiving team needs.

The receiving team needs to recognize a failure and respond to it, which means the triggering conditions and the handling. A bare list, frequencies and a record of failed attempts are all supporting detail around that.