CCAR-P practice questions, answered and explained

A compliance review must check each of 300 contracts against the same twelve clauses and produce one consolidated exceptions report. Reviews are nightly and completeness matters more than latency. Which structure is most appropriate?

• A Parallel workers each appending directly to the shared exceptions report

Direct appends give no opportunity to reconcile, so the same clause failure appears several times in different words.

• B A single agent looping over all 300 contracts in one session

300 contracts will not fit one context, and quality degrades well before the limit as earlier material is crowded out.

• C Parallel workers over contract subsets, with a coordinator that consolidates and de-duplicates exceptions

Correct. Independent work parallelises, and the consolidation step is what a coordinator exists to do.

• D A sequential pipeline passing each contract's findings into the next contract's review

Sequencing implies a dependency that does not exist here, and it serializes work that could run concurrently.

The contracts are independent, so there is nothing for sequencing to exploit, and 300 of them will exhaust a single context. Parallel workers give coverage; the coordinator is what turns 300 partial results into one report without duplicate or contradictory entries. Workers writing straight to a shared report produce exactly those inconsistencies with nothing to reconcile them.

An agent can call a tool that issues customer refunds. The business wants refunds under $50 to be automatic and anything larger approved by a human. Where does that rule belong?

• A In the tool's description, so the model knows not to exceed the limit

A description is text sent to the model like any other, so it constrains no more strongly than an instruction does.

• B Enforced in the refund tool itself, which refuses amounts above the threshold without an approval token

Correct. The tool enforces the threshold deterministically, so the limit holds regardless of what the model concludes.

• C In a post-hoc review that reverses any over-threshold refund the next day

After-the-fact reversal concedes that money already moved, which is precisely what the threshold exists to prevent.

• D In the system prompt, as an instruction the model must follow

A system prompt is context the model weighs; an injected instruction or an unusual case can talk it out of a limit.

A money-moving limit is a control, and controls belong where they cannot be argued with. The tool holds the threshold and refuses without an approval token, so compliance does not depend on the model's judgement or on the prompt surviving injection. Prompt and description are both instructions the model weighs against other context, and reversing refunds after the fact means the control has already failed.

In a twelve-worker fan-out summarising regional reports, two workers fail after retries. What should a well-designed coordinator do?

• A Retry the two workers indefinitely until they succeed

Unbounded retries block the run indefinitely if the failure is permanent rather than transient.

• B Return the ten results, stating explicitly which two regions are missing

Correct. Partial coverage is useful when its boundaries are explicit; that is what makes the degradation safe.

• C Silently drop the two regions and present the remainder as the result

This is the dangerous option: the output looks complete, so downstream consumers act as though two regions were checked.

• D Abort the run so no partial result is mistaken for a complete one

Aborting throws away ten valid results to avoid a problem that stating the gap already solves.

Reliability in a fan-out means degrading visibly. Ten twelfths of the coverage is worth having provided the gap is stated, so the consumer knows what the result does not cover. Aborting discards good work, unbounded retries stall the run behind a possibly permanent failure, and dropping silently converts partial coverage into a false claim of completeness — the only genuinely dangerous option.

A long-running agent's quality degrades over a session even though nothing exceeds the context limit. What is the most likely cause?

• A The model gets progressively less capable the longer a session runs

There is no fatigue mechanism; what changes across a session is the composition of the context, not the model.

• B Long sessions increase the probability of sampling errors

Sampling behavior does not degrade with session length; the input is what changed.

• C Context is processed more slowly as it grows, forcing shallower reasoning

Processing time and reasoning depth are not linked in the way this suggests.

• D Accumulated tool output and intermediate reasoning are crowding out the instructions and the current task

Correct. Relevant material becomes a shrinking fraction of the window, so it competes with accumulated noise.

Staying under the limit is not the same as using the window well. As transcripts, tool results and intermediate reasoning accumulate, the material that matters — the instructions and the current subtask — becomes a smaller share of what the model is attending to. The fix is curation: summarize or evict what is no longer needed. The other options describe mechanisms that do not exist.

A retrieval-backed assistant answers correctly when the right passage is retrieved but fabricates when it is not. Which change most directly addresses the failure?

• A Lower the temperature so output is more deterministic

Temperature affects variability. A deterministic fabrication is still a fabrication.

• B Use a more capable model, which fabricates less

A better model fabricates less often, which is a smaller version of the same problem rather than a solution to it.

• C Instruct the model to answer only from the supplied passages and to say when they are insufficient, and verify it does

Correct. The gap is that absent evidence is not treated as a stopping condition; making abstention explicit and testing it is the fix.

• D Increase the number of passages retrieved so the right one is more likely to appear

More passages lower the frequency of the failure but leave the behavior unchanged when retrieval still misses.

The failure is behavioral: the model treats absent evidence as license to reconstruct. Making abstention an explicit, tested behavior addresses it directly. Retrieving more passages reduces how often the gap occurs without changing what happens when it does, a stronger model narrows the gap but does not close it, and temperature governs variability rather than whether the model requires grounding.

A pipeline classifies incoming tickets, then drafts responses for the subset needing a written reply. Which routing design is most defensible?

• A Use the most capable model for both stages for consistency

Consistency is not the goal; the two stages differ in difficulty, and paying top capability for classification is spend without return.

• B Use a small model for classification and a more capable one for drafting, with the classification accuracy monitored

Correct. Match capability to task difficulty, and monitor the cheap stage because its errors are silent and propagate.

• C Use a small model for both stages and escalate only on customer complaint

Escalating on complaint means the customer discovers the failure, which is the most expensive possible detection point.

• D Alternate models by time of day to balance cost and quality

Time of day has no relationship to task difficulty, and it makes identical inputs behave differently.

The stages have different difficulty profiles: classification into known categories is a narrow decision a small model handles well and cheaply, while drafting a customer-facing reply is open-ended and benefits from capability. Monitoring matters because a classification error silently routes work wrongly. Uniform maximum capability overpays for the easy stage, escalating on complaint uses customers as the test suite, and time-based alternation makes behavior depend on something irrelevant.

A tool returns a 4,000-row result set that the agent must reason over, but only a handful of rows are ever relevant. What is the best design change?

• A Add filtering and pagination parameters so the tool returns only what was asked for

Correct. Selection belongs where the data is, so the tool returns the relevant subset instead of everything.

• B Instruct the model to ignore irrelevant rows

The rows still occupy the context and compete for attention whether or not the model is told to ignore them.

• C Summarize the result set with a second model call before the agent sees it

An extra call adds latency and a lossy compression step to work around an interface problem that has a direct fix.

• D Truncate the result to the first fifty rows

First-fifty is arbitrary with respect to relevance and will often exclude exactly the rows that mattered.

A tool that cannot be asked a narrow question forces the caller to pay for breadth it does not need, in tokens and in attention. Filtering and pagination move the selection to where the data lives. An instruction to ignore rows still spends the context, truncation is an arbitrary cut that may drop the relevant rows, and a summarization call adds latency and a lossy step to compensate for an interface that could simply be better.

Two MCP servers each expose a tool named `search`, one over internal documents and one over the public web. Calls are being routed to the wrong one. What is the correct fix?

• A Instruct the model in the system prompt to prefer the internal server

A standing preference for one server produces the opposite error whenever the other is the appropriate one.

• B Remove one of the servers so the ambiguity cannot arise

Both capabilities are presumably wanted; discarding one solves the routing problem by removing a feature.

• C Give each tool a distinct name and a description stating exactly what it searches and when to prefer it

Correct. Selection is made from names and descriptions, so that is where the ambiguity has to be resolved.

• D Order the servers so the intended one is registered first

Registration order is not a selection criterion, so relying on it is relying on an implementation detail.

Routing is driven by names and descriptions, so two tools that look alike will be confused whatever else is done. Distinguishing them at the point of selection fixes the cause. Removing a server sacrifices a capability, a system-prompt preference is a blanket rule that will be wrong whenever the other server is the right choice, and registration order is not a documented selection criterion.

An agent updates records in a CRM. Occasionally a network timeout leaves the agent unsure whether its write succeeded. What design property most directly addresses this?

• A Idempotent writes keyed on a client-supplied identifier, so a retry cannot create a duplicate

Correct. A client-supplied key makes the retry recognisable as the same operation, so repetition is safe.

• B Retrying the write until a success response is received

Retrying a non-idempotent write is the mechanism that produces duplicates, not a defense against them.

• C Logging every attempt for reconciliation the next day

Reconciliation is after-the-fact repair; the record is wrong for everyone reading it in the meantime.

• D Reducing timeout duration so failures are detected sooner

A shorter timeout increases how often the outcome is unknown, making the underlying problem more common.

Ambiguous failures are unavoidable in distributed systems, so the write must be safe to repeat. A client-supplied key lets the server recognize a retry as the same operation. Retrying without idempotency is exactly what creates duplicates, next-day reconciliation cleans up damage after the CRM has already been wrong all day, and a shorter timeout makes ambiguous outcomes more frequent rather than fewer.

A team evaluates a summarization feature by having an engineer read ten outputs each release and judge them acceptable. What is the most significant weakness?

• A There is no fixed set of cases with agreed expectations, so results are not comparable between releases and regressions are invisible

Correct. Without fixed cases and agreed expectations, two releases are never measured against the same thing.

• B Ten samples is by any measure far too few for the judgement to be statistically meaningful in any useful sense of the word here

Sample size is a real limitation, but a larger ad-hoc sample judged fresh each time still cannot detect a regression.

• C Engineers are not qualified to judge summary quality, which is a matter for the people who read them

Engineers can judge summaries perfectly well; the problem is that the judgement is not anchored to anything stable.

• D The evaluation should be automated in order to save the engineering time it currently consumes

Automation is worth having, but an automated ad-hoc evaluation has the same defect more cheaply.

The purpose of an eval harness is comparability: the same cases, the same criteria, so a change in score means a change in the system. Ad-hoc samples judged fresh each time cannot support that, which is why regressions slip through. Sample size matters but is secondary to having a fixed set at all, engineer judgement is a reasonable signal, and automation is a means rather than the missing property.

After a prompt change, aggregate eval scores are unchanged but users report the assistant now refuses reasonable requests. What does this most likely indicate?

• A The eval set does not contain the cases where behavior changed, so the aggregate hides a real regression

Correct. The reports point at cases the set does not cover, and adding them makes the regression visible and testable.

• B Aggregate scores are the wrong metric and should be replaced with user reports

User reports are a valuable signal but lag and are unrepeatable; the fix is to extend the eval set, not abandon it.

• C The prompt change had no effect and the reports are coincidental

Coincidence is a weak explanation when reports follow a specific change and describe a specific behavior.

• D Users are mistaken, since the eval scores are the objective measure

An eval measures what it contains. Where users and the harness disagree, the harness's coverage is the thing in question.

A stable aggregate over a set that lacks the affected cases says nothing about those cases. The reports are evidence of a gap in coverage, and the response is to add the failing cases to the eval set so the regression becomes measurable. Treating the score as authoritative over direct observation inverts the relationship, and replacing evals with user reports discards a repeatable measure for a lagging one.

A support assistant sends the same 6,000-token policy preamble on every request, followed by a short user question. Volume is high and the preamble rarely changes. What is the most effective optimization?

• A Cache the stable prefix so the unchanging portion is not reprocessed on every request

Correct. A stable prefix with a variable suffix is the canonical caching case: same behavior, less reprocessing.

• B Move the preamble after the user's question so it is processed last

Ordering does not change how much is processed, and putting stable content last defeats caching.

• C Split each request into two calls, one for the preamble and one for the question

Two calls sever the link between policy and question, so the second call answers without the grounding.

• D Shorten the preamble by removing detail until the cost is acceptable

Removing detail reduces cost by reducing the grounding, which changes answer quality rather than optimising it.

A large, stable prefix in front of a small variable suffix is exactly the shape prompt caching exists for, and it addresses cost and latency without changing behavior. Cutting detail trades away the grounding the answers depend on, reordering does not reduce what is processed, and splitting into two calls loses the connection between the policy and the question it should inform.

An agent summarizes inbound emails and can call a tool that forwards messages. A crafted email contains text instructing the agent to forward the mailbox contents to an external address. What is the most robust mitigation?

• A Use a more capable model, which is less likely to be misled

Capability raises the bar without changing the fact that the model is reading attacker-controlled text.

• B Scan inbound email for instruction-like phrasing and reject matches

Detecting instruction-like phrasing is an arms race; paraphrase and indirection defeat it while false positives block real mail.

• C Instruct the agent in its system prompt to ignore instructions found inside emails

That instruction competes with the injected one inside the same context, and sometimes it loses.

• D Require forwarding to be confirmed out of band, and restrict destinations to an approved list enforced by the tool

Correct. Constrain what the action can do rather than relying on the model to reason correctly about hostile input.

Untrusted content will sometimes win an argument with a system prompt, so the durable defense constrains the consequence rather than the reasoning: the tool enforces an allow-list and a human confirms. An instruction to disregard embedded instructions is itself just context, phrasing detection is an arms race against paraphrase, and a stronger model narrows the attack surface without eliminating it.

An agent must query a database that requires credentials. What is the safest arrangement?

• A Place the credentials in the system prompt so the agent can authenticate when needed

Anything in the system prompt is in the model's context, where it can be echoed, logged, or extracted.

• B Store the credentials in a tool description the model can read when required

A tool description is sent to the model like any other text; this is the same exposure with a different label.

• C Have the tool hold the credentials and expose only the query capability, so the secret never enters the model's context

Correct. The tool exercises the capability while the value never enters the conversation, so there is nothing to leak.

• D Ask the user to supply the credentials at the start of each session

This routes the secret through the conversation and burdens the user every session, for no security gain.

A secret in context can be echoed into a response, written to a transcript or log, or extracted by an injected instruction. Keeping it inside the tool means the capability is exercised without the value ever being visible to the model. The prompt and the tool description are both context; asking the user each session moves the secret into the conversation while adding friction.

After an agent takes an incorrect action in production, the team needs to establish why. Which record is most useful?

• A Aggregate metrics showing error rates around the time of the incident

Aggregates tell you that something happened and roughly when, not why this request went wrong.

• B The inputs, retrieved context, tool calls with arguments, and results, correlated by a request identifier

Correct. Inputs, retrieved context and the tool-call trace are what let a specific decision be reconstructed.

• C The final response the agent returned to the user

The response is the outcome under investigation; it does not show what led to it.

• D The system prompt in force at the time

The prompt is one input among several and cannot explain the action without the retrieved context and tool results.

Reconstructing a decision requires what the agent saw and what it did: inputs, retrieved context, the tool calls and their arguments and results, tied together so one request can be followed end to end. The final response is the symptom, aggregate metrics locate an incident without explaining it, and the system prompt is necessary but far from sufficient on its own.

A stakeholder asks for an agent that will "handle all customer emails automatically." What is the most productive first response?

• A Decline, since fully automatic handling of every customer email is not an achievable goal

A useful version of this is achievable; refusing the whole request discards it along with the unrealistic framing.

• B Agree and begin the work straight away, refining the scope as issues emerge over the course of the build itself as they arise

Refining scope as issues emerge means discovering the disagreement after the build, which is the expensive moment to find it.

• C Establish which categories can be handled end to end, which need review, and which must escalate — and set the accuracy bar for each

Correct. Partitioning by category and agreeing a bar for each turns an unbounded ask into something that can be delivered and judged.

• D Propose a smaller pilot first, without changing the eventual goal that was described

A pilot is sensible, but without a partitioned scope it just reaches the same ambiguity later.

"All" hides a distribution: some categories are safely automatable, some need review, and some must reach a person. Partitioning the work and agreeing an accuracy bar per category converts an unbounded ask into a deliverable and sets expectations before anything is built. Starting without that defers the disagreement, refusing rejects a workable version of the request, and a pilot with the same undefined goal postpones the same conversation.

An agentic system is being handed to an operations team who did not build it. Which artifact matters most for their ability to run it?

• A The full prompt history from development

Prompt history documents how the system was arrived at, which is rarely what an operator needs.

• B Benchmark results from the final evaluation run

A benchmark is a snapshot of one moment and says nothing about behavior under failure.

• C A diagram of the agent's internal orchestration

A diagram supports understanding but does not tell the on-call engineer what to do when something breaks.

• D The failure modes with their symptoms, the checks that detect them, and what to do about each

Correct. Symptoms, detection and response is precisely the knowledge that transfers operational capability.

An operations team's job is recognising and responding to failure, so the artifact that carries the most weight is a runbook of failure modes, detection and response. Architecture diagrams help them build a mental model, prompt history is development archaeology, and a final benchmark describes one moment rather than telling them what to do at 3am.

A prompt change is expected to improve one category of request but might affect others. The system is live. What is the most appropriate rollout?

• A Deploy behind a flag that a person can toggle if complaints arrive

A flag is a good rollback mechanism but complaints are a slow, lossy detector for a category-level regression.

• B Deploy to everyone, monitoring aggregate quality metrics

An aggregate can stay flat while one category degrades and another improves, which is exactly the risk described.

• C Run the change against the eval set, then roll out to a slice of traffic with per-category metrics before going wider

Correct. Eval first for known cases, then a slice with per-category metrics for the ones the eval set does not cover.

• D Deploy during a low-traffic window so fewer users are exposed

Fewer users exposed is not the same as noticing the problem; detection is unchanged.

The stated risk is category-specific, so detection has to be category-specific too: the eval set catches known regressions before any user sees them, and a traffic slice with per-category metrics catches what the eval set does not cover. An aggregate metric can hide a regression in one category behind an improvement in another, a quiet window reduces exposure without improving detection, and waiting for complaints uses users as the detector.

A team wants an agentic coding assistant to follow their conventions — test framework, directory layout, review expectations — without restating them in every session. What is the appropriate mechanism?

• A Reviewing the assistant's output against the conventions after the fact

Post-hoc review finds violations after the work exists, which is the expensive point to correct them.

• B Project-level configuration committed to the repository, so the conventions load with the codebase

Correct. Committed configuration versions with the code, applies to everyone, and cannot be forgotten.

• C A shared document that each developer pastes at the start of a session

Pasting depends on memory and discipline, and it drifts as the document and the codebase diverge.

• D A team norm that developers describe the conventions when they matter

Relying on people to notice when a convention is relevant fails in exactly the cases where they do not.

Conventions belong with the code they govern: committed to the repository, they version with it, apply to everyone automatically, and cannot be forgotten. Pasting depends on each developer remembering, a norm to mention conventions when they matter fails precisely when someone does not realise they matter, and post-hoc review catches violations after the work is done.

Six months after a successful pilot, adoption across the wider organization has stalled. Usage data shows most people tried the tool once and did not return. What is the most useful first investigation?

• A Find out what those first sessions were used for and where they fell short, since single-use suggests the first attempt failed to deliver

Correct. Single-use is a signal about first experience, so the question is what was attempted and how it fell short.

• B Run additional training sessions covering the same material again, on the basis that people simply did not absorb it properly the first time round

More training assumes people did not understand, which is one hypothesis among several and not yet the evidenced one.

• C Add capabilities, since the tool is evidently not powerful enough to hold anybody's interest beyond a single use

Adding capability assumes the tool was insufficient, which the data does not show — most people barely used it.

• D Mandate use so that people build the habit, which is what turns a trial into an established practice

A mandate produces compliance rather than adoption and leaves the original obstacle in place.

A try-once-and-stop pattern points at a first-use failure, and the diagnostic question is what people attempted and what they got. Repeating training assumes the problem was comprehension, adding capabilities assumes the tool was too weak, and mandating use forces the behavior without addressing why it did not happen — all three commit to a cause before it has been established.