The 14 CCAR-P anti-patterns: wrong answers, catalogued

Fourteen wrong answers that recur across the CCAR-P question bank. Several are Foundations-level traps re-asked at organisational scale; the rest exist only at this level.

1. Choosing an architecture before the constraints are known

Wrong: Selecting a pattern from the functional requirement alone.

Right: Establish latency budget, data residency, retention, review requirements and success measures first. A design that meets the functional need and violates a constraint found later is the canonical professional failure.

2. Reaching for agentic where a workflow satisfies the requirement

Wrong: Autonomy because it is more capable and more flexible.

Right: Agentic systems are harder to bound, audit and cost-forecast. In a regulated context that is often decisive on its own. Justify autonomy from a requirement in the stem, never from ambition.

3. Optimising before instrumenting

Wrong: Switching model tiers because costs look high.

Right: Measure cost per request and find where it concentrates first. The right lever — caching, routing, batching, trimming scope — depends entirely on that answer, and it is frequently not the model.

4. Changing prompt, model and retrieval together

Wrong: Shipping several improvements at once to move faster.

Right: One variable per measured change, or the result cannot be attributed and cannot be rolled back intelligently.

5. Treating a vendor certification as your compliance case

Wrong: "They hold SOC 2 and ISO 42001, so we are compliant."

Right: A vendor attestation is an input to your case. Lawful basis, minimisation, retention, access control and audit evidence for your own system remain yours.

6. Hardening the prompt where the capability could be removed

Wrong: Defending against tool misuse with system-prompt instructions.

Right: Architectural controls beat prompt controls. Prompt defences lower the probability of a successful attack; removing the capability removes the possibility of damage.

7. One privileged service account for the whole agent

Wrong: The agent queries the database with a single application identity.

Right: Propagate the caller's identity and enforce authorisation at the data layer, so the agent inherits the user's permissions. Otherwise any user can be returned any row.

8. Blaming the model for a retrieval failure

Wrong: Diagnosing ungrounded answers as hallucination and rewriting the prompt.

Right: Check what was retrieved first. In RAG systems a large share of "hallucination" reports are the model answering faithfully from chunks that never contained the answer — a chunking, enrichment, hybrid-search or reranking problem.

9. Requiring human review without designing it

Wrong: Adding a review step to satisfy a control, with no thought to volume or turnaround.

Right: Decide which cases are reviewed, give the reviewer the evidence and citations, measure the override rate, and size the queue. A review that creates a backlog gets bypassed, and then the control exists only on paper.

10. Committing to accuracy in an SLA

Wrong: Promising a correctness guarantee on arbitrary inputs.

Right: Commit to availability, latency percentiles, accuracy against a defined eval set and threshold, escalation turnaround, and time to detect degradation. Convert a demand for a guarantee into a measurable target plus a review path for the residual.

11. Monitoring infrastructure instead of behaviour

Wrong: Error rates, uptime and latency dashboards only.

Right: The characteristic failure is a successful-looking response with wrong content. Track escalation and override rates, validation failures, guardrail triggers, retrieval hit rates, refusals and cost per transaction.

12. An unvalidated LLM judge

Wrong: Building an eval pipeline on a model grader nobody checked against human labels.

Right: Validate the judge before scaling it, with a specific empirical rubric and a parseable output format. An unvalidated instrument produces confident numbers about nothing.

13. Presenting one design as the only option

Wrong: Bringing a stakeholder a single recommendation with no alternatives or costs.

Right: Two or three viable designs with cost, latency, accuracy and risk profiles, plus a recommendation and its reasons — so the decision-maker makes a decision rather than ratifying yours.

14. Treating a deployed system as finished

Wrong: It works, so leave it alone.

Right: Model versions change, data distributions shift and regulations move. Schedule eval re-runs, refresh the eval set from real traffic, re-check the original constraints, and keep an explicit decision point to keep, rework or retire.

← Previous: Domain 7 — Developer Productivity and Operational Enablement  |  Next: Cheat sheet →