CCAR-F questions and answers — Domain 4: Prompt Engineering

12 practice questions on Prompt Engineering & Structured Output, the domain weighted at 20% of the CCAR-F exam. Every answer explains why the distractors fail — that is the part that transfers to questions you have not seen before.

Question 1

An extraction pipeline occasionally receives JSON with a missing required field despite an explicit prompt instruction. What is the fix?

1. Repeat the instruction at the end of the prompt as well as the start
2. Use output_config.format with a JSON Schema so conformance is enforced at generation time
3. Add a stop sequence after the closing brace
4. Lower the temperature

Answer: B. Prompting for JSON is best-effort. Structured outputs constrain generation against your schema, so required fields are present and types are correct. Repetition, stop sequences and temperature all reduce the failure rate without eliminating the failure mode.

Question 2

What does strict: true on a tool definition guarantee?

1. The tool will be called at least once
2. Tool names and inputs conform to their schemas
3. The final assistant response is valid JSON
4. The tool result is validated before being returned to Claude

Answer: B. strict governs tool calls: names and inputs validate against their schemas. Constraining the final response is output_config.format. Neither validates your tool's output — that is your code's job. Use both together in agentic workflows that must also return a structured final answer.

Question 3

Which JSON Schema feature is NOT supported by structured outputs?

1. enum
2. anyOf
3. pattern (regex)
4. additionalProperties

Answer: C. Regex pattern is unsupported, as are allOf, not, if/then/else, dependentSchemas and recursive schemas. If you need a regex constraint, express the format in the field description and validate it yourself after generation.

Question 4

An invoice extraction returns a schema-valid object, but total does not equal the sum of the line items. What does this tell you?

1. The schema was configured incorrectly
2. Schemas enforce structure, not business correctness — you still need your own validation layer
3. Structured outputs are unreliable and should be replaced with prompting
4. The document must be re-scanned at higher resolution

Answer: B. A schema guarantees shape: the right fields, the right types. It says nothing about whether the values are true or internally consistent. Cross-field business rules — totals, date ranges, referential integrity — belong in a validation step after generation, feeding failures back into a targeted retry.

Question 5

An extraction fails validation. What should the retry prompt contain?

1. The original prompt only, resent unchanged
2. 'The previous output was invalid. Try again.'
3. The specific validation error and the offending value, e.g. 'line_items[2].amount was N/A; it must be an integer in cents'
4. A larger set of few-shot examples on every retry

Answer: C. A retry is only useful if the model learns what was wrong. Resending unchanged reproduces the same failure; a generic 'that was invalid' gives no signal. Feed back the field, the value and the constraint, cap the attempts, and route persistent failures to human review with the partial result attached.

Question 6

Which technique should you try FIRST when a prompt produces inconsistent output formats?

1. Fine-tune a custom model
2. Add examples showing the desired format
3. Increase max_tokens
4. Switch to a larger model

Answer: B. Examples are the strongest and cheapest lever for consistency of format and judgement. Prompt-engineering questions reward the most direct intervention; model swaps and fine-tuning are distractors when the defect is an underspecified prompt. Include edge cases and near-misses, not only happy paths.

Question 7

You are analysing a 90-page contract. How should the prompt be structured?

1. Question first, then the document, so the model knows what to look for
2. Document first inside XML tags with an ID, question last, and a request to quote relevant passages before analysing
3. Split into 90 single-page prompts and average the answers
4. Summarise the contract first, then ask the question about the summary

Answer: B. Long-context practice: long material first, question last, each document tagged and identified, and grounding requested by asking for quotes before analysis. Summarising first discards the detail the question probably depends on; page-by-page averaging destroys cross-reference. Multi-pass extraction is a valid pattern, but it merges structured results rather than averaging answers.

Question 8

What is the purpose of prefilling the assistant turn with {?

1. It reduces token cost
2. It forces the response to begin as JSON and skips conversational preamble
3. It disables tool use
4. It guarantees schema conformance

Answer: B. Prefill steers format and removes preamble. It is a prompting technique, not a guarantee — for an actual guarantee of schema conformance you want output_config.format. Combining the two is unnecessary; the schema already constrains the whole response.

Question 9

Your schema-constrained call returns output that does not match the schema. Which cause is plausible?

1. The schema was cached and went stale
2. stop_reason was refusal or max_tokens
3. Structured outputs only apply to the first response in a conversation
4. The temperature was above zero

Answer: B. Two documented edge cases: a safety refusal (stop_reason: "refusal") may not match the schema, and hitting max_tokens truncates the JSON mid-structure. Both are reasons to keep checking stop_reason even when a schema is in force. Schema grammars are cached for performance, not staleness; temperature does not defeat the constraint.

Question 10

When is chain-of-thought prompting the wrong choice?

1. Complex multi-step analysis
2. Mathematical reasoning
3. High-volume, latency-sensitive classification with a fixed label set
4. Debugging an ambiguous failure

Answer: C. Chain of thought buys accuracy on genuinely analytical work at the cost of tokens and latency. For a simple classification into known labels at volume, it adds cost and delay without improving the decision — and it makes the response harder to constrain to a bare label.

Question 11

What should a CI review prompt specify to produce machine-consumable feedback?

1. Only the severity threshold
2. Severity thresholds, a fixed output shape (file, line, severity, finding, suggested fix), and examples of both a reportable and a suppressible finding
3. A request for a friendly tone
4. A maximum word count

Answer: B. Machine-consumed output needs a fixed shape so the pipeline can parse it, a threshold so the volume is right, and examples on both sides of the line so the model calibrates what counts. Showing the negative case — a finding that should be suppressed — is what actually reduces false positives.

Question 12

Which pairing correctly matches mechanism to purpose?

1. output_config.format validates tool inputs; strict validates the final response
2. output_config.format constrains the final response; strict constrains tool names and inputs
3. Both constrain the final response; they differ only in syntax
4. Both constrain tool calls; output_config.format is the newer form

Answer: B. Two complementary features. JSON outputs via output_config.format guarantee the shape of Claude's final response — extraction, classification, API payloads. Strict tool use guarantees the shape of tool calls — the thing that breaks downstream systems in agentic workflows.

← Previous: Questions & answers — Domain 3  |  Next: Questions & answers — Domain 5 →