GenAI Basic — 151: stress-test refusal paths on `Benchmarks read with scepticism` — memo `399254 [151]`

Lesson 151: Benchmarks read with scepticism

Focus

Treat placeholders as compulsory—swap nouns immediately after reading. Token Benchmarks read with scepticism:151 keeps neighbouring lessons differentiable.

Key ideas

• Thread: Benchmarks read with scepticism · drill v1 · spin 465143.
• Habit: pair every model utterance with a trace_id you could paste into Grafana.
• Guardrail: write one RACI bullet referencing this lesson tomorrow.

Deep dive notebook

Synthetic drill artefacts

Prompt scaffold

ROLE: Incident analyst cohort 54
INPUTS:
- excerpts tagged [chunk_id ...]
- guardrails referencing policy_bundle_9
TASK:
  1) Summarize deltas with citations
  2) Confidence label LOW|MED|HIGH + evidence
  3) If facts missing emit MISSING_FACTS list
USER_SEED_QUESTION >>> What changed between rollout 35 and 34?

Practice

Practice Simulate degraded retrieval once; screenshot graceful degradation copy. — 151 Bump literals mindset by 15.