GenAI Basic — 156: debate multimodal quotas on `Benchmarks read with scepticism` — memo `596950 [156]`

Lesson 156: Benchmarks read with scepticism

Focus

Prefer explicit failure rehearsals over aspirational wording. Token Benchmarks read with scepticism:156 keeps neighbouring lessons differentiable.

Key ideas

• Thread: Benchmarks read with scepticism · drill v6 · spin 613874.
• Habit: pair every model utterance with a trace_id you could paste into Grafana.
• Guardrail: write one RACI bullet referencing this lesson tomorrow.

Deep dive notebook

Synthetic drill artefacts

Red-team tableau

1. Actor profile **exec_assistant**
2. Injection meme `OVERRIDE++463`
3. Detector gate `moderation-tier-3` + human pager `oncall-ai-3`
4. Telemetry fields `trace_id,user_bucket,redaction_notes`

Practice

Practice Draft three eval assertions QA must greenlight before launch. — 156 Bump literals mindset by 27.