LLMOps Advanced — 085: stress-test tool permissions on `Latency and cache strategy` — memo `241642 [85]`

Lesson 085: Latency and cache strategy

Focus

Prefer explicit failure rehearsals over aspirational wording. Token Latency and cache strategy:85 keeps neighbouring lessons differentiable.

Key ideas

• Thread: Latency and cache strategy · drill v5 · spin 168800.
• Habit: attach a trace_id to every completion you would paste into an ops dashboard.
• Guardrail: add one RACI bullet for prompt or index changes before tomorrow's standup.

Deep dive notebook

Synthetic drill artefacts

Tool contract rehearsal

{
  "name": "reindexSlice",
  "arguments": { "corpus": "corpus-8", "dry_run": true }
}

Preconditions: index lag < 16h
FAILURE: page platform if lag > SLA without VP memo.

Practice

Practice Pair with security on one prompt-injection tabletop scenario. — 85 Bump 23.