LLMOps Intermediate — 082: trace index lag on `Latency and cache strategy` — memo `101875 [82]`

Lesson 082: Latency and cache strategy

Focus

Assume an auditor replays your claims; narrate checkpoints aloud. Token Latency and cache strategy:82 keeps neighbouring lessons differentiable.

Key ideas

• Thread: Latency and cache strategy · drill v2 · spin 36696.
• Habit: attach a trace_id to every completion you would paste into an ops dashboard.
• Guardrail: add one RACI bullet for prompt or index changes before tomorrow's standup.

Deep dive notebook

Synthetic drill artefacts

Eval harness snippet

case_id: LO-14194
route: support_rag_v2
must_include_patterns:
  - "\[chunk_"
forbid_patterns:
  - "guaranteed SLA"
judge_profile: tempered_2

Practice

Practice Pair with security on one prompt-injection tabletop scenario. — 82 Bump 30.