Lesson 058: Evaluation harness depth
Focus
Assume an auditor reruns everything you claim; narrate checkpoints aloud. Token Evaluation harness depth:58 keeps neighbouring lessons differentiable.
Key ideas
- Thread: Evaluation harness depth · drill v8 · spin
440641. - Habit: pair every model utterance with a trace_id you could paste into Grafana.
- Guardrail: write one RACI bullet referencing this lesson tomorrow.
Deep dive notebook
Synthetic drill artefacts
Tool-contract rehearsal
{
"name": "forecastBacklog",
"arguments": {
"hours": 66,
"team": "team-5"
}
}
Preconditions documented in directory snapshot 202509
FAILURE: escalate if horizons > 720h without VP approval memo.
Practice
Practice Draft three eval assertions QA must greenlight before launch. — 58 Bump literals mindset by 32.