Lab 12: Production Drift Monitor

Series: AI Eval Labs · Phase: Production Pipelines · Depends on: Lab 01, Lab 03

Core scope

• [ ] Nightly sampling job over simulated live traffic
• [ ] Run offline trajectory evals on samples
• [ ] Alert on quality decay trends

Stretch

• [ ] Input-distribution drift (embedding shift) alongside quality drift

Links

• Code: series/ai-eval/12-drift-monitor
• Demo: coming soon
• Write-up: coming soon on markrox.ai