Lab 02: Statistical Significance Engine

Series: AI Eval Labs · Phase: Foundations · Depends on: Lab 01

Core scope

• [ ] Bootstrap utility for model-vs-model deltas
• [ ] Output win margin with confidence interval and p-value
• [ ] Flag deltas that are noise at the current sample size

Stretch

• [ ] Power analysis (sample size needed to detect a given delta)

Links

• Code: series/ai-eval/02-significance-engine
• Demo: coming soon
• Write-up: coming soon on markrox.ai