Course artifact
Pilot reliability scorecard
A reliability and game-day checklist for deciding whether an inference pilot is ready for production.
Pilot reliability scorecard
| Area | Evidence required | Result |
|---|---|---|
| Service contract | p95/p99 latency, availability, quality, queue and error policy | |
| Release | immutable version, smoke test, shadow/canary, rollback | |
| Capacity | warm pool, burst startup, batch policy, shortage behavior | |
| Data | flow diagram, residency, retention, deletion, access logs | |
| Security | secrets, tenant isolation, image provenance, vulnerability path | |
| Incident | owner, pager, status updates, recovery and postmortem | |
| Economics | achieved utilization, idle, failures, support, gross margin |
Game-day cards
- A GPU disappears during a burst.
- Model weights fail integrity validation.
- A prompt exceeds the normal context by 20×.
- A client disconnects mid-stream.
- The preferred region has no new capacity.
For each: symptom, automatic protection, operator action, recovery target, prevention.