Build & operate
Production reliability
Design retries, fallbacks and rollout evidence together
Reliability is an end-to-end deadline, not an unlimited retry count. Combine route eligibility, provider health, bounded fallback and application cancellation while preserving protocol semantics.
Use one retry budget
Keep Router attempts inside the caller's deadline. Retry only classified transient failures and apply backoff to rate limits; authentication, policy and validation errors require a request change.
Handle partial streams
After output begins, a replacement attempt may duplicate text or side effects. Mark incomplete output, avoid hidden replay and let the application decide whether a user-visible retry is safe.
Promote with evidence
Validate contract drift, provider qualification, shadow policy behaviour and rollback before shifting traffic. Public status and build identity should distinguish source, candidate, shadow and deployed state.