When reviewing vendor-supplied case studies or ROI decks, request the following:
- Baseline definitions (e.g., MAPE, bias) and the measurement period
- Comparison against a naive baseline and incumbent system
- Clear description of what changed (process vs algorithm)
- Evidence of whether improvements persisted after promotions, product launches, or supply shocks
- Inventory KPIs (fill rate, OTIF, turns, stockouts, obsolescence) and tradeoffs
Critical note: Vendor-published ROI is often optimistic; treat such materials as directional until validated by independent references.