Your candidate decision model is nearly ready for prime time. It’s passed acceptance tests, it’s performed valiantly in a shadow test using production data, and now you want to see how it performs in your operational environment. It’s time for switchback testing!
Switchback tests are similar to A/B tests, but specific to decision models. For example, last-mile delivery routing use cases have pooled resources for drivers to consider. Switchback testing accounts for the resulting network effects by randomizing which models are applied to given units of time.
Switchback tests provide further confidence in a decision model before rolling it out to production in full. The impacts of this experiment demonstrate not only changes in KPIs but broader effects, such as customer satisfaction, that would be hard to affect without a model making operational decisions. Join our team for a live demo of performing and analyzing the results of switchback tests.
Key topics
- Overview of switchback testing (and when to use it)
- Performing a switchback test on a decision model
- Q&A with engineers behind the features
In the meantime, give Nextmv a try for free — it only takes 5 minutes. And feel free to dive into our documentation or contact us to learn more.
