You need to change your decision model, but you’re not sure how it’ll impact your KPIs. Will there be unexpected effects? Batch experiments allow for exploration through summary statistics to orient yourself to impacted metrics.

You’ve got a production decision model and an updated decision model. Should you ship the new model to prod? Will the new model meet your acceptable KPI thresholds? Acceptance testing provides the answers by delivering a documented, repeatable decision-making process.

Do you launch decision model A or B? What happens if an operator makes a manual override? How does a new optimization model perform against real-world data? Testing and experimentation has the answers, but getting them has traditionally been a challenge.

Create a release process for your custom model that mirrors your experience for shipping any other software. Develop locally, deploy to operational environments, test performance, use version control for easy rollbacks, and iterate with confidence.

We built a docs site that references code samples from external files to avoid copy-paste errors, makes it easier to manage code updates over time, and allows for separate testing processes to be run to validate referenced code. Here’s how.