Signy Whitt leads Grubhub’s data science team, which develops machine learning and operations research solutions for the popular food delivery app. The team builds a variety of models (e.g., scheduling, forecasting, estimation, etc.) designed to solve problems related to how long it will take a delivery to make it to a diner, when to send a driver, which driver to send, whether or not to have a driver pick up multiple orders, and so on.
We spoke with Signy about building and managing decision systems at Grubhub, how they use Nextmv to accelerate model development and success, collaboration across stakeholder interfaces, and what’s on the horizon for optimization at Grubhub.
This interview was edited for length and clarity.
What is a top challenge that comes to mind for you when applying optimization as a solution for different types of business problems?
Signy Whitt: Helping the business clearly understand the impact of the change optimization and decision science will have. Getting from “Here's a theoretical thing that we could do” to “This is something that's really going to fundamentally change how the business is operating” is an ongoing journey. Part of that challenge is how you translate the work into something the business understands and finding the right level of transparency for the results that the new solution is providing.
Is OR unique in that way? Is finding that right translation/transparency easier in other data science domains?
Signy: No, I don’t think it’s noticeably easier in other domains. What does help is familiarity and comfort with the models, and having everyone around the problem be well versed in what we’re trying to achieve — from the modeling to the software components to a deep understanding of the business dynamics. That can make a big difference.
Your team uses Nextmv in your data science tech stack. How do you introduce people to what Nextmv is?
Signy: In my many years of experience, what I've come to realize more and more is that we've underestimated the challenges with getting stakeholders to really be comfortable with the optimization solutions we could provide. That can be a big limiting factor.
Nextmv solves the problem of getting results into the hands of stakeholders, such as operators and other business users, faster. With Nextmv, we have the ability to get an optimization model stood up easily and then put our work into a place where a stakeholder can interact with it and help build trust in the solutions that we're developing.
Before we had Nextmv, we had a person from the data science team helping run a decision model for any given stakeholder exploration. Now, stakeholders are able to play with the available model options, see the output, and analyze it themselves. This gives them the comfort that what the decision model is giving them is a good solution and is comparable, if not better, than the alternative or manual path they were taking before.
Can you speak more about the prior manual path?
Signy: Yes. For example, a planner might download a spreadsheet from a data warehouse, massage it in Tableau, and paste it into three different places. Instead, my team is looking to evolve that decision making process by reducing extra steps and having an interface where planners can get really comfortable with more efficient decision models.
But it's just taking these little steps to say, OK you had a manual spreadsheet-to-Tableau-based process, now we have a decision workflow: data flows from the data warehouse into Nextmv, you can click some buttons to generate different scenarios, and you can explore the output. And it’s all driven by that user.
You’ve spoken to the OR-business operations interface a bit. Tell me about the OR-software interface. What does that look like?
Signy: Nextmv is giving us a more robust approach for models as microservices. The engineering or service teams need to have something that they can modify easily and not have to worry about a process breaking. They want to be able to manage the service, understand it, monitor it, debug it, fix it easily, not have to coordinate with 10 different teams and have 50 meetings to resolve an issue.
So how do we manage that? Robust testing with Nextmv and creating better contracts. What are we testing for? What are they testing for? And making sure we’re not breaking things for each other.
How does your team think about using multiple types of optimization solvers and modeling frameworks as you develop decision solutions?
Signy: We use commercial and open source solutions. We typically use open source options where speed isn’t as much of a factor, often in prototyping. And I want more of that, more iteration on early solutions that we’re prepared to evolve at some point.
Generally, I believe that if you’re managing more than three rules, you should create a decision model. So in the prototyping space, that translates to creating a Nextmv app and planning and iterating in that context.
For example, in a pricing use case, you may have these rules:
- Given this subset of orders, price it this way
- Except in this region, where you do something slightly different
- And then for this category of subscriptions, follow this other process
It’s really any place where you’re putting a lot of if-then statements and having to manage all of those interactions. So if you’re doing more than three of those, you really want to formulate it as a decision problem.
What does your future state look like?
Signy: Every decision model has its own Nextmv app. Any time you have a question like, "What if I change this input?” or “How would that impact our operations?", you can explore that easily as a stakeholder, which is usually our operations team members. They can go into the platform and say, “I want to play with the range on this input” and make those runs themselves.
Then, as a data science and OR team, we can focus our conversations on if we need to evolve the overall goals of the model because the business strategy is changing, or split the model into two different parts. This is the more complicated part of the modeling process. With a platform like Nextmv, our modelers are not tied up supporting any of the idea generation that the operations team has because the ops team has the latitude to run and explore on their own. Instead, our data science and OR team focus can be on evaluating building models and understanding how they’re connected to the business strategy.
To hear more from Signy, check out her discussion on agility in operations research and decision science with practitioners from IKEA and Aimpoint Digital.
