‍
In many cases, you have a general model but different segments of interest that you want to measure (VIP users / different marketing channels / etc.), or you are looking for underperforming sub-groups.Â
In all of these cases, you want to remember that low performance scores for sub-groups don’t necessarily mean that these groups are bad or they reduce the overall performance of the model! Some metrics are not simply additive or not additive at all, like AUC.
In this sense, group performance may look bad when considered “by itself”, but it contributes to the overall performance, when you look at the whole population.
Let’s say your model estimates the Lifetime Value (LTV) of your users, and you have two groups of interests: group_1 and group_2, and your metric is “Mean Error”.
Each of these groups seem to perform badly, but these two groups combined are having a perfect score of zero mean error!
Sign up for our on-demand webinar
Sign up for our on-demand webinar
Sign up for our on-demand webinar
Why marketing use cases require a robust AI Assurance strategy
Why marketing use cases require a robust AI Assurance strategy
Why marketing use cases require a robust AI Assurance strategy
superwise.ai was recognized in the Gartner September 2020 Cool Vendors in Enterprise AI Governance
superwise.ai was recognized in the Gartner September 2020 Cool Vendors in Enterprise AI Governance
superwise.ai was recognized in the Gartner September 2020 Cool Vendors in Enterprise AI Governance
How fraud detection solution vendors can leverage their ML monitoring solution to boost the efficiency of their fraud and data science teams
How fraud detection solution vendors can leverage their ML monitoring solution to boost the efficiency of their fraud and data science teams
How fraud detection solution vendors can leverage their ML monitoring solution to boost the efficiency of their fraud and data science teams
Quick list of questions you want to answer as you consider how to monitor your models in production
Quick list of questions you want to answer as you consider how to monitor your models in production
Quick list of questions you want to answer as you consider how to monitor your models in production
The Data Exchange Podcast: Ofer Razon on building machine learning tools to scale AI operations.
The Data Exchange Podcast: Ofer Razon on building machine learning tools to scale AI operations.
The Data Exchange Podcast: Ofer Razon on building machine learning tools to scale AI operations.
What happens when the #1 go-to Work OS, serving more than 100,000 teams around the world starts to understand that their models are misbehaving?
What happens when the #1 go-to Work OS, serving more than 100,000 teams around the world starts to understand that their models are misbehaving?
What happens when the #1 go-to Work OS, serving more than 100,000 teams around the world starts to understand that their models are misbehaving?
Rip the full benefits of your AI program. Risk-Free
Rip the full benefits of your AI program. Risk-Free
Rip the full benefits of your AI program. Risk-Free
Outsmart Fraudsters
Outsmart Fraudsters
Outsmart Fraudsters
Learn how to transform how credit and risks are allocated
Learn how to transform how credit and risks are allocated
Learn how to transform how credit and risks are allocated
So what's the deal? How can we scale AI efforts while fostering trust and without losing sight?
So what's the deal? How can we scale AI efforts while fostering trust and without losing sight?
So what's the deal? How can we scale AI efforts while fostering trust and without losing sight?
