A common measure for “regression” or “probability estimation” tasks in ML is the percentage error of the predictions. Either as Mean Percentage Error (MPE)/Mean Abs Percentage Error (MAPE)/ sMAPE, and more..
However, such relative measures as “percentage error” can cause acute % differences in very small residuals, which are usually less significant to measure or to draw conclusions for the performance.Â
Let’s take the following example:
‍
The relative error is highly impacted from case #4, where the percentage error is really high, but the actual absolute error is very low - meaning that it should rather be overlooked to measure performance.Â
There are many methods known to handle such cases: from bounding the maximal percentage error, to ignoring errors under a certain threshold, or to using Median percentage error instead of Mean.
We recommend using a normalization factor (see suggestion below) where we can express in the “a” parameter the sensitivity level to be normalized and scale down big relative errors in small numbers.
We hope you find our tip useful!
Make sure you register here to receive our Weekly Tips straight to your inbox!
And please share with us on social media what you would want to hear about!
‍
‍
Sign up for our on-demand webinar
Sign up for our on-demand webinar
Sign up for our on-demand webinar
Why marketing use cases require a robust AI Assurance strategy
Why marketing use cases require a robust AI Assurance strategy
Why marketing use cases require a robust AI Assurance strategy
superwise.ai was recognized in the Gartner September 2020 Cool Vendors in Enterprise AI Governance
superwise.ai was recognized in the Gartner September 2020 Cool Vendors in Enterprise AI Governance
superwise.ai was recognized in the Gartner September 2020 Cool Vendors in Enterprise AI Governance
How fraud detection solution vendors can leverage their ML monitoring solution to boost the efficiency of their fraud and data science teams
How fraud detection solution vendors can leverage their ML monitoring solution to boost the efficiency of their fraud and data science teams
How fraud detection solution vendors can leverage their ML monitoring solution to boost the efficiency of their fraud and data science teams
Quick list of questions you want to answer as you consider how to monitor your models in production
Quick list of questions you want to answer as you consider how to monitor your models in production
Quick list of questions you want to answer as you consider how to monitor your models in production
The Data Exchange Podcast: Ofer Razon on building machine learning tools to scale AI operations.
The Data Exchange Podcast: Ofer Razon on building machine learning tools to scale AI operations.
The Data Exchange Podcast: Ofer Razon on building machine learning tools to scale AI operations.
What happens when the #1 go-to Work OS, serving more than 100,000 teams around the world starts to understand that their models are misbehaving?
What happens when the #1 go-to Work OS, serving more than 100,000 teams around the world starts to understand that their models are misbehaving?
What happens when the #1 go-to Work OS, serving more than 100,000 teams around the world starts to understand that their models are misbehaving?
Rip the full benefits of your AI program. Risk-Free
Rip the full benefits of your AI program. Risk-Free
Rip the full benefits of your AI program. Risk-Free
Outsmart Fraudsters
Outsmart Fraudsters
Outsmart Fraudsters
Learn how to transform how credit and risks are allocated
Learn how to transform how credit and risks are allocated
Learn how to transform how credit and risks are allocated
So what's the deal? How can we scale AI efforts while fostering trust and without losing sight?
So what's the deal? How can we scale AI efforts while fostering trust and without losing sight?
So what's the deal? How can we scale AI efforts while fostering trust and without losing sight?
