A common measure for “regression” or “probability estimation” tasks in ML is the percentage error of the predictions. Either as Mean Percentage Error (MPE)/Mean Abs Percentage Error (MAPE)/ sMAPE, and more..
However, such relative measures as “percentage error” can cause acute % differences in very small residuals, which are usually less significant to measure or to draw conclusions for the performance.Â
Let’s take the following example:
‍
The relative error is highly impacted from case #4, where the percentage error is really high, but the actual absolute error is very low - meaning that it should rather be overlooked to measure performance.Â
There are many methods known to handle such cases: from bounding the maximal percentage error, to ignoring errors under a certain threshold, or to using Median percentage error instead of Mean.
We recommend using a normalization factor (see suggestion below) where we can express in the “a” parameter the sensitivity level to be normalized and scale down big relative errors in small numbers.
We hope you find our tip useful!
Make sure you register here to receive our Weekly Tips straight to your inbox!
And please share with us on social media what you would want to hear about!
‍
‍
When to retrain? What data should be used? What should be retrained? Each of the three questions above can be answered separately, and can help determine the optimal strategy for each case.
When to retrain? What data should be used? What should be retrained? Each of the three questions above can be answered separately, and can help determine the optimal strategy for each case.
When to retrain? What data should be used? What should be retrained? Each of the three questions above can be answered separately, and can help determine the optimal strategy for each case.
How data science teams can make sure they are one step ahead of a negative review
How data science teams can make sure they are one step ahead of a negative review
How data science teams can make sure they are one step ahead of a negative review
Not all sub-groups behave equally in production
Not all sub-groups behave equally in production
Not all sub-groups behave equally in production
25 members to create the canonical stack for Artificial Intelligence projects
25 members to create the canonical stack for Artificial Intelligence projects
25 members to create the canonical stack for Artificial Intelligence projects
Leveraging over billions of data points to influence the decisions that shape and constantly improve cutting-edge games, applying machine learning to the gaming space is a complex play.
Leveraging over billions of data points to influence the decisions that shape and constantly improve cutting-edge games, applying machine learning to the gaming space is a complex play.
Leveraging over billions of data points to influence the decisions that shape and constantly improve cutting-edge games, applying machine learning to the gaming space is a complex play.
A framework for anyone who has an interest in building, testing, and implementing a robust monitoring strategy in their organization or elsewhere.
A framework for anyone who has an interest in building, testing, and implementing a robust monitoring strategy in their organization or elsewhere.
A framework for anyone who has an interest in building, testing, and implementing a robust monitoring strategy in their organization or elsewhere.
A framework to scale AI with a production-first approach
A framework to scale AI with a production-first approach
A framework to scale AI with a production-first approach
Foster the independence of the business teams when it comes to using your models output
Foster the independence of the business teams when it comes to using your models output
Foster the independence of the business teams when it comes to using your models output
Best CD practices for the painless deployment of ML models and versions
Best CD practices for the painless deployment of ML models and versions
Best CD practices for the painless deployment of ML models and versions
What happens when the #1 productivity solution needs to scale its use of AI? Check out the highlights of the webinar led by monday.com to learn the best practices of their marketing and data science teams!
What happens when the #1 productivity solution needs to scale its use of AI? Check out the highlights of the webinar led by monday.com to learn the best practices of their marketing and data science teams!
What happens when the #1 productivity solution needs to scale its use of AI? Check out the highlights of the webinar led by monday.com to learn the best practices of their marketing and data science teams!
Best CI practices for the painless deployment of ML models and versions
Best CI practices for the painless deployment of ML models and versions
Best CI practices for the painless deployment of ML models and versions
Sign up for our on-demand webinar
Sign up for our on-demand webinar
Sign up for our on-demand webinar
Why marketing use cases require a robust AI Assurance strategy
Why marketing use cases require a robust AI Assurance strategy
Why marketing use cases require a robust AI Assurance strategy
superwise.ai was recognized in the Gartner September 2020 Cool Vendors in Enterprise AI Governance
superwise.ai was recognized in the Gartner September 2020 Cool Vendors in Enterprise AI Governance
superwise.ai was recognized in the Gartner September 2020 Cool Vendors in Enterprise AI Governance
How fraud detection solution vendors can leverage their ML monitoring solution to boost the efficiency of their fraud and data science teams
How fraud detection solution vendors can leverage their ML monitoring solution to boost the efficiency of their fraud and data science teams
How fraud detection solution vendors can leverage their ML monitoring solution to boost the efficiency of their fraud and data science teams
Quick list of questions you want to answer as you consider how to monitor your models in production
Quick list of questions you want to answer as you consider how to monitor your models in production
Quick list of questions you want to answer as you consider how to monitor your models in production
The Data Exchange Podcast: Ofer Razon on building machine learning tools to scale AI operations.
The Data Exchange Podcast: Ofer Razon on building machine learning tools to scale AI operations.
The Data Exchange Podcast: Ofer Razon on building machine learning tools to scale AI operations.
How marketing and data science teams are using superwise.ai to assure the health of their models
How marketing and data science teams are using superwise.ai to assure the health of their models
How marketing and data science teams are using superwise.ai to assure the health of their models
Reap the full benefits of your AI program. Risk-Free
Reap the full benefits of your AI program. Risk-Free
Reap the full benefits of your AI program. Risk-Free
Outsmart Fraudsters
Outsmart Fraudsters
Outsmart Fraudsters
Learn how to transform how credit and risks are allocated
Learn how to transform how credit and risks are allocated
Learn how to transform how credit and risks are allocated
So what's the deal? How can we scale AI efforts while fostering trust and without losing sight?
So what's the deal? How can we scale AI efforts while fostering trust and without losing sight?
So what's the deal? How can we scale AI efforts while fostering trust and without losing sight?
