{"id":1293,"date":"2011-12-15T17:33:32","date_gmt":"2011-12-15T22:33:32","guid":{"rendered":"https:\/\/demand-planning.com\/?p=1293"},"modified":"2011-12-15T17:33:32","modified_gmt":"2011-12-15T22:33:32","slug":"2012-forecasting-performance-objectives","status":"publish","type":"post","link":"https:\/\/demand-planning.com\/2011\/12\/15\/2012-forecasting-performance-objectives\/","title":{"rendered":"2012 Forecasting Performance Objectives"},"content":{"rendered":"<div id=\"attachment_1033\" style=\"width: 160px\" class=\"wp-caption alignright\"><a href=\"https:\/\/demand-planning.com\/wp-content\/uploads\/2010\/12\/The-BFD-square.jpg\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-1033\" class=\"size-thumbnail wp-image-1033\" title=\"Mike Gilliland: The BFD\" src=\"https:\/\/demand-planning.com\/wp-content\/uploads\/2010\/12\/The-BFD-square-150x150.jpg\" alt=\"Mike Gilliland: The BFD\" width=\"150\" height=\"150\" \/><\/a><p id=\"caption-attachment-1033\" class=\"wp-caption-text\">Mike Gilliland AKA: The BFD<\/p><\/div>\n<p><a href=\"http:\/\/ibf.org\/\">Forecasting performance objectives<\/a> are usually set in one of three ways:<\/p>\n<ul>\n<li>Relative to \u201cbest-in-class\u201d industry benchmarks.<\/li>\n<li>Improvement over prior year performance.<\/li>\n<li>Arbitrarily \u2013 based on what management wants or needs to happen.<\/li>\n<\/ul>\n<p>All three are wrong.<\/p>\n<ul>\n<li>There are many perils in relying on industry benchmarks to set your own organization\u2019s performance objectives, the most important of which is relevance. The organization with <a href=\"http:\/\/ibf.org\/conferences.cfm?fuseaction=conferenceDetail&amp;conID=341\">best-in-class forecast accuracy<\/a> probably achieves this because they have the easiest to forecast demand.<\/li>\n<\/ul>\n<p>If your organization does not enjoy similarly favorable \u201cforecastability\u201d of its demand patterns, then there is little hope of achieving best-in-class performance.\u00a0 Setting unreachable goals just demoralizes the forecasting staff, and encourages them to cheat.<\/p>\n<ul>\n<li>In general, improvement over prior performance is an appropriate objective. However, we must be wary of the context in which that improvement is measured.<\/li>\n<\/ul>\n<p>If there are no substantive changes in the forecastability of demand patterns from year to year, then improvement in forecast accuracy (or at least, not doing worse!) is a reasonable objective.\u00a0 However, what if forecastability changes?\u00a0 This occurs when demand patterns change, either organically (without our intervention), or due to our own sales, marketing, and financial practices.<\/p>\n<p>For example, switching a product from everyday low pricing (EDLP \u2013 where prices remain constant) to hi-lo pricing (where temporary price cuts create spikes in demand), would greatly increase demand volatility, and reduce forecastability.\u00a0 If a product were under EDLP in 2011 and hi-lo pricing in 2012, we would actually expect reduced forecast accuracy in 2012.\u00a0 Insisting on improved accuracy after such a change would be unreasonable.<\/p>\n<ul>\n<li>Forecasting performance objectives must be based on what is reasonable to expect, given the nature of demand patterns.\u00a0 Simply pulling a number out of the air, such as \u201cMAPE for all products must be &lt;20%\u201d is inappropriate and irresponsible.<\/li>\n<\/ul>\n<p>What if demand patterns are highly volatile, and 20% MAPE is not achievable \u2013 then you give the forecasters every reason to give up or find a way (by gaming the metrics) to achieve the goal.\u00a0 Or, perhaps demand patterns are very easy to forecast and the goal can be reached by just using a na\u00efve model? How hard are your forecasters going to work to improve accuracy if they can beat the goal by doing nothing?<\/p>\n<p><strong>My Gift: Your 2012 Forecasting Performance Objective<\/strong><\/p>\n<p>As pathetic as this may sound, perhaps the only reasonable objective for 2012 forecasting performance is to beat a na\u00efve model (or at least do no worse), and <a href=\"http:\/\/ibf.org\/index.cfm?fuseaction=showObjects&amp;objectTypeID=50\">continuously improve the process<\/a>.<\/p>\n<p>Improvement can be demonstrated by reducing the error and bias in the forecast, increasing the Forecast Value Added, and becoming more efficient at executing the forecasting process (spending fewer resources).<\/p>\n<p>If you can achieve 20% MAPE by using automated statistical forecasting software \u2013 or by using an elaborate collaborative and consensus process occupying all of your sales reps, planners, customers, suppliers, and executive staff for several hours every month \u2013 which is the better way to go?<\/p>\n<p><strong>Why Management May Hate This Gift<\/strong><\/p>\n<p>So for your 2012 performance objective, what MAPE (or whatever other particular metric you use) must you achieve?\u00a0 Sorry, I cannot tell you.\u00a0 (That is the part that management hates.) Your goal is to do no worse than a na\u00efve model in 2012, and we won\u2019t know how well a na\u00efve model does until the end of 2012.<\/p>\n<p>You must first choose an appropriate na\u00efve model (e.g., random walk, seasonal random walk, etc.).\u00a0 Then you must track your organization\u2019s forecasting performance each period and compare that to what your na\u00efve model achieved.\u00a0 Whichever does better in any particular week or month doesn\u2019t matter \u2013 short term results can be due to chance.\u00a0 But by the end of the year, you should be able to draw one of three conclusions:<\/p>\n<ol>\n<li>We forecast worse than a na\u00efve model.<\/li>\n<li>We forecast about the same as a na\u00efve model.<\/li>\n<li>We forecast better than a na\u00efve model.<\/li>\n<\/ol>\n<p>Conclusion 2 means that your process is in a statistical dead heat with the na\u00efve model \u2013 you cannot reject the null hypothesis that there is <em>no difference<\/em> between the two approaches.\u00a0 If you are committing a lot of resources to forecasting, you may want to redirect those resources to more productive activities.<\/p>\n<p>If you achieve 3, congratulations, your forecasting process is doing its job.\u00a0 Take pride \u2013 as this can be surprisingly more difficult than it appears it should be.<\/p>\n<p>If you achieve 1, then welcome to the unfortunate reality of business forecasting \u2013 where organizations are often best at making a tough problem worse.<\/p>\n<p>Happy holidays\u2026<\/p>\n<p><strong>Bio:<\/strong> Michael Gilliland is Product Marketing Manager at SAS, and author of <a href=\"https:\/\/support.sas.com\/pubscat\/bookdetails.jsp?catid=1&amp;pc=62908\"><em>The Business Forecasting Deal<\/em><\/a>.\u00a0 Mike is a frequent contributor to the <em><a href=\"http:\/\/ibf.org\/index.cfm?fuseaction=showObjects&amp;objectTypeID=21\">Journal of Business Forecasting<\/a><\/em>, and (along with Ryan Rickard of Newell Rubbermaid) will be delivering a half-day workshop \u201cWhat Management Must Know About Forecasting\u201d at the <a href=\"http:\/\/ibf.org\/conferences.cfm?fuseaction=conferenceDetail&amp;conID=341\">IBF Supply Chain Forecasting Conference in Scottsdale, AZ,<\/a> February 26-28, 2012.\u00a0 Conference attendees will receive a free signed copy of his book.\u00a0 You can follow Mike\u2019s blog, <em>The Business Forecasting Deal<\/em>, at <a href=\"file:\/\/\/C:\/Users\/Anish\/AppData\/Local\/Microsoft\/Windows\/Temporary%20Internet%20Files\/Content.Outlook\/7KAI0KKR\/blogs.sas.com\/content\/forecasting\">blogs.sas.com\/content\/forecasting<\/a>.\u00a0 Furthermore, Mike will be contributing on a monthly basis for IBF&#8217;s demand-planning.com blog.\u00a0 Please submit your topic ideas to the IBF here: info(at)ibf.org<\/p>\n<p style=\"text-align: center;\"><span style=\"color: #ff0000;\"><strong>Hear Mike Speak at IBF&#8217;s:\u00a0<\/strong><\/span><\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/ibf.org\/conferences.cfm?fuseaction=conferenceDetail&amp;conID=341\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-1294\" title=\"scottsdale banner\" src=\"https:\/\/demand-planning.com\/wp-content\/uploads\/2011\/12\/scottsdale-banner.jpg\" alt=\"IBF's Supply Chain Forecasting &amp; Planning Conference\" width=\"448\" height=\"209\" \/><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Forecasting performance objectives are usually set in one of three ways: Relative to \u201cbest-in-class\u201d industry benchmarks. Improvement over prior year performance. Arbitrarily \u2013 based on what management wants or needs to happen. All three are wrong. There are many perils in relying on industry benchmarks to set your own organization\u2019s performance objectives, the most important [&hellip;]<\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[33],"tags":[60,61,62,64,65,66,34,67,39,68,69,70,35,42,43,44,36,48,37,73,74,38],"class_list":{"0":"post-1293","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-forecasting-and-planning","7":"tag-best-practices","8":"tag-business-forecasting","9":"tag-collaborative-forecasting","10":"tag-demand-forecast","11":"tag-demand-forecasting","12":"tag-demand-management","13":"tag-demand-planning","14":"tag-demand-planning-and-forecasting-conference","15":"tag-economic-forecasting","16":"tag-executive-sop","17":"tag-forecast-accuracy","18":"tag-forecast-error","19":"tag-forecasting","20":"tag-forecasting-metrics","21":"tag-forecasting-models","22":"tag-forecasting-system","23":"tag-ibf","24":"tag-institute-of-business-forecasting-and-planning","25":"tag-sop","26":"tag-sales-operations-planning","27":"tag-sales-forecasting","28":"tag-supply-chain"},"_links":{"self":[{"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/posts\/1293"}],"collection":[{"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/comments?post=1293"}],"version-history":[{"count":0,"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/posts\/1293\/revisions"}],"wp:attachment":[{"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/media?parent=1293"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/categories?post=1293"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/tags?post=1293"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}