{"id":7540,"date":"2019-01-21T09:12:22","date_gmt":"2019-01-21T14:12:22","guid":{"rendered":"https:\/\/demand-planning.com\/?p=7540"},"modified":"2019-01-21T09:12:22","modified_gmt":"2019-01-21T14:12:22","slug":"beware-the-correlation-causation-forecasting-trap","status":"publish","type":"post","link":"https:\/\/demand-planning.com\/2019\/01\/21\/beware-the-correlation-causation-forecasting-trap\/","title":{"rendered":"Beware The Correlation\/Causation Forecasting Trap"},"content":{"rendered":"

\u201cShallow men believe in luck or in circumstance. Strong men believe in cause and effect.\u201d<\/strong>\u00a0– Ralph Waldo Emerson<\/strong><\/p>\n

It would be easy to see the demise of Sears Roebuck retail stores as a triumph for e-commerce. In actuality, if you were to look at newspaper advertising compared to annual revenue of Sears, you will notice an obvious pattern emerge. When we look at the steady increase of newspaper revenue since the 1950\u2019s, it tracks both the growth of Sears and its decline. The fortunes of the newspaper industry and Sears peaked around the turn of this century with a small resurgence in 2006, only for the former to decline, and the latter forced into bankruptcy.<\/p>\n

Looking at this data, it is clear that what Sears must do to rejuvenate itself is pour all of its remaining marketing dollars into newspaper print advertising to boost newspaper revenue to save Sear\u2019s from pending liquidation.<\/p>\n

Obviously.<\/p>\n

If you did not pick up on my hint of sarcasm and how this is an example of the difference between causation and correlation, don\u2019t feel too bad. Newspaper revenue and Sears’ revenue are only an obscure correlation much like linking ice cream sales to the murder rate in New York City.<\/p>\n

My point is that humans are evolutionarily predisposed to see patterns and psychologically inclined to gather and fill in the blanks of information to make connections.<\/p>\n

The Human Brain Looks For Correlation, Even If There Is None<\/h2>\n

The question of cause versus correlation, has haunted science and philosophy from their earliest days, and still dogs our heels for numerous reasons. You would think with the vast number of articles warning of its perils, this would not be a problem anymore, but we still get fooled.<\/p>\n

We confuse coincidence with correlation and correlation with causality<\/p><\/blockquote>\n

We confuse coincidence with correlation, and correlation with causality. Just think of the last time something great happened when you had those special socks on \u2013 admit it, they are now your \u201clucky\u201d socks even though you know deep down it had to be coincidence.<\/p>\n

Unfortunately, we run into problems with this in business planning and forecasting as well.\u00a0 Of course, if we stock out or have too much inventory the \u201ccause\u201d is the forecast. With poor planning we may see a correlation between forecast error and missed sales, or between long supplier lead times and excess inventory – in both scenarios, the cause is most likely not<\/em> the forecast.<\/p>\n

Forecasts do not cause excess inventory, uncertainty does (supply and demand). In addition, forecast error is not always the result, or caused by good (or bad) forecasting but is indicative of the uncertainty of the demand you are actually measuring.<\/p>\n

Forecast accuracy is ultimately limited by the nature of the behavior we are trying to forecast<\/p><\/blockquote>\n

Forecasts Do Not Reduce Demand Variability<\/h2>\n

Forecast accuracy is ultimately limited by the nature of the behavior we are trying to forecast. Accuracy is the degree of closeness of the statement of quantity to that quantity\u2019s actual (true) value. Whilst I accept that one\u2019s ability to create an accurate forecast is correlated to demand variability, we also need to remember\u00a0 that an accurate forecast does not reduce demand variability. Demand variability is an expression of how much the demand changes over time, and, to some extent, the predictability of the demand.<\/p>\n

Analysis suggests that whatever we can do to reduce volatility in demand for our products, the better we should be at predicting the forecast. Unfortunately, most organizational policies and practices are designed to add volatility to demand rather than make it more stable. We continue to contribute to SKU proliferation<\/a>, shorter life cycles and more complex channels with dynamic marketing – and still our business partners still attribute missing the forecast as demand planners’ fault.<\/p>\n

Historical Patterns Always Repeat, Right?!<\/h2>\n

There are other ways we can become victim to the deception of causation.\u00a0 Inexperienced forecasters, and those outside of forecasting, may assume that history will tell us exactly what will happen in the future. If we sold 2,000 units last year, then we will sell 2,000 this year as well. If we are using sophisticated software and the fitted model is showing 10% error, then we should expect no more than 10% variation. If there is an obvious historic pattern, then it will obviously repeat itself with the same predictability.<\/p>\n

And we are surprised when this doesn’t happen. This mindest incorporates bad assumptions because we fall into the trap of correlation verse causation.<\/p>\n

Know The Limitations Of Time Series Forecasting<\/h2>\n

Time series analysis looks for correlations in successive observations, not causation. One of the 5 laws of demand planning<\/a> are that you can\u2019t expect to have the same results from forecasting if there are some changes in environmental conditions. Seems obvious but we too slip into unwittingly thinking seasonality<\/a> will cause a lift, not remembering the seasonal index<\/a> is only a correlation and the underlying cause could be dynamic weather conditions or holidays that have whole other dependent variables.<\/p>\n

We wear our historic data like a pair of lucky socks<\/p><\/blockquote>\n

We wear our historic data like a pair of lucky socks without ever understanding the cause and stating it as absolute. We become comfortable and do not do enough to look more at the cause than just the correlation<\/a>.<\/p>\n

In today\u2019s business environment, changes in the marketplace are swift, sudden, and may not follow the historical correlation. Just looking at historic shipments alone may not give you what you need or tell the whole picture.<\/p>\n

No matter how many articles are written on causation versus correlation, people are wired to focus more on descriptive analytics and making real or imaginary connections to what happened. But we come from a different background \u2013 one that must use the data to infer what is going to happen next. To accomplish what we need, many times, we may not fully understand the \u201cwhy\u201d but use algorithms to help reveal \u201cwhat\u201d the future holds. We just need to remember that one may not always cause the other, but just rhyme and move in similar direction and manner.<\/p>\n

That said, I too will most likely have my lucky orange stripped socks for the next\u00a0 special executive S&OP<\/a> meeting or if I am speaking at a conference.<\/p>\n

 <\/p>\n<\/span>","protected":false},"excerpt":{"rendered":"

\u201cShallow men believe in luck or in circumstance. Strong men believe in cause and effect.\u201d\u00a0– Ralph Waldo Emerson It would be easy to see the demise of Sears Roebuck retail stores as a triumph for e-commerce. In actuality, if you were to look at newspaper advertising compared to annual revenue of Sears, you will notice […]<\/p>\n","protected":false},"author":3470,"featured_media":7546,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[390,386,339],"tags":[469],"class_list":{"0":"post-7540","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-advanced-analytics","8":"category-analytics","9":"category-predictive-analytics-predictive-analytics","10":"tag-correlation-vs-causation"},"_links":{"self":[{"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/posts\/7540"}],"collection":[{"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/users\/3470"}],"replies":[{"embeddable":true,"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/comments?post=7540"}],"version-history":[{"count":0,"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/posts\/7540\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/media\/7546"}],"wp:attachment":[{"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/media?parent=7540"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/categories?post=7540"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/demand-planning.com\/wp-json\/wp\/v2\/tags?post=7540"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}