Forecasting is a fundamental business task, yet few companies perform it efficiently and well – or at least not as efficiently and well as they would like! If our attention is misdirected to the current fads, hype, and peripheral issues, we can lose focus on the fundamentals that truly impact effective forecasting. This article provides a few basic guidelines for business forecasting in the form of aphorisms – concise statements of principle. These seven statements as well as their corollaries and lessons are meant to raise awareness of underlying issues and direct effort into key areas for improvement.
APHORISM 1
Forecasting is a huge waste of management time. This is not to imply that forecasting is unnecessary and should be eliminated. Clearly, good forecasts can help a company become more effective in utilizing its resources and satisfying its customers. Rather, this aphorism is meant to highlight the inordinate amount of company resources invested in the typical forecasting process, and to question whether all this effort is really making a worthwhile difference.
Unless your company runs in a totally automated forecasting mode, management resources are involved in the forecasting process. At minimum, this includes the forecast analysts and demand planners who manage the statistical forecasting models and provide manual overrides. In the usual situation, however, forecasting draws time and attention from areas such as sales, marketing, finance, operations, and executive management. This is high-cost management time. Are these forecasting participants skilled and can they improve the forecast? Or are they contaminating what should be an unbiased and scientific process with their politics, wishes, and personal agendas?
MAPE and the other standard performance metrics cannot detect waste in the forecasting process. Instead, Forecast Value Added (FVA) analysis is used to identify the change in performance metrics caused by particular steps or participants in the forecasting process. Efforts that reduce MAPE are adding value by making the forecast better. Efforts that fail to reduce MAPE are simply wasteful and need to be removed from the process.
Lesson: Measure the performance of each step and of each participant in your forecasting process. Identify and mercilessly eliminate non-value adding activities. The result will be better forecasts with less effort.
APHORISM 2
Accuracy is determined more by the nature of the demand pattern being forecast than by the specific method being used to forecast it. Under favorable conditions, demand can be forecast accurately with simple techniques. At other times, we can never quite reach the level of accuracy we want, no matter how much data, statistical analysis, and human intervention we employ. This is not our fault – it is simply the reality of dealing with randomness and variation in demand.
The Coefficient of Variation (CV) is the metric for expressing demand variation (or “volatility”). CV, which is expressed as a percentage, is the ratio of a pattern’s standard deviation to its mean. For example, if demand for item XYZ averages 100 units and the standard deviation of the demand is 30 units, then the “demand volatility” is CV = 30%. CV exceeding 100% is not unheard of at granular levels of detail, such as the retailer’s Store / SKU. A 50% CV would be common for a consumer products manufacturer at the Warehouse / SKU level.
Highly seasonal, greatly promoted, and short lifecycle “fashion” items will have higher volatility than long running core or “basic” items – making it no coincidence that fashion items are more difficult to forecast! Note that both volatility and forecast error will decrease as you aggregate granular level demand to higher levels in the organizational hierarchy. This is because the ups and downs of granular level demand (as well as granular level forecast errors) will cancel each other out in the total, resulting in “smoother” demand patterns at higher levels as well as easier-to-forecast demand.
A visual representation of the general relationship between demand volatility and forecast accuracy is easy to create. Simply calculate the CV and MAPE for each of your items and graph the points on a scatter plot. You will see that the higher the volatility, the greater the forecast error.
Lesson: Be careful when interpreting forecasting performance benchmarks. The accuracy achieved by those companies with the best results may be due more to the forecastability of their demand patterns than to the excellence of their forecasting process. It is a worthwhile exercise to benchmark your own forecasting performance against what you would have achieved with a naïve forecast.
Do not set arbitrary forecasting performance goals. Goals for forecasting performance must be based on the demand forecastability. Arbitrary performance goals may be set too low, thereby rewarding inferior performance. Goals that are set unrealistically high will demoralize the forecasting staff and encourage cheating. Consider these two scenarios:
- Your job each day is to forecast the results of a fair coin toss (heads or tails). Over several years, you have consistently achieved 50% accuracy in your forecasts. Management is dissatisfied with your results and gives you a new goal of 60% accuracy or you will be fired. What would you do next?
- You are responsible for forecasting the sales of a popular product that has a volatile demand. Your bonus is based purely on forecast accuracy, but management has set unrealistically high accuracy goals. How would you maximize your bonus?
In the first scenario, you can either resign or you can wait around long enough to be fired and receive a severance package! By the nature of the pattern you are forecasting (tossing a fair coin) you are doomed to failure – you will never consistently achieve 60% accuracy.
In the second scenario, you may be able to cheat your way to highly accurate forecasts by constraining supply. Since this is a popular product with high demand, a constrained supply will assure that almost every available unit is sold. By making sales forecast equal to the expected supply in each time period, your error is only as large as the error in your supply projection. Whether constraining supply is in the best interest of the company is a separate matter – but at least your bonus is maximized.
Lesson: Given the nature of your demand patterns, seek to understand what level of forecast accuracy is “reasonable to expect.” The accuracy achieved by a “naïve” forecasting method (such as random walk or moving average) provides a lower bound for the accuracy that you should be able to achieve. A reasonable goal for the forecasting function – as pathetic as this may sound – is to beat the accuracy of a naïve forecast. It is also reasonable to strive continuously for process improvement. Improvement can be in terms of increased accuracy, reduced bias, and elimination of process waste.
APHORISM 3
Organizational policies and politics can have a significant impact on forecasting effectiveness. The forecast should be an unbiased “best guess” of what is going to happen in the future. It should be based on a rational, objective, and dispassionate evaluation of the historical facts (what has been sold and under what conditions) and future expectations (about pricing, promotional activities, the competitive environment, supply considerations, and the like). Unfortunately, real-life business forecasting often is contaminated by the wishes, wants, and personal agendas of the forecasting process participants.
Although the managers, executives, and sales force of your firm may be stellar citizens of the highest moral fiber, do not assume they are trustworthy when it comes to forecasting. The forecast is an easy tool for driving personal objectives that may not align with long-term company objectives. Will the sales force forecast be low during the annual planning cycle? After all, lower expectations mean lower sales quotas. Will their forecasts be high during the rest of the year to assure plenty of supply is available to fill orders? Will a chief executive make a low forecast to beat Wall Street expectations for next quarter? Or will the executive’s forecast be high to create a buzz and boost the short-term stock price? You must consider the ultimate motive of anyone providing or approving forecasts. The performance of all forecasting process participants should be tracked to provide objective and data-based evidence of their impact on the process.
Lesson: Use FVA analysis to monitor contributing participants. Those who fail to improve the forecast should be coached how they can do so. Otherwise, they should be removed from the process. If a participant’s efforts fail to improve the forecast, then those forecasting efforts are a waste. Such a participant should be directed to activities that are more productive and helpful to the company.
APHORISM 4
You may not control the accuracy achieved, but you can control the process used and the resources you invest. You can’t just buy a better forecast. There is no guarantee – no matter how much you invest in people, technology, and a process – that you will achieve the accuracy targets your company needs. While it is disappointing to realize that you can’t make the forecasting problem go away by writing a big check to consultants and software vendors, focus on what you can do, such as:
- Determine what level of accuracy is reasonable to expect for your demand patterns.
- Direct all efforts toward achieving that level of accuracy with the least cost in time and company resources.
As mentioned in Aphorism 2, the naïve forecast provides a lower bound on the accuracy you should achieve. If your process isn’t beating a naïve forecast, the quickest (and cheapest) fix is to just use the naïve forecast and eliminate your existing process.
Estimating the upper bound on forecast accuracy is a more difficult problem and beyond the scope of this article. However, be thankful if you are significantly beating a naïve forecast! In this case, you can further improve the process by focusing on process efficiency.
Efficiency can be achieved, wherever possible, by automation and removing non-value adding activities from the forecasting process. Large-scale automated forecasting software is available, and can deliver forecasts as accurate as can be expected reasonably with minimal analyst involvement. Not all demand is well-behaved, of course, so no method will work well under every circumstance. The key is to automate when you can. And when you can’t automate, track the impact of your activities to minimize wasted efforts.
Corollary
Do not overspend in pursuit of unrealistic accuracy goals. Otherwise, you will waste the company’s assets without adding any value.
Lesson: When you have reached the limits of forecast accuracy improvement but you are not achieving the accuracy mandated by your business needs (or by your annual performance objectives), it is time to stop and assess. The money spent in pursuit of unachievable accuracy goals could fund alternative approaches to your business problems. Two such alternatives are described in Aphorisms 5 and 6.
APHORISM 5
The surest way to get a better forecast is to make the demand forecastable. An underused yet highly effective solution to the forecasting problem is to make your demand more forecastable! Most (if not all) companies have some power to shape customer demand by adjusting pricing, promotions, and distribution practices. Rather than passively accepting demand patterns as “given,” a more proactive approach would be to shape the demand into more favorable (that is, less volatile and more predictable) patterns.
Organizational policies and practices largely are responsible for creating excessive or “artificial” volatility in demand. Sales incentives encourage hitting “record weeks” rather than smooth, steady, and predictable growth. Quarterly financial reporting results in the hockey stick effect – with undue amounts of sales crammed into the end of each quarter to meet short-term targets. These demand spikes are usually followed by a huge fall-off in demand at the beginning of the next quarter.
Lesson: Learn the difference between Inherent Volatility (variation in the consumption of your products by consumers) and Artificial Volatility (the variation in sales or shipments due to organizational policies and practices). It is not unusual to find the consumption of a consumer product (as indicated by the Point-of-Sale data at retail) to be stable and predictable, yet shipments of the product (from manufacturer to retail store) to be erratic. By identifying and eliminating those practices that encourage volatile shipments, you will be able to smooth shipment patterns. Reduced inventory requirements, better customer service, and better forecasting are immediate outcomes.
Corollary
Any knucklehead can forecast a straight line. In the study of process behavior, we find that a well-behaved process operates within identifiable bounds. Future observations of that process are predictable. Sales is a process. One of the fundamental tenants of the quality management is that there is no prediction without control. Taking a lesson from statistical process control, business managers must ask what they are doing to reduce variation in business processes (such as sales!), and/or what they are doing to find causes of variation. Unfortunately, our sales, marketing, and financial policies and programs normally encourage volatility rather than stability in our demand patterns. We practice quarter end pushes to meet short-term revenue targets, but they spike the demand. We award prizes for weeks with record-hitting sales. Instead, wouldn’t consistent, stable, and predictable growth be a more appropriate (and profitable) objective?
APHORISM 6
Minimize your organization’s reliance on forecasting. We forecast not because we want to, but because we have to. When forecasts are not as accurate as our organization requires, the knee-jerk reaction is to invest in more elaborate systems and processes, and then expect these expensive schemes will solve the problem. Unfortunately, such investments are often to no avail.
It may be possible to minimize your organization’s need for highly accurate forecasts. This can be achieved by improving the speed and flexibility of your supply chain. For example, if your lead times are six months, can you reduce them to six weeks? If so, your supply chain will be much more responsive in adjusting to forecast errors – so forecast accuracy is less important. Another effective approach is for a “make-to-stock” manufacturer to postpone the final product configuration or assembly until receiving an order. In this case, the manufacturer is less dependent on the accuracy of individual item forecasts, and can focus on the (easier to forecast) aggregation of raw materials, components, and production capacity that will go into the end items.
APHORISM 7
Before investing in a new process or system, put it to the test. The cost of forecasting systems can run into millions of dollars for hardware, software, training, and implementation. Yet spending all this money is never a guarantee that your forecasting problem will be solved. New systems are a great source of frustration for companies when they fail to deliver the level of forecast accuracy that was expected (or promised!) by those selling the change.
At minimum, a “Proof of Concept” should be required of each proposed solution. The POC should utilize real historical data, with holdout samples for assessing the accuracy it was able to achieve. Not only does a POC allow you to compare the performance of competing solutions, it also establishes realistic expectations for the forecasting accuracy you will be able to achieve.
(This article originally appeared in Journal of Business Forecasting, Summer 2006, and is republished here with their permission. To find out more about the Journal of Business forecasting, visit their website www.ibf.org.)