Monitoring information and materials to enhance logistics performance

John F. Hill

Introduction
Many enterprises, especially in the manufacturing sector, have a production lead time (including procurement of raw materials) longer than the delivery time required by their customers. In these cases it becomes necessary to anticipate demand by making forecasts. It is not intended to discuss here the techniques of forecasting, which have been explored thoroughly elsewhere (see, for example, [1]). It must be recognized that, however scientific the methods of forecasting might seem, they are still an attempt to foretell the future. There is no method of seeing ahead with certainty and, to some greater or lesser extent, all forecasting techniques involve projecting experience of the past into the future. This uncertainty makes it necessary for forecasts to be monitored to assess their quality, and to allow corrections to be made if systematic errors are discovered.

The author John F. Hill is based at Warwick Manufacturing Group, University of Warwick, Coventry, UK. Abstract The need to make forecasts is commonplace in industry. The accuracy (and hence, usefulness) of forecasts varies from one sector to another, but is rarely high. Describes an approach to the monitoring of forecasts which aims to detect significant inaccuracy and to quantify the correction(s) that need to be made, based on well-established methods of SPC, and so it also allows any changes in the pattern of demand to be detected, providing information for the revision of the forecasting model. Since the monitoring technique is based on statistical methods, it provides information which can be used directly in conventional models for inventory management. The user can choose the level of coverage required, and the monitoring technique indicates how well the target is being met. Again, any departure from the required level can be indicated at a stage early enough to take corrective action.

Types of variation/error and their detection
In statistical language “error” is the term used to describe scatter or random variation in a set of data. In the short term this variation in demand data can appear quite considerable. In the medium to long term it is usually the case that demand conforms to some kind of pattern; indeed, if this were not true, any kind of planning would be impossible. The first requirement of the assessment or monitoring method is to separate the underlying pattern from the random variation (or “error” in statistical terminology). Given that statistical error is present in all data, then even when the basic pattern in demand has been identified it is difficult to detect when a change occurs in that pattern. Apparent changes could be due to further random variation. Thus the second requirement is a method of identifying real changes, and distinguishing them from background “noise”. The third requirement is to be able to measure the accuracy with which the forecast predicts the actual demand, so that corrections may be made where necessary. The data shown in Figure l(a) and l(b) illustrate some of the characteristics referred
An earlier version of this paper was presented at the “Customer focused logistics” Conference, 1-2 November 1995, Birmingham, UK.

Logistics Information Management Volume 9 · Number 2 · 1996 · pp. 10–15 © MCB University Press · ISSN 0957-6053

10

Monitoring information and materials to enhance logistics

Logistics Information Management Volume 9 · Number 2 · 1996 · 10–15

John F. Hill

to in the last three paragraphs. These examples are taken from a series of studies recently carried out at the University of Warwick, and are typical of the majority of the 30 or so cases examined. In each case the forecast is the customer’s own assessment of expected needs one month (or in some cases one week) ahead of due date. The actual demand was notified by telephone, fax or other electronic means on the due date (or sometimes shortly after the due date!). It can be seen that there is little apparent correlation between forecast and

actual demand in the short term. Subjectively, it seems that the forecasting in case A is better than in case B, but both leave much to be desired. In fact, the situation described above can be greatly improved by the use of some simple statistical tools. Figure 2(a) and 2(b) show the same data as those of Figure 1(a) and 1(b), but plotted as cumulative forecast and cumulative actual demand. Even viewed intuitively, there is a greater semblance of order to the information.

Figure 1 Comparison of forecast and actual demand for manufactured products Demand 12,000 10,000 8,000 6,000 4,000 2,000 0

1 2 3 4 Week number

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 (a) Data from Case (a)

Demand 700 600 500 400 300 200 100 0

1 2 3 4 Week number Key

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 (b) Data from Case (b)

Actual demand

Forecast demand

11

Monitoring information and materials to enhance logistics

Logistics Information Management Volume 9 · Number 2 · 1996 · 10–15

John F. Hill

Cumulative data charts, known as cumulative sums (or CUSUMS), provide a different, often more revealing view of the information. Since the horizontal axis of the chart shows equal intervals of time, if the level of demand (actual or forecast) were constant this would give a regular increment on the vertical axis, and the resultant graph would be a perfect straight line. The slope indicates the average demand. In the more likely, real-life situation, the weekly demand varies around some constant, or average value. The size of each deviation from the average is usually random, and may be positive or negative. When the data

are plotted as a CUSUM the random deviations tend to cancel out, and the result is as shown in Figure 2(a), with a line which is effectively straight, but with minor perturbations along its length. Any change in average demand will be indicated by a change in the slope of the line, as in the forecast values in Figure 2(b). The first two requirements identified above for assessing the pattern of demand can be satisfied by this approach, then. The slope of the line provides the average demand, and the deviations around this may be quantified quite easily in the conventional way, using the

Figure 2 Forecast and actual demand plotted as cumulative sums Cumulative demand 200,000 180,000 160,000 140,000 120,000 100,000 80,000 60,000 40,000 20,000 0 1 2 3 4 5 Week number 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 (a) Data from Case (a)

Cumulative demand 12,000 10,000 8,000 6,000 4,000 2,000 0 1 2 3 4 5 Week number Key Actual 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 (b) Data from Case (b) Forecast

12

Monitoring information and materials to enhance logistics

Logistics Information Management Volume 9 · Number 2 · 1996 · 10–15

John F. Hill

standard deviation or some other quantitative measure of variation. In a similar fashion, the accuracy of the predictions is shown by plotting the forecast values and adding actual demand data as they become available, as in Figure 2. Again, it is clear from even a cursory examination, that the forecast provided in case A is much better than in case B. In fact, the error in case A is about 1 per cent over the half year shown in the example, while for case B the error over the first 12 weeks averages about 50 per cent, and over the second 12 it is about 6 per cent. The improvement is indicated by the lines for forecast and actual demand becoming almost parallel. An alternative, and probably better, approach would be to calculate the difference between forecast and actual demand, and to monitor this value by the same method as that described previously.

Quantifying the approach
So far, the methods have been illustrated graphically, indicating the principles on which a system of monitoring may be based. However, in order to be of real use it has to be possible to provide objective, quantitative criteria. The magnitude of the change that it is required to detect must be related to the use to which the forecast is to be put. For example, a change in demand that would deplete safety stocks so that they no longer provided adequate cover would need to be signalled as soon as possible. As with all decision making based on statistics, the criteria for action are based on probabilities. In this example the signal that a change has occurred, or that a correction to the forecast is needed, is given when the probability exceeds some preselected level. The chosen probabilities have to achieve a compromise between the risk of taking action when no change has occurred, and the risk of failing to act soon enough when a change has occurred. One device for indicating a change uses a “V-mask”, an example of which is shown in Figure 3. The geometry of this mask (principally the angle between the arms of the V) can be chosen to give an acceptable level of sensitivity. The balance takes account of the magnitude of the change which it is desired to detect, and the speed with which changes are signalled, as well as the risk of false alarms, as mentioned earlier. 13

As an illustration of its use, a mask is shown in Figure 4, superimposed on a CUSUM plot derived from the actual demand in case B. The average of the first ten values has been used to establish the norm, and the plotted values are the cumulative deviations from this norm. It can be seen that the cumulative total (CUSUM) of the first ten points is zero, as would be expected from the method of calculation. In this schematic example the mask is shown being used in week 15. The intersection of the graph with one of the arms of the mask indicates a change has occurred, greater than the critical value for the selected geometry. The design of V-masks for CUSUM charts may be based on the average run length (ARL). After a change has occurred, this is the average number of values which will be recorded before it is indicated by the monitoring system. In general, it is desirable for the ARL to be long when the situation is stable, with no significant change occurring, and for the ARL to be short when changes in demand do arise or the forecast does deviate from the actual demand.

Tabular CUSUM charts
As an alternative to the graphical approach illustrated above, it is sometimes preferable to use a tabular method, which can be prepared, and maintained on a spreadsheet. Applying the tabular CUSUM method to the examples used earlier, there is no indication given of any need for adjustment or corrective action for case A, i.e. there are no spurious signals when the relationship between Forecast and Actual demand is stable and well behaved, and demand is reasonably constant. However, applying the tabular calculations to case B we note: the discrepancy between forecast and actual data is indicated by week three; the

Figure 3 V-mask for use with CUSUM charts

Monitoring information and materials to enhance logistics

Logistics Information Management Volume 9 · Number 2 · 1996 · 10–15

John F. Hill

Figure 4 CUSUM chart of deviations from “normal” demand indicated by V-mask CUSUM 500.00 0 –500.00 –1,000.00 –1,500.00 –2,000.00 –2,500.00 –3,000.00 –3,500.00 1 2 3 4 Week number 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

change in the average forecast error is indicated in week 15. The points at which these deviations are indicated by the CUSUM monitoring system are shown in Figure 5. In summary, the method has detected the changes and/or discrepancy which did occur within two or three data points. Where no change or discrepancy occurred, the method avoided any irrelevant call for action.

Control of inventories
The traditional methods for control of inventory rely on a forecast of demand, plus some adjustment for “safety stock”. There are a number of reasons for holding safety stock,

but they all amount to insurance against unforeseen events, such as demand being higher than expected, or supply being lower than expected. It is often recommended that the level of safety allowed for should be calculated on the basis of the probability of a “stock out”, and textbook techniques are usually based on this approach. However, many firms lack the necessary data or insight into risk to use this method, and so arbitrary levels are held, usually without much idea of the balance between the cost and the protection being provided. The techniques described above can be used to quantify the levels of risk, because they are based on the probabilities of over-

Figure 5 Changes in deviation and forecast average indicated by the tabular CUSUM method 12,000 10,000 8,000 6,000 4,000 2,000 0 1 2 3 4 5 Week number Key Actual 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Deviation of forecast from actual demand Change in average forecast indicated

Forecast

14

Monitoring information and materials to enhance logistics

Logistics Information Management Volume 9 · Number 2 · 1996 · 10–15

John F. Hill

reacting or under-reacting, and are aimed at a known level of demand. This offers the opportunity to plan production schedules and inventory policies together, with a much greater understanding of the risks being taken in relation to the stocks being held. For example, in case A above, it would be safe to level production to the same quantity each week, and the safety stock necessary to ensure supply could be calculated very easily from the data provided by the systems described. On the other hand, while case B would also allow levelled production, the actual quantity made would involve an adjustment from the forecast level. Continuous monitoring of the forecast would suggest in week 15 that some change had occurred around week 12. This, in turn, would require an alteration to the schedule, and a consultation with the customer to see if their intentions had really altered since the start of the programme. In both cases the safety levels could be predetermined, and confidence in the system could be gained by using historical data in a simulation or “what-if ?” trial.

Benefits and conclusions
The need to plan activities before any firm and reliable data are available underlies many of the difficulties of effective management. Scheduling, control of capacity and of inventory, all require the kind of gambles that we identify with logistics and operations management. Knowledge about the quality of our forecasts, and those of our customers, must be of benefit, therefore. The management of “independent demand” inventories is commonly based on models of usage and replenishment which require some knowledge of future demand. In addition, the methods for calculating safety stocks, by taking account of “probable” usage rates within the replenishment lead time are well known and widely documented. In both of these cases the lack of certainty about the quality of the forecast data is a major drawback, which frequently leads to arbitrary decisions, such as “we will hold safety stocks equal to three weeks demand”. These rules undermine the drive to optimize inventory levels, and can rarely be explained by any quantitative analysis of the real purpose and need for safety stocks. The examples used to illustrate this paper have been drawn from industrial data which show, only too clearly, the difficulties faced by many supply companies as a matter of daily routine. The ability to monitor the quality of forecast data, and to modify the figures if and when they are shown to contain systematic errors, will not solve all of the problems of establishing decision rules for inventory management. Nonetheless it does provide a greater possibility of quantifying the risk associated with any chosen policy, and puts responsibility and control back into the hands of management.

Alternative and additional methods
There is a spectrum of statistical techniques available for process monitoring, and while some (such as the conventional Shewhart control chart) are particularly suitable for detecting large, sudden deviations, others (such as CUSUMS) are better when the deviations are gradual and/or relatively small. The optimum approach may be to use a combination of techniques, therefore. However, if it is desired to keep the overall monitoring in a compact, and easily operated form then the CUSUM methods (especially Tabular CUSUMS) offer a practical compromise, with known and predictable performance. The examples presented in this paper indicate the kind of approach that can be adopted, and show how the quality of forecasts can be quantified and tuned to a higher and more useful standard.

Reference
1 Makridakis, S., Wheelwright , S.C. and McGee, V.B., Forecasting, Wiley, Chichester, 1983.

15

Sign up to vote on this title
UsefulNot useful