Skip to main content

Forecast Error Screen

Updated

Summary

The Forecast Error Screen has the goal of helping you understand the average forecast error of your items over time, as well as give you some data about per-period error amounts of your statistical and operational forecasts.

You can reach it by clicking to Forecast --> Forecast Error

The screen is divided into two main sections, with a master report at the top, referred to as the Forecast Error Detail, and some tabs of additional information at the bottom.

 

Forecast Error Detail Grid

The Forecast Error Detail grid contains the details about your forecast error information.  You can view saved forecast error at all levels of your forecast hierarchy using the options in the Display Options (see below).  By default, StockIQ starts with data shown at the bottom of your hierarchy, which is often the item-site-forecast-group level.

In addition to the usual item, site, and category columns, the following data is available, including if there is an alert on this record, then you will see an alert priority icon.  See Forecast Error Alert

After the initial category and alert columns, the same sets of columns are available in four different "bands", which differ based on the time periods considered in the error calculations:

  • Lead-Time Ago Errors (vs Current Actuals) - These values are all calculated from whatever forecast snapshot is one lead-time ago, aka the Lead-Time Lag.  This then compares the error of that snapshot vs now-known and backdated values to calculate the average error of this lagging forecast snapshot.  So, if today is June 1st, and the lead-time-ago snapshot is from March 1st, that is a 3-period Lag.  The error will be calculated using error values from June of last year to May of this year, which now includes 3 periods not known when the forecast snapshot was taken (March, April, May), so your average error includes 9 unknown and 3 known periods for the error calculation.  This reduces under-reporting error due to "Overfitting" of the forecast. Matches the last group of 'Forecast Errors @ Lead Time Lag -mmddyy - vs Current Actuals' in Forecast Manager Forecast Error tab.
  • Lag Period Errors - Unlike the Lead-Time Ago vs Current (above), this looks only at the errors in the lag period, e.g. the now known values vs values unknown at the time of snapshot.  In the example above, this would be your average error of March, April, and May time periods.  Shows in Snapshots Tab grid as Lag Err % in Forecast Manager.
  • Current Forecast Errors - This is a backwards looking calculations of your current forecasts, with no lag period involved.  Matches the First group column 'Forecast Error -Current' in Forecast Manager Forecast Error tab.
  • Lead-Time Ago Errors (Backwards Looking) - This is backwards looking calculations of the lead-time-ago snapshot.  Using the example above, if today is June 1st, and we have a snapshot from 3 months ago on March 1st, this would be your average error from March of the previous year to February of this year. Matches the Middle column 'Forecast Errors @ Lead Time Lag' in Forecast Manager Forecast Error tab.  These calculations are a snapshot of calculations from one lead time ago, so if data has changed that is not reflected.   

Each of these error calculations can be useful for different reasons.  The important thing is to decide on which set you want to standardize on and use that.

After selecting one or a few of those groups, in each of these groups, the following columns are available:

  • LT Ago Snapshot - Shows the date of the snapshot being used as "one lead time ago".  Since we will not always have a snapshot from exactly one lead time ago, StockIQ will find the closest one available and use that.
  • LT Ago Lag - Shows the number of periods-ago corresponding to this lag.  For example, if today is June 1st, and the snapshot is from April 1st, your lag is 2 periods = 2 months.
  • Benchmark Error % - Average error (vs current) of the naive benchmark forecast.  Typically this is MAPE (Mean Absolute Percent Error)
  • Benchmark MAE - Mean (aka Average) Absolute Error (in units) of the benchmark forecast.
  • Benchmark Error $ - Units of error translated to enterprise value.  This is used to prioritize which errors are most important to the business.
  • Benchmark Bias % - Bias percentage of the benchmark.
  • Stat Error % - Average error (vs current) of the current statistical forecast. Typically this is MAEP (Mean Absolute Percent Error)
  • Stat MAE - Mean (aka Average) Absolute Error (in units) of the stat forecast.
  • Stat Error $ - Units of error translated to enterprise value.  This is used to prioritize which errors are most important to the business.
  • Stat Bias % - Bias percentage of the stat model.
  • Stat vs Benchmark FVA - Calculates the forecast-value-added of the statistical forecast vs the naive forecast, e.g. are we doing a better job than the simplest algorithm possible.
  • Op Fcst Error % - Average error (vs current) of the current statistical forecast. Typically this is MAEP (Mean Absolute Percent Error)
  • Op Fcst MAE - Mean (aka Average) Absolute Error (in units) of the current op forecast series.
  • Op Fcst $ - Units of error translated to enterprise value.  This is used to prioritize which errors are most important to the business.
  • Op Fcst Bias % - Bias percentage of the stat model.
  • Op Fcst vs Statistical FVA - Calculates the forecast-value-added of the operational forecast vs the current statistical model.  

IMPORTANT: If the Op Forecast Statistical Forecast is negative on manually forecasted items, this indicates the demand planner may not be adding any value and this item should return to auto forecast.  Similarly, if it is positive, it indicates they are providing good insight versus what the algorithm is able to produce.

Snapshots Tab

The snapshots tab allows you to see more about each snapshot and how things have changed over time - see the Forecast Snapshots tab for more information.

Since there are potentially a lot of snapshots, this table can be a bit easier to view if you use the "maximize" button to expand that chart panel when you are comparing forecasts closely.

Compare Forecasts

The Compare Forecasts tab allows you to compare measures from your different forecast series over different lag periods, to see how they are changing over time.

This tab is broken up into a grid and a chart displaying measures from the grid.

Compare Forecasts Grid

The grid columns are as follows:

  • Series - What is the forecast series for which this row applies?  Note that you can easily filter on this column so you can focus on one series at a time if you wish.  Also, you can group on this column as well.  You will see values like:
    • Benchmark - Values from the naive benchmark forecast.  These are generated on-the-fly for whatever snapshot lags we find in the database.
    • Stat Forecast - Stat forecasts are generated on-the-fly with various lags (hiding data from the algorithm) so you can see "what would my stat forecast have been at the time?"
    • Operational Forecast - This will be values from your operational forecast snapshots at various time periods.
    • <Other Forecasts> - You may have other forecasts for which StockIQ will keep snapshots and be able to show at different lag periods.
  • Date - Shows the date of the snapshot/series that is being shown
  • Lag - The lag, in number of periods, that date corresponds two, e.g. 1 month ago, 2 months ago, etc.

After these are bands of different errors, showing the same "Vs Current", "Backwards Looking" and "Lag period" as described above.

 

Interesting things to do with this grid include:

  • Group by lag period to visually see errors of each series over time as they change
  • Filter on one series to see how its error changes over time
  • Filter on one lag period to see how each series compared at a given number of periods ago
  • Combine some of the above with Benchmark vs Stat or Stat vs Operational

Compare Forecasts Chart

The chart will display whatever error measure you select using the select box at the top right of the screen.  Use this to provide a nice visual of your key metric of choice over time, on the series(one or many) over time:

 

Display Options

There are a handful of display options to tweak what you see in the Forecast Error History screen:

  • Select Hierarchy- Which forecast hierarchy you would like to view 
  • Hierarchy Level - Which level of the hierarchy at which to view your forecast error history data.  Typically this will be at your bottom level - item-site-forecast group.
  • Forecast Series - Which forecast series should be displayed in the forecast error detail grid, e.g. your operational forecast, or one of your supplemental forecast series.
  • Show Annotations- Shows annotations that were made in Forecast Manager

Alert Settings

You can change the alert settings in the Forecast Error Alert settings screen by clicking the "Alerts" button.

Troubleshooting

Keep in mind some metrics will be blank if historical data does not exist.  StockIQ stores snapshots of historical data such as forecasts as well as calculations at that time.   Also, StockIQ changes and updates may clear out some of the saved data.

Tags/Metadata

Forecast Error Tab

 

Related articles