📉 Time-Series Forecasting Methods for Engineers: Classical Statistics vs. Machine Learning

Forecasting Is a Different Problem From Standard Machine Learning

Time-series forecasting — predicting future values of a process from its own history, such as tomorrow's energy demand, next week's throughput, or next month's maintenance backlog — differs from standard supervised machine learning in one crucial way: the order of the data carries information that cannot be discarded or shuffled. This article compares the classical statistical forecasting methods engineers have relied on for decades against modern machine learning approaches, explains the concept of stationarity that underlies the classical methods, and covers how to correctly split time-series data for training and evaluation using this studio's Train/Test Split & Cross-Validation Planner as a starting point for the mechanics. This is a general methods guide — for a worked, applied case study specific to building loads and renewable generation, see this studio's separate article on AI for energy forecasting.

The Building Blocks: Trend, Seasonality, and Noise

Almost every engineering time series can be thought of as the sum (or product) of three components. Trend is the long-term direction of the series — gradually increasing energy demand as a facility grows, or gradually increasing vibration amplitude as a bearing wears. Seasonality is a repeating pattern at a fixed, known period — daily temperature cycles, weekly production schedules, annual heating/cooling load patterns. Noise (or the "residual") is everything left over after trend and seasonality are accounted for — the irreducible randomness from measurement error and unmodeled short-term effects. Identifying which of these components dominate your series is the first diagnostic step before choosing a forecasting method, because different methods are built to handle different combinations of these components.

Stationarity: The Assumption Behind Classical Methods

A time series is stationary if its statistical properties — mean, variance, and the way values correlate with their own past values (autocorrelation) — remain roughly constant over time. A series with a strong upward trend is not stationary, because its mean keeps changing; a series with growing seasonal swings is not stationary, because its variance keeps changing. Many classical forecasting methods, ARIMA in particular, are built on mathematics that assumes stationarity, so a non-stationary raw series typically needs to be transformed before those methods can be applied reliably.

Making a Series Stationary: Differencing

The most common transformation is differencing — replacing each value with the difference between it and the previous value (or the value one seasonal period earlier, for seasonal differencing). A series with a linear trend often becomes stationary after first-order differencing; a series with a strong repeating seasonal pattern often needs seasonal differencing as well. Statistical tests such as the Augmented Dickey-Fuller (ADF) test can check whether a series is stationary, but in practice, a visual inspection of the plotted series (does the mean level drift? does the spread of values grow over time?) is often enough to guide the decision.

Classical Method 1: Moving Average

The simplest forecasting method, a simple moving average, forecasts the next value as the unweighted average of the last N observed values. It is easy to compute and understand, and it does a reasonable job of smoothing out short-term noise, but it has real limitations: it weights an observation from 20 periods ago exactly the same as yesterday's observation, it lags behind genuine trends (since it is, by construction, always looking backward), and it has no mechanism for handling seasonality at all. Moving averages are most useful as a quick baseline or as a smoothing step within a larger analysis, rather than as a serious forecasting method on their own for anything with meaningful trend or seasonal structure.

Classical Method 2: Exponential Smoothing

Exponential smoothing improves on the moving average by weighting recent observations more heavily than older ones, with the weight decaying exponentially the further back in time you go, controlled by a smoothing parameter α (alpha) between 0 and 1. A higher α makes the forecast more responsive to recent changes; a lower α makes it smoother and less reactive to short-term noise.

• Simple exponential smoothing handles series with no clear trend or seasonality — just a slowly evolving level.
• Holt's linear trend method (double exponential smoothing) adds a second smoothed component to track a trend, appropriate for series that are steadily rising or falling.
• Holt-Winters (triple exponential smoothing) adds a third smoothed component to track seasonality on top of level and trend, appropriate for series like daily energy demand with a clear repeating weekly or daily pattern.

Exponential smoothing methods are widely used in industry because they are computationally cheap, easy to update as new data arrives, and produce forecasts that are usually competitive with far more complex methods for short-to-medium forecast horizons.

Classical Method 3: ARIMA

ARIMA (AutoRegressive Integrated Moving Average) is a more flexible and more mathematically involved classical method, combining three components: the AR (autoregressive) part predicts a value as a weighted combination of its own recent past values; the I (integrated) part applies differencing to make the series stationary before modeling; and the MA (moving average) part (not the same as the simple moving average method above — a different, more technical use of the term) predicts a value based on the errors of recent past forecasts, allowing the model to self-correct. An ARIMA model is written ARIMA(p, d, q), where p is the number of autoregressive terms, d is the degree of differencing applied, and q is the number of moving-average error terms. A SARIMA (Seasonal ARIMA) extension adds a matching set of seasonal terms for series with a clear repeating period.

ARIMA's strengths are its strong statistical grounding — it produces genuine confidence intervals around its forecasts, not just point estimates — and its ability to capture more complex autocorrelation structure than exponential smoothing. Its costs are added complexity: selecting good (p, d, q) values requires either careful diagnostic work (examining autocorrelation and partial autocorrelation plots) or an automated search procedure, and ARIMA models generally handle only a single series at a time, with limited ability to incorporate external explanatory variables without moving to an extended variant (ARIMAX).

Machine Learning Approaches to Forecasting

Machine learning reframes forecasting as a standard supervised learning problem: use lag features, rolling window features, and calendar features (day of week, hour of day, holiday flags) — exactly the kind of features covered in this studio's feature engineering article — as inputs, and train a regression model (gradient-boosted trees like XGBoost/LightGBM, random forests, or neural networks such as LSTMs for longer sequences) to predict the target value.

A practical, low-risk strategy: always fit a classical baseline (exponential smoothing is usually enough) first. It takes minutes to build, and it sets a bar that a more complex ML model must clearly and consistently beat, using a proper time-series validation, before the added complexity is justified. Teams that skip this step sometimes deploy an elaborate ML forecasting pipeline that turns out to perform no better than Holt-Winters smoothing would have.

Splitting Time-Series Data Correctly

The single most important rule specific to time-series evaluation: never use a random train/test split. Randomly shuffling rows before splitting means the training set can end up containing observations from after the test period, letting the model implicitly "see the future" during training — a form of data leakage that inflates apparent accuracy in a way that will not hold up once the model is deployed and genuinely has no access to future data. This studio's Train/Test Split & Cross-Validation Planner supports this distinction directly; for time series, always choose a chronological split rather than a random or stratified one.

Chronological (Out-of-Time) Splitting

The correct approach is a chronological split: pick a cutoff date, use everything before it for training, and everything after it for testing — mimicking exactly how the model will be used in production, where only past data is ever available at prediction time.

Forward-Chaining (Rolling-Origin) Cross-Validation

To get a more robust estimate than a single chronological split provides, use forward-chaining cross-validation (also called rolling-origin or walk-forward validation): train on data up to time T₁, test on the period immediately after; then train on data up to a later time T₂ (incorporating the previously held-out test period into training), test on the next period after that; and repeat this process forward through the dataset. Each fold's test set always occurs strictly after its corresponding training set, preserving the chronological integrity that time-series evaluation requires, while still producing multiple performance estimates to average, similar in spirit to standard k-fold cross-validation but respecting time order.

Choosing a Forecast Horizon and Evaluation Metric

Forecast accuracy typically degrades the further into the future you try to predict, so always evaluate a model at the specific forecast horizon (next hour, next day, next week) that matches how it will actually be used — a model that forecasts one step ahead well may perform much worse at ten steps ahead, and reporting only the easier metric can be misleading. Common time-series-specific metrics include MAE (mean absolute error, in the original units, easy to interpret), RMSE (root mean squared error, penalizes large errors more heavily), and MAPE (mean absolute percentage error, useful for comparing forecast quality across series with different scales, though it becomes unstable when actual values are near zero).

A Practical Decision Framework

• Single series, clear trend/seasonality, need for confidence intervals: start with Holt-Winters exponential smoothing; move to SARIMA if residual diagnostics show autocorrelation the smoothing model is not capturing.
• Many related series, rich external features, enough history: a gradient-boosted tree model (XGBoost/LightGBM) with well-engineered lag, rolling, and calendar features is usually the strongest, most practical choice.
• Always: check stationarity before applying ARIMA, split chronologically (never randomly), validate with forward-chaining rather than standard k-fold, and evaluate at the specific forecast horizon your application actually needs.

Worked Example: Baselining a Compressor Load Forecast

Suppose you need to forecast hourly compressed-air demand for a manufacturing plant one day ahead, using two years of hourly historian data. A quick visual check of the plotted series shows a clear daily cycle (higher demand during production shifts, near-zero overnight) and a weaker weekly cycle (lower demand on weekends), with no strong long-term trend — a good candidate for Holt-Winters exponential smoothing as a first baseline, since the series has both seasonal structure and a level component but no obvious nonstationary trend requiring differencing.

Fitting Holt-Winters with a 24-hour seasonal period gives a baseline forecast; evaluating it with a chronological split (train on the first 20 months, test on the last 4 months, never shuffled) yields a mean absolute percentage error (MAPE) of, say, 9%. Before investing engineering time in a more complex ML pipeline, this baseline number becomes the bar to beat: a gradient-boosted tree model using lag features (demand 1, 24, and 168 hours ago), rolling 24-hour mean and standard deviation, and calendar features (hour of day, day of week, production-shift flag) is then trained and evaluated using the same chronological test period and the same MAPE metric. If the ML model achieves, say, 6% MAPE — a real, meaningful improvement over the 9% baseline — the added complexity is justified. If it instead lands at 8.5% MAPE, only marginally better than Holt-Winters despite far more engineering effort, the simpler classical model is very likely the better production choice, since it is easier to maintain, faster to retrain, and easier for a plant engineer to interpret and trust.