Time Series: Difference between revisions

From BloomWiki
Jump to navigation Jump to search
BloomWiki: Time Series
 
BloomWiki: Time Series
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
<div style="background-color: #4B0082; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
{{BloomIntro}}
{{BloomIntro}}
AI for time series and forecasting applies machine learning and deep learning techniques to sequential, time-indexed data to predict future values, detect anomalies, and extract patterns. Time series data is ubiquitous: stock prices, electricity demand, web traffic, sensor readings, weather measurements, and patient vital signs all evolve over time. Traditional forecasting relied on statistical models like ARIMA; modern AI-driven approaches — including LSTMs, Temporal Fusion Transformers, and foundation models for time series — now achieve state-of-the-art performance across domains.
Time Series Analysis is a statistical technique that deals with time series data, or trend analysis. Time series data is a sequence of data points indexed in time order (e.g., stock prices, weather data, or population growth). Unlike standard statistics, which assumes data points are independent, time series analysis recognizes that "what happened yesterday" often affects "what happens today." By identifying patterns such as trends, seasonality, and cycles, time series analysis allows us to forecast the future, detect anomalies, and understand the dynamic forces that drive a system over time.
</div>


== Remembering ==
__TOC__
* '''Time series''' — A sequence of data points indexed in time order, typically at regular intervals.
* '''Forecasting''' — Predicting future values of a time series based on its historical patterns.
* '''Univariate time series''' — A single variable measured over time (e.g., daily sales).
* '''Multivariate time series''' — Multiple variables measured simultaneously over time (e.g., temperature, humidity, and pressure together).
* '''Trend''' — The long-term direction of a time series (upward, downward, or flat).
* '''Seasonality''' — Regular, periodic patterns that repeat at known intervals (daily, weekly, yearly).
* '''Residuals''' — The component remaining after removing trend and seasonality; ideally random noise.
* '''Stationarity''' — A time series is stationary if its statistical properties (mean, variance) do not change over time. Many models require stationarity.
* '''Autocorrelation''' — The correlation of a time series with its own past values (lags).
* '''Lag''' — A prior time step. Lag-1 is yesterday's value; lag-7 is last week's value.
* '''ARIMA''' — AutoRegressive Integrated Moving Average; a classical statistical model for univariate forecasting.
* '''LSTM (Long Short-Term Memory)''' — A type of RNN with gating mechanisms that captures long-range dependencies in sequences.
* '''Temporal Fusion Transformer (TFT)''' — A transformer-based model for multi-horizon time series forecasting, incorporating attention across time.
* '''Anomaly detection''' — Identifying data points, intervals, or patterns that deviate significantly from expected behavior.
* '''Horizon''' — The number of future time steps to forecast (1-step-ahead vs. multi-step/multi-horizon).
* '''Rolling forecast''' — Re-fitting or updating the model as new data arrives, maintaining accuracy over time.


== Understanding ==
<div style="background-color: #000080; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
Time series forecasting is inherently a sequential problem: the order of observations matters, and the past contains information about the future. This distinguishes it from tabular classification, where rows are exchangeable.
== <span style="color: #FFFFFF;">Remembering</span> ==
* '''Time Series''' — A series of data points indexed (or listed or graphed) in time order.
* '''Trend''' — The long-term increase or decrease in the data (e.g., global warming).
* '''Seasonality''' — Predictable patterns that repeat over a fixed period (e.g., ice cream sales in summer).
* '''Cyclical Component''' — Patterns that repeat but not over a fixed period (e.g., economic "boom and bust").
* '''Noise (Irregular)''' — Random variations that cannot be explained by trend or seasonality.
* '''Stationarity''' — A property where the statistical characteristics of the series (mean, variance) do not change over time.
* '''Autocorrelation''' — The correlation of a signal with a delayed version of itself (how today relates to yesterday).
* '''Lag''' — The amount of time by which one variable is separated from another.
* '''Moving Average''' — A calculation to analyze data points by creating a series of averages of different subsets of the full data set.
* '''Forecasting''' — The process of making predictions of the future based on past and present data.
* '''ARIMA (AutoRegressive Integrated Moving Average)''' — A popular statistical model for time series forecasting.
* '''Smoothing''' — A technique to remove "noise" from a data set to see the underlying pattern.
* '''Exponential Smoothing''' — A technique that gives more "weight" to recent data points when forecasting.
* '''Seasonally Adjusted''' — Data that has had the seasonal component removed to reveal the true trend.
</div>


'''The decomposition framework''' is key to understanding time series:
<div style="background-color: #006400; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
<syntaxhighlight lang="text">
== <span style="color: #FFFFFF;">Understanding</span> ==
Observed = Trend × Seasonal × Residual  (multiplicative)
Time series analysis is about "Decomposing" a complex signal into its parts.
        = Trend + Seasonal + Residual  (additive)
</syntaxhighlight>
Decomposing a series into these components enables targeted modeling: model the trend with regression, the seasonality with Fourier features or indicator variables, and the residual with a neural network or ARIMA.


'''Why deep learning?''' Classical models like ARIMA excel at capturing simple autocorrelation but struggle with:
'''The Decomposition Equation''':  
* Non-linear relationships between variables
$Data = Trend + Seasonality + Cycle + Noise$
* Multiple interacting series (multivariate)
By pulling these apart, we can see that a 10% rise in sales might just be "Seasonality" (it's Christmas), not a "Trend" (your business is growing).
* Complex, multi-scale seasonality
* Incorporating exogenous variables (weather, holidays, promotions)


LSTMs can capture non-linear temporal dependencies and handle arbitrary-length sequences. Transformers add the ability to attend to any past time step directly, avoiding the vanishing gradient problem over long sequences. Foundation models for time series (TimeGPT, MOIRAI, Chronos) pre-trained on billions of time points can zero-shot forecast on new series.
'''Stationarity (The Golden Rule)''': Most statistical models only work if the data is "Stationary." If your data has a trend (it's going up), it's not stationary. To fix this, we use '''Differencing'''—looking at the ''change'' from day to day ($X''t - X''{t-1}$) instead of the raw values.


'''Evaluation discipline''': A critical mistake in time series is using random train/test splits. This causes data leakage — future data leaks into the training set. Always use chronological splits: train on the first 70–80%, validate on the next 10–15%, test on the most recent 10–15%.
'''Autoregression (AR)''': This is the idea that the current value is a linear function of previous values. If it rained yesterday, it is more likely to rain today. The strength of this relationship is measured by '''Autocorrelation'''.
 
</div>
== Applying ==
'''Multi-horizon forecasting with Temporal Fusion Transformer (PyTorch Forecasting):'''


<div style="background-color: #8B0000; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
== <span style="color: #FFFFFF;">Applying</span> ==
'''Calculating a Simple Moving Average (SMA):'''
<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
import pandas as pd
def simple_moving_average(data, window_size):
from pytorch_forecasting import TimeSeriesDataSet, TemporalFusionTransformer
     """
from pytorch_forecasting.metrics import QuantileLoss
     Smooths out a time series by averaging over a window.
import lightning.pytorch as pl
     """
 
     averages = []
# Load data: each row = one time step for one series
     for i in range(len(data) - window_size + 1):
df = pd.read_csv("sales_data.csv")
        window = data[i : i + window_size]
df["time_idx"] = (df["date"] - df["date"].min()).dt.days  # integer time index
        averages.append(sum(window) / window_size)
 
    return averages
max_encoder_length = 60  # Use 60 past days as context
max_prediction_length = 14  # Forecast 14 days ahead
 
# Training dataset
training = TimeSeriesDataSet(
    df[lambda x: x.time_idx <= x.time_idx.max() - max_prediction_length],
     time_idx="time_idx",
    target="sales",
     group_ids=["store_id", "product_id"],      # Multiple series
     min_encoder_length=30,
    max_encoder_length=max_encoder_length,
    max_prediction_length=max_prediction_length,
    static_categoricals=["store_id", "product_id"],
     time_varying_known_reals=["time_idx", "price", "day_of_week", "is_holiday"],
     time_varying_unknown_reals=["sales"],      # Only target is "unknown" in future
    target_normalizer="auto",
)
 
validation = TimeSeriesDataSet.from_dataset(training, df, predict=True, stop_randomization=True)
train_dl = training.to_dataloader(train=True, batch_size=64)
val_dl = validation.to_dataloader(train=False, batch_size=64)


# TFT model
# Stock prices over 10 days
tft = TemporalFusionTransformer.from_dataset(
prices = [100, 102, 101, 105, 110, 108, 115, 120, 118, 125]
    training,
sma_3 = simple_moving_average(prices, 3)
    learning_rate=0.03,
    hidden_size=64,
    attention_head_size=4,
    dropout=0.1,
    hidden_continuous_size=32,
    output_size=7,         # 7 quantile predictions (p10 to p90)
    loss=QuantileLoss(),
    log_interval=10,
)


trainer = pl.Trainer(max_epochs=30, accelerator="gpu", gradient_clip_val=0.1)
print(f"Original Prices: {prices}")
trainer.fit(tft, train_dl, val_dl)
print(f"3-Day SMA:      {sma_3}")
# Note how the 'zig-zags' are smoothed out, making the upward
# trend easier to see.
</syntaxhighlight>
</syntaxhighlight>


; Model selection guide by forecasting scenario
; Time Series in the Real World
: '''Simple univariate, clean seasonality''' → SARIMA, Prophet (Meta), ETS
: '''Stock Market''' → Analyzing price "momentum" and volatility.
: '''Univariate with complex patterns''' → N-BEATS, N-HiTS, PatchTST
: '''Epidemiology''' → Tracking the "R-number" (spread of a virus) over time.
: '''Multivariate with known future covariates''' → Temporal Fusion Transformer, DeepAR
: '''IoT / Sensors''' → Detecting an "Anomaly" (e.g., a machine vibrating weirdly) in a constant stream of data.
: '''Very short series or irregular intervals''' → Gaussian Processes, ARIMA
: '''Inventory Management''' → Predicting how many turkeys to stock before Thanksgiving based on the last 5 years of data.
: '''Many series, zero-shot''' → TimeGPT, Chronos, MOIRAI (foundation models)
</div>
: '''Anomaly detection''' → Isolation Forest (tabular features), LSTMAD, Anomaly Transformer


== Analyzing ==
<div style="background-color: #8B4500; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
== <span style="color: #FFFFFF;">Analyzing</span> ==
{| class="wikitable"
{| class="wikitable"
|+ Time Series Model Comparison
|+ Additive vs. Multiplicative Seasonality
! Model !! Type !! Strengths !! Weaknesses
! Type !! Description !! Example
|-
| ARIMA/SARIMA || Statistical || Interpretable, fast, works on small data || Assumes linearity, one series at a time
|-
| Prophet || Statistical || Handles holidays, trend changepoints || Limited to single series; no covariates
|-
| DeepAR || Deep Learning (LSTM) || Probabilistic, many series || Needs lots of data, slow training
|-
| TFT || Transformer || Multi-horizon, covariate-rich, interpretable || Complex, high data requirement
|-
|-
| N-BEATS || Deep Learning (MLP) || Fast, competitive, no feature engineering || Limited covariate support
| Additive || Seasonal variation is constant (±10 units) || Monthly gym attendance
|-
|-
| Chronos (foundation) || LLM-style || Zero-shot, no training needed || No covariate support yet; large model
| Multiplicative || Seasonal variation grows with the trend (±10%) || Airline passengers (more people travel in summer as the total industry grows)
|}
|}


'''Failure modes:'''
'''The Random Walk''': Some time series (like the stock market in the short term) are "Random Walks." This means the best prediction for tomorrow is simply "whatever it is today" plus some random noise. In a true random walk, technical analysis (looking at charts) is impossible—a hard lesson for many traders.
* '''Chronological leakage''' — Random train/test splits allow future data to inform past predictions, producing falsely optimistic results. Always split chronologically.
</div>
* '''Ignoring non-stationarity''' — Many models assume stationarity. Differencing (ARIMA) or normalization per-series is required.
* '''Ignoring distributional shift''' — Retail models trained pre-COVID performed terribly during COVID. Extreme events cause structural breaks that no model trained on historical data anticipates.
* '''Point forecast overconfidence''' — Reporting only mean forecasts without uncertainty intervals. Downstream planning needs to understand the range of outcomes, not just the median.
* '''Evaluation on last segment only''' — Evaluating only on the final test period may not represent the model's general quality. Use rolling window backtesting across multiple historical windows.
 
== Evaluating ==
Expert time series evaluation uses multiple metrics and rigorous experimental design:
 
'''Regression metrics''': MAE (Mean Absolute Error), RMSE, MAPE (Mean Absolute Percentage Error), sMAPE. MAPE is undefined when actual=0 and is skewed by near-zero values; sMAPE or MAE are more robust.
 
'''Probabilistic metrics''': For probabilistic forecasts (quantile or interval), use CRPS (Continuous Ranked Probability Score) or Winkler score. These reward well-calibrated uncertainty.
 
'''Rolling window backtesting''': Instead of one train/test split, slide a window across history — train on windows [0:T], [0:T+1], … and evaluate on each subsequent step. This tests the model across many historical regimes and avoids cherry-picking a favorable test period.
 
'''Naive benchmarks''': Always compare to: naive (last value), seasonal naive (same period last cycle), and exponential smoothing. If a complex deep learning model cannot beat seasonal naive, it's not adding value.
 
Expert practitioners report backtesting results as distributions (mean ± std across windows) rather than a single number, and explicitly test for robustness during unusual periods (holidays, pandemics, market crashes).


== Creating ==
<div style="background-color: #483D8B; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
Designing a production time series forecasting system:
== <span style="color: #FFFFFF;">Evaluating</span> ==
 
Evaluating a forecast:
'''1. Data architecture'''
# '''RMSE (Root Mean Square Error)''': How far off were our predictions on average?
<syntaxhighlight lang="text">
# '''Overfitting''': Does the model work perfectly on old data but fail on new data?
Raw time series sources (databases, IoT, APIs)
# '''Lag''': Does the model "lag" behind reality (only predicting a crash ''after'' it starts)?
    ↓
# '''White Noise''': Are the "residuals" (the errors) random? If the errors have a pattern, it means our model is missing something.
[Time-indexed storage: InfluxDB, TimescaleDB, or Parquet partitioned by date]
</div>
    ↓
[Feature engineering pipeline:]
│  ├── Temporal features: hour, day of week, month, quarter
│  ├── Lag features: lag-1, lag-7, lag-28, rolling mean/std
│  ├── Fourier features for seasonality
│  └── External covariates: weather, holidays, promotions
    ↓
[Stationarity tests + differencing if needed]
    ↓
[Train/val/test split: chronological]
</syntaxhighlight>
 
'''2. Model training and selection'''
<syntaxhighlight lang="text">
Train multiple models (baseline naive, SARIMA, TFT, foundation model)
    ↓
Evaluate each with rolling window backtesting
    ↓
Select winner by held-out test MAPE and CRPS
    ↓
Train ensemble: weighted average of top-3 models (often beats any single model)
    ↓
Register in model registry with evaluation metrics
</syntaxhighlight>


'''3. Production serving and retraining'''
<div style="background-color: #2F4F4F; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
* Serve forecasts via API with caching (same-day forecast is rarely regenerated)
== <span style="color: #FFFFFF;">Creating</span> ==
* Nightly retrain on latest data window (rolling retrain strategy)
Future Frontiers:
* Monitor forecast accuracy vs. actuals in real-time; alert on anomalies
# '''LSTM (Long Short-Term Memory)''': Using deep learning (RNNs) to capture extremely long-term dependencies in data (e.g., weather patterns over decades).
* Detect distribution shift: plot forecast distribution vs. actuals weekly
# '''Real-time Stream Processing''': Analyzing time series as they happen at a rate of millions of points per second (used in fraud detection).
* Trigger manual review when MAE exceeds historical 95th percentile
# '''Multivariate Time Series''': Predicting one variable (e.g., sales) using 50 others (e.g., weather, social media sentiment, competitor prices).
# '''Causal Time Series''': Determining if one time series ''causes'' another (Granger Causality).


[[Category:Artificial Intelligence]]
[[Category:Statistics]]
[[Category:Machine Learning]]
[[Category:Data Science]]
[[Category:Time Series]]
[[Category:Finance]]
</div>

Latest revision as of 02:00, 25 April 2026

How to read this page: This article maps the topic from beginner to expert across six levels � Remembering, Understanding, Applying, Analyzing, Evaluating, and Creating. Scan the headings to see the full scope, then read from wherever your knowledge starts to feel uncertain. Learn more about how BloomWiki works ?

Time Series Analysis is a statistical technique that deals with time series data, or trend analysis. Time series data is a sequence of data points indexed in time order (e.g., stock prices, weather data, or population growth). Unlike standard statistics, which assumes data points are independent, time series analysis recognizes that "what happened yesterday" often affects "what happens today." By identifying patterns such as trends, seasonality, and cycles, time series analysis allows us to forecast the future, detect anomalies, and understand the dynamic forces that drive a system over time.

Remembering[edit]

  • Time Series — A series of data points indexed (or listed or graphed) in time order.
  • Trend — The long-term increase or decrease in the data (e.g., global warming).
  • Seasonality — Predictable patterns that repeat over a fixed period (e.g., ice cream sales in summer).
  • Cyclical Component — Patterns that repeat but not over a fixed period (e.g., economic "boom and bust").
  • Noise (Irregular) — Random variations that cannot be explained by trend or seasonality.
  • Stationarity — A property where the statistical characteristics of the series (mean, variance) do not change over time.
  • Autocorrelation — The correlation of a signal with a delayed version of itself (how today relates to yesterday).
  • Lag — The amount of time by which one variable is separated from another.
  • Moving Average — A calculation to analyze data points by creating a series of averages of different subsets of the full data set.
  • Forecasting — The process of making predictions of the future based on past and present data.
  • ARIMA (AutoRegressive Integrated Moving Average) — A popular statistical model for time series forecasting.
  • Smoothing — A technique to remove "noise" from a data set to see the underlying pattern.
  • Exponential Smoothing — A technique that gives more "weight" to recent data points when forecasting.
  • Seasonally Adjusted — Data that has had the seasonal component removed to reveal the true trend.

Understanding[edit]

Time series analysis is about "Decomposing" a complex signal into its parts.

The Decomposition Equation: $Data = Trend + Seasonality + Cycle + Noise$ By pulling these apart, we can see that a 10% rise in sales might just be "Seasonality" (it's Christmas), not a "Trend" (your business is growing).

Stationarity (The Golden Rule): Most statistical models only work if the data is "Stationary." If your data has a trend (it's going up), it's not stationary. To fix this, we use Differencing—looking at the change from day to day ($Xt - X{t-1}$) instead of the raw values.

Autoregression (AR): This is the idea that the current value is a linear function of previous values. If it rained yesterday, it is more likely to rain today. The strength of this relationship is measured by Autocorrelation.

Applying[edit]

Calculating a Simple Moving Average (SMA): <syntaxhighlight lang="python"> def simple_moving_average(data, window_size): 

   """
   Smooths out a time series by averaging over a window.
   """
   averages = []
   for i in range(len(data) - window_size + 1):
       window = data[i : i + window_size]
       averages.append(sum(window) / window_size)
   return averages
  1. Stock prices over 10 days

prices = [100, 102, 101, 105, 110, 108, 115, 120, 118, 125] sma_3 = simple_moving_average(prices, 3)

print(f"Original Prices: {prices}") print(f"3-Day SMA: {sma_3}")

  1. Note how the 'zig-zags' are smoothed out, making the upward
  2. trend easier to see.

</syntaxhighlight>

Time Series in the Real World
Stock Market → Analyzing price "momentum" and volatility.
Epidemiology → Tracking the "R-number" (spread of a virus) over time.
IoT / Sensors → Detecting an "Anomaly" (e.g., a machine vibrating weirdly) in a constant stream of data.
Inventory Management → Predicting how many turkeys to stock before Thanksgiving based on the last 5 years of data.

Analyzing[edit]

Additive vs. Multiplicative Seasonality
Type Description Example
Additive Seasonal variation is constant (±10 units) Monthly gym attendance
Multiplicative Seasonal variation grows with the trend (±10%) Airline passengers (more people travel in summer as the total industry grows)

The Random Walk: Some time series (like the stock market in the short term) are "Random Walks." This means the best prediction for tomorrow is simply "whatever it is today" plus some random noise. In a true random walk, technical analysis (looking at charts) is impossible—a hard lesson for many traders.

Evaluating[edit]

Evaluating a forecast:

  1. RMSE (Root Mean Square Error): How far off were our predictions on average?
  2. Overfitting: Does the model work perfectly on old data but fail on new data?
  3. Lag: Does the model "lag" behind reality (only predicting a crash after it starts)?
  4. White Noise: Are the "residuals" (the errors) random? If the errors have a pattern, it means our model is missing something.

Creating[edit]

Future Frontiers:

  1. LSTM (Long Short-Term Memory): Using deep learning (RNNs) to capture extremely long-term dependencies in data (e.g., weather patterns over decades).
  2. Real-time Stream Processing: Analyzing time series as they happen at a rate of millions of points per second (used in fraud detection).
  3. Multivariate Time Series: Predicting one variable (e.g., sales) using 50 others (e.g., weather, social media sentiment, competitor prices).
  4. Causal Time Series: Determining if one time series causes another (Granger Causality).
Retrieved from "http://bloomwiki.org/index.php?title=Time_Series&oldid=5033"