Data & methods
The machine learning, statistics and data engineering behind GridCast: the tools, how the data flows, how look-ahead leakage is prevented, and what the limitations are.
Skills & tools
Machine learning & regression
Gradient-boosted quantile regression with LightGBM; LASSO-regularised linear regression with scikit-learn (the LEAR benchmark); neural forecasters N-HiTS, PatchTST and TFT in PyTorch via neuralforecast, currently in validation.
Statistics & time series
SARIMAX with statsmodels; Diebold–Mariano and Giacomini–White tests with Holm correction; model confidence sets with arch; Chow and Kolmogorov–Smirnov tests, Hill tail-index estimates and change-point detection (ruptures, SciPy) for the market-redesign question.
Probabilistic prediction & evaluation
21-quantile forecasts; conformal prediction (CQR) for calibrated ranges; CRPS, pinball loss, mean absolute error and interval coverage; rolling-origin backtesting with scheduled refits; Optuna hyperparameter search on validation data only.
Data analysis & engineering
pandas and NumPy; DuckDB SQL over Apache Parquet; schema validation with Pandera; XML and CSV parsing with lxml; rate-limited API clients for IESO, Open-Meteo and Grid Status (httpx, gridstatusio); automated data-quality reports.
Research design
Questions, data splits and decision rules pre-registered before modelling; a test window sealed in code; placebo-calibrated break tests; look-ahead leakage tests with pytest and Hypothesis.
Engineering & deployment
Python 3.12 with uv, Ruff, mypy (strict) and pre-commit; GitHub Actions CI; an Azure Linux server running the hourly collector and the 9:00 forecast on systemd timers; this site in plain JavaScript with hand-built SVG charts on Cloudflare Workers.
Pipeline
No peeking at the future
- Every value carries the time it became public. A forecast issued at 9:00 can only use rows published by 9:00, the single gate all inputs pass through.
- IESO timestamps are interpreted on its own clock (Eastern Standard Time all year), verified against the server's file times.
- Weather forecasts count only from 8 hours after their model run, so a forecast can never use weather information it wouldn't have had.
- Automated tests (pytest, Hypothesis), run on every code change: a randomised property-based check against brute-force lookups, a "canary" value published one second late that must never appear, the weather rule, and identical results on re-runs.
- Test-period results are sealed in code until model settings are frozen and committed.
Limitations
- The old market's price (HOEP) and the new zonal price are different quantities; comparisons across the redesign use the closest comparable series.
- Operator forecasts (demand, wind, solar outlooks) only exist from the start of live collection, so most model training relies on weather and lagged data.
- Battery results assume a price-taker that doesn't move prices, which is unrealistic for very large batteries.
- About one year of post-redesign test data limits statistical power; the live scorecard extends it over time.
Sources & attribution
- IESO public reports: prices, demand, generation, interties, operator forecasts. Copyright © 2004-2022 Independent Electricity System Operator, all rights reserved. This information is subject to the Terms of Use set out in the IESO's website (www.ieso.ca).
- Grid Status: day-ahead price history from May 2025 to mid-July 2026, where IESO's own public archive begins (used for training only; not republished here). Data provided by Grid Status.
- Open-Meteo: GFS weather forecasts as issued (Previous Runs API), CC BY 4.0.