The Solar Forecast Arbiter: An Open Source Evaluation Framework for Solar Forecasting
Clifford W. Hansen1, William F. Holmgren2, Aidan Tuohy3, Justin Sharp4, Antonio T. Lorenzo2, Leland J. Boeman2, Anastasios Golnas5
1Sandia National Laboratories, Albuquerque, NM, United States
/2University of Arizona, Tucson, AZ, United States
/3Electric Power Research Institute, Chicago, IL, United States
/4Sharply Focused, Portland, OR, United States
/5U.S. Department of Energy, Washington, DC, United States

We describe an open source evaluation framework for solar forecasting to support the DOE Solar Forecasting 2 program and the broader solar forecast community. The framework enables evaluations of solar irradiance, solar power, and net-load forecasts that are impartial, repeatable and auditable. First, we define the use cases of the framework. The use cases, developed from the project’s initial stakeholder engagement sessions, include comparisons to reference data sets, private forecast trials, evaluation of probabilistic forecast skill, and examinations of forecast errors during critical periods. We discuss the framework’s data validation toolkit, reference data sources, and data privacy protocols. We describe the framework’s benchmark forecast capabilities for intra-hour and day ahead forecast horizons. Finally, we summarize the reports and metrics that communicate the relative merits of the test and benchmark forecasts. The reports are created from standardized templates and include graphics for quantitatively evaluating deterministic and probabilistic forecasts and standard metrics for quantitatively evaluating forecasts.