Metrics

Common evaluation metrics for time series forecasts.

metrics.corr(data_samples: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec748c0198>, data_truth: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec748c0160>, agg=None, **kwargs)

Returns the Pearson correlation coefficient between observed values and aggregated predictions.

Parameters:
  • data_samples – Predicted time series values (n_timesteps, n_timeseries).
  • data_truth – Actual values observed.
  • agg – Property of the forecast distribution to use for evaluation.
metrics.coverage(data_samples: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747d0780>, data_truth: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747d0cf8>, percentiles=None, **kwargs)

Computes coverage rates of the prediction interval.

Parameters:
  • data_samples – Predicted time series values (n_timesteps, n_timeseries).
  • data_truth – Actual values observed.
  • percentiles – Percentiles to compute coverage for.
metrics.mae(data_samples: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec7485a048>, data_truth: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec7485a470>, agg=None, **kwargs)

Computes mean absolute error (MAE)

Parameters:
  • data_samples – Predicted time series values (n_timesteps, n_timeseries).
  • data_truth – Actual values observed.
  • agg – Property of the forecast distribution to use for evaluation.
metrics.mape(data_samples: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec7485a4a8>, data_truth: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747fb6a0>, agg=None, **kwargs)

Computes mean absolute percentage error (MAPE)

Parameters:
  • data_samples – Predicted time series values (n_timesteps, n_timeseries).
  • data_truth – Actual values observed.
  • agg – Property of the forecast distribution to use for evaluation.
metrics.mase(data_samples: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747fb668>, data_truth: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747fb438>, data_insample, frequencies, agg=None, **kwargs)

Computes mean absolute scaled error (MASE)

Parameters:
  • data_samples – Predicted time series values (n_timesteps, n_timeseries).
  • data_truth – Actual values observed.
  • data_insample – Insample time series values.
  • frequencies – Frequencies used for seasonal naive forecast.
  • agg – Property of the forecast distribution to use for evaluation.
metrics.mse(data_samples: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747fb5c0>, data_truth: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747fb630>, agg=None, **kwargs)

Computes mean squared error (MSE)

Parameters:
  • data_samples – Predicted time series values (n_timesteps, n_timeseries).
  • data_truth – Actual values observed.
  • agg – Property of the forecast distribution to use for evaluation.
metrics.pinball_loss(data_samples: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec749be320>, data_truth: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec748ed978>, percentiles=None, **kwargs)

Computes pinball loss of a quantile \(\tau\) given the actual value \(y\) and the predicted quantile \(z\).

\[\begin{split}L_{\tau} ( y , z ) = \begin{cases} & ( y - z ) \tau &\mbox{if } y \geq z \\ & ( z - y ) ( 1 - \tau ) &\mbox{if } z > y \end{cases}\end{split}\]
Parameters:
  • data_samples – Predicted time series values (n_timesteps, n_timeseries).
  • data_truth – Actual values observed.
  • percentiles – Percentiles to compute coverage for.
metrics.rmse(data_samples: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747d0748>, data_truth: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747d07f0>, agg=None, **kwargs)

Computes root-mean squared error (RMSE)

Parameters:
  • data_samples – Predicted time series values (n_timesteps, n_timeseries).
  • data_truth – Actual values observed.
  • agg – Property of the forecast distribution to use for evaluation.
metrics.rse(data_samples: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747d0eb8>, data_truth: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747d0860>, agg=None, **kwargs)

Computes root relative squared error (RSE)

Parameters:
  • data_samples – Predicted time series values (n_timesteps, n_timeseries).
  • data_truth – Actual values observed.
  • agg – Property of the forecast distribution to use for evaluation.
metrics.smape(data_samples: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747fb3c8>, data_truth: <sphinx.ext.autodoc.importer._MockObject object at 0x7fec747fb710>, agg=None, **kwargs)

Computes symmetric mean absolute percentage error (SMAPE) on the mean

Parameters:
  • data_samples – Predicted time series values (n_timesteps, n_timeseries).
  • data_truth – Actual values observed.
  • agg – Property of the forecast distribution to use for evaluation.