User Tools

Site Tools


metrics:bss

Brier Skill Score

The (half) Brier score (BS, [brier_verification_1950]) measures the mean square error of $J$ predicted probabilities that $Q$ exceeds $q$,

\begin{equation} \mathrm{BS} = \frac{1}{J} \sum \limits_{j=1}^{J} { \left\lbrace F_{Q_j} \left( q \right) -F_{S_j} \left( q \right) \right\rbrace }^2 \textrm{,} \end{equation}

where

\begin{equation} F_{S_j} \left( q \right) = \mathrm{Pr} \left[ Q_j > q \right] \nonumber \end{equation}

and \begin{equation} F_{Q_j} \left( q \right) = \left\lbrace \begin{array}{ll} 1 & \mbox{if $Q_j > q$}
0 & \mbox{otherwise} \end{array} \right\rbrace \nonumber \end{equation}

The Brier Skill Score (BSS) is a scaled representation of forecast quality that relates the quality of a particular system $\mathrm{BS}$ to that of a perfect system $\mathrm{BS_{perfect}}$ (which is equal to 0) and to a reference system $\mathrm{BS_{ref}}$,

\begin{eqnarray} \mathrm{BSS} &=& \frac{ \mathrm{BS} - \mathrm{BS}_\mathrm{ref} }{ \mathrm{BS}_\mathrm{perfect} - \mathrm{BS}_\mathrm{ref} } \\ &=& \frac{ \mathrm{BS} - \mathrm{BS}_\mathrm{ref} }{ 0 - \mathrm{BS}_\mathrm{ref} } \nonumber \\ &=& \frac{ \mathrm{BS}_\mathrm{ref} - \mathrm{BS} }{ \mathrm{BS}_\mathrm{ref} } \nonumber \\ &=& 1 - \frac{ \mathrm{BS} }{ \mathrm{BS}_\mathrm{ref} } \nonumber \end{eqnarray}

BSS ranges from $-\infty$ to 1. The highest possible value is 1. If $\mathrm{BSS}=0$, the BS is as good as that of the reference system. If $\mathrm{BSS}<0$ then the system's Brier score is less than that of the reference system.

metrics/bss.txt · Last modified: 2020/02/02 06:30 (external edit)