Metrics
Brier Skill Score
CRPSS
Reliability plot
Relative Economic Value
Relative Mean Error
Relative Operating Characteristic score
References
References
Verification software
Miscellaneous
LateX users
Metrics
Brier Skill Score
CRPSS
Reliability plot
Relative Economic Value
Relative Mean Error
Relative Operating Characteristic score
References
References
Verification software
Miscellaneous
LateX users
The (half) Brier score (BS, [brier_verification_1950]) measures the mean square error of $J$ predicted probabilities that $Q$ exceeds $q$,
\begin{equation} \mathrm{BS} = \frac{1}{J} \sum \limits_{j=1}^{J} { \left\lbrace F_{Q_j} \left( q \right) -F_{S_j} \left( q \right) \right\rbrace }^2 \textrm{,} \end{equation}
where
\begin{equation} F_{S_j} \left( q \right) = \mathrm{Pr} \left[ Q_j > q \right] \nonumber \end{equation}
and
\begin{equation}
F_{Q_j} \left( q \right) = \left\lbrace
\begin{array}{ll} 1 & \mbox{if $Q_j > q$}
0 & \mbox{otherwise} \end{array}
\right\rbrace \nonumber
\end{equation}
The Brier Skill Score (BSS) is a scaled representation of forecast quality that relates the quality of a particular system $\mathrm{BS}$ to that of a perfect system $\mathrm{BS_{perfect}}$ (which is equal to 0) and to a reference system $\mathrm{BS_{ref}}$,
\begin{eqnarray} \mathrm{BSS} &=& \frac{ \mathrm{BS} - \mathrm{BS}_\mathrm{ref} }{ \mathrm{BS}_\mathrm{perfect} - \mathrm{BS}_\mathrm{ref} } \\ &=& \frac{ \mathrm{BS} - \mathrm{BS}_\mathrm{ref} }{ 0 - \mathrm{BS}_\mathrm{ref} } \nonumber \\ &=& \frac{ \mathrm{BS}_\mathrm{ref} - \mathrm{BS} }{ \mathrm{BS}_\mathrm{ref} } \nonumber \\ &=& 1 - \frac{ \mathrm{BS} }{ \mathrm{BS}_\mathrm{ref} } \nonumber \end{eqnarray}
BSS ranges from $-\infty$ to 1. The highest possible value is 1. If $\mathrm{BSS}=0$, the BS is as good as that of the reference system. If $\mathrm{BSS}<0$ then the system's Brier score is less than that of the reference system.