User Tools

Site Tools


Relative Operating Characteristic score

The relative operating characteristic (ROC; [green_signal_1966]) plots the hit rate versus the false alarm rate. These are calculated using the elements of a contingency table, which is valid for a single probabilistic decision rule,

Event observed Event NOT observed $\sum$
Warning issued hits $h \left( C + L_u \right)$ false alarms $f \left( C \right) $ $w$
Warning NOT issued missed events $m \left( L_a + L_u \right)$ quiets/correct negatives $q \left( - \right) $ $w'$
$\sum$ $o$ $o'$ $N$

Contingency table. The consequences of the items listed are in brackets. Taken from [verkade_estimating_2011].

and are defined as follows

\begin{align} \textrm{hit rate} & = \frac{\textrm{# hits}}{\textrm{# observed events}} = \frac{h}{o}\\ \textrm{false alarm rate} & = \frac{\textrm{# false alarms}}{\textrm{# events not observed}} = \frac{f}{o'} \textrm{,} \nonumber \end{align}

The ROC score measures the area under the ROC curve (AUC) after adjusting for randomness, i.e.

\begin{equation} \mathrm{ROCS} = 2 \times \left( \mathrm{AUC} - 0.5 \right) \textrm{.} \end{equation}

metrics/rocs.txt · Last modified: 2020/02/02 06:30 (external edit)