The relative operating characteristic (ROC; [green_signal_1966]) plots the hit rate versus the false alarm rate. These are calculated using the elements of a contingency table, which is valid for a single probabilistic decision rule,

Contingency table. The consequences of the items listed are in brackets. Taken from [verkade_estimating_2011].

and are defined as follows

\begin{align} \textrm{hit rate} & = \frac{\textrm{# hits}}{\textrm{# observed events}} = \frac{h}{o}\\ \textrm{false alarm rate} & = \frac{\textrm{# false alarms}}{\textrm{# events not observed}} = \frac{f}{o'} \textrm{,} \nonumber \end{align}

The ROC score measures the area under the ROC curve (AUC) after adjusting for randomness, i.e.

\begin{equation} \mathrm{ROCS} = 2 \times \left( \mathrm{AUC} - 0.5 \right) \textrm{.} \end{equation}