Information on Contingency Tables

The verification of the seasonal forecast in category is done using 3x3 contingency tables. The forecast and observed categories are simply classified in a table of 3 rows and 3 columns (see figure 1 below). There is a row for each observed category and a column for each forecast category (above, near and below normal). For each stations or grid points over Canada, 1 is added to the grid element of the contingency table for each event according to the intersection of the forecast category and the observed category (Stanski et al., 1989).

This table describes the types of scores used in the contingency tables
CONTINGENCY TABLE FORECASTS TOTAL
BELOW NORMAL ABOVE
OBSERVATIONS BELOW A B C D
NORMAL E F G H
ABOVE I J K L
TOTAL M N O P

Many scores can be calculate from the tables. The percent correct is the sum of the diagonal elements over the total elements (multiply by 100) and it is used as the standard score for the long-range forecasts at CMC (see percent correct maps of the models). You will find below a list of the scores shown on the web page menu Verification of Past Forecast. For a detailed explanation of each score please refer to World Meteorological Organization document of Stanski et al (1989).

Scores from the contingency tables:

Scores from the contingency tables
SYMBOL DESCRIPTION COMPUTATION
PC PERCENT CORRECT 100*(A+F+K)/P
C1 FOR HSS1 0.4*(M+O)+0.2*N
C2 FOR HSS2 (D*M+H*N+L*O)/P
HSS1 HEIDKE SKILL SCORE ((A+F+K)-C1)/(P-C1)
HSS2 HEIDKE SKILL SCORE ((A+F+K)-C2)/(P-C2)
BUSTC PROPORTION OF ERRORS OF TWO CATEGORIES - COLD I/M
BUSTW PROPORTION OF ERRORS OF TWO CATEGORIES - WARM C/O
BUSTT PROPORTION OF ERRORS OF TWO CATEGORIES - TOTAL (C+I)/(M+O)
BIASC BIAS - COLD M/D
BIASN BIAS - NORMAL N/H
BIASW BIAS - WARM O/L
PODC PROBABILITY OF DETECTION OF A COLD EVENT A/D
PODN PROBABILITY OF DETECTION OF A NORMAL EVENT F/H
PODW PROBABILITY OF DETECTION OF A WARM EVENT K/L
FARC FALSE ALARM RATIO OF COLD FORECASTS (E+I)/M
FARN FALSE ALARM RATIO OF NORMAL FORECASTS (B+J)/N
FARW FALSE ALARM RATIO OF WARM FORECASTS (C+G)/O
RELC RELIABILITY OF COLD FORECASTS A/M = (1 - FARC)
RELN RELIABILITY OF NORMAL FORECASTS F/N = (1 - FARN)
RELW RELIABILITY OF WARM FORECASTS K/O = (1 - FARW)
CSIC CRITICAL SUCCESS INDEX - COLD A/(D+M-A)
CSIN CRITICAL SUCCESS INDEX - NORMAL F/(H+N-F)
CSIW CRITICAL SUCCESS INDEX - WARM K/(L+O-K)

Reference

Stanski, H. R., L. J. Wilson, and W. R. Burrows, 1989: Survey of common verification methods in meteorology. WWW Tech. Rep. 8, WMO/TD 358, 114 pp.