BACKGROUND: Choosing to replace or maintain an existing cancer staging system is a difficult task. The system plays a critical role in patient counselling and treatment decision making because the staging system conveys prognosis.

METHODS: Many issues may be considered when deciding the preferred system (i.e. old or new), such as the level of evidence for one or more factors included in the system or the general opinions of expert clinicians. However, given the major objective of estimating prognosis on an ordinal scale, we argue that the rival staging system candidates should be compared on their ability to predict outcome. We sought to outline an algorithm that would compare two rival ordinal systems on their predictive ability.

RESULTS: We devised an algorithm based largely on the concordance index, which is appropriate for comparing two models in their ability to rank observations. We demonstrate our algorithm with a prostate cancer staging system example.

CONCLUSION: We have provided an algorithm for selecting the preferred staging system based on prognostic accuracy. It appears to be useful for the purpose of selecting between two ordinal prediction models.