World University Rankings: Statistical Issues and Possible Remedies

World university ranking started one and a half decades ago for the purpose of understanding what makes an excellent institution of higher education. Subsequent to the appearance of the Academic Ranking of World Universities at the Shanghai Jiaotong University, there soon emerged the QS World University Rankings and the Times Higher Education World University Rankings. These three ranking systems are considered the classics as they are the fore-runners, although no less than ten new systems have come to the arena.

The various ranking systems adopt a common approach of weight-and-sum to process the indicator data. Each system, somewhat arbitrarily, decides on a set of indicators and assigns different weights to these, presumably reflecting their relative importance. This simple (and simplistic) approach meets well common sense. And, in fact, much of the discussion on world university rankings is conducted at the commonsensical level.

However, analyses conducted in the recent years uncovered several problems of the prevalent approach: spurious precision, mutual compensation, weight discrepancy, indicator redundancy, etc., which render the overall scores and ranking suspect in terms of validity. These are due to systems ignoring the fact that world university rankings are a form of social measurement and therefore need be seen from this perspective.

Moreover, rankings encourage competition and, in the highly competitive world of today, it is natural that institutional attention is focused on the ranking results. By now, the original purpose of world university ranking seems to have been overshadowed, and world university rankings look more like international academic contests, as though they are annual sports meets.

This monograph collects together many articles pertaining to the identified measurement and statistical issues of world university rankings and suggests remedies to make ranking results more trustworthy.