University rankings don't tell the whole story

TALKBACK:GLOBAL RANKINGS of universities have gained new currency in Ireland in recent weeks, appearing to influence government policy on funding in higher education. Support for the current Review of Higher Education and the establishment of the TCD/UCD innovation alliance noted that they "are the only Irish universities ranked inside the world top 200" (The Irish Times, January 30th and March 12th 2009). This preoccupation with the relative standing of our universities reflects the consensus that higher education is the engine of economic growth. But are global rankings the most appropriate benchmark for shaping policy? asks ELLEN HAZELKORN

Less than a decade ago, few people outside of the US had heard of university rankings. Today, national rankings exist in more than 40 countries, and global rankings are mushrooming. The first, the Shanghai Jiao Tong Academic Ranking of World Universities (SJT), began in 2003. It was followed by Webometrics and Times QS World University Ranking in 2004 and the Taiwan Performance Ranking of Scientific Papers for Research Universities in 2007. The EU has recently announced its intention to develop a “new multi-dimensional university-ranking system with global outreach” to be piloted in 2010.

Rankings compare higher education institutions using a range of indicators, which are weighted differently by each ranking system. For example, the SJT focuses on research, using international indicators such as peer-reviewed publications, citations and Nobel prizes, while the Times QS balances research looks at peer review, student/staff ratio and internationalisation and employer feedback.

The main emphasis of these rankings is on the bio-sciences, with limited accuracy for social science, while the humanities and arts are largely ignored. Despite such global challenges as climate change, energy, healthcare delivery and urban infrastructure, rankings have difficulty measuring interdisciplinary or collaborative research. They do not recognise research which leads to new products and services, high-performance start-ups or social innovation, nor measure the quality of teaching or the impact of research on teaching.

To paraphrase, Einstein, global rankings focus on what is easily measured rather than measuring what counts.

Around the world – and now in Ireland – governments are using rankings as a benchmark for national or institutional ambition. International evidence shows that educational and research priorities, resource allocation and student-entry requirements, to name a few, are being altered in order to effect better rankings. While rankings are focusing more attention on performance and quality, they are also distorting higher education and encouraging many to “game” the system.

Despite the different methodologies, the same 25 universities tend to appear at the top in both rankings. They are distinguished by large budgets, large endowments, excellent staff to student ratios, and most importantly, access to large pools of highly developed human capital (staff and students). Access to the top 25, for the moment, is beyond most nations. Estimates vary, but a “world class” university generally has an annual budget of about €1.5 billion. For Ireland to develop a university on this scale would require diverting the entire higher education budget to a single institution.

Because size matters, governments are struggling with how best to respond. It is easier to have well-performing universities in a large country than a small one. Rankings are not controlled for population size or GDP; if they were Ireland would rank 7th in the world on the Times QS according to population size and 10th for GDP, while the US would be 14th.

Interestingly, the Lisbon Council ranking of university systems (2008) compares how the system as a whole helps to meet the economic and social goals vital to success. Using that comparison, Ireland ranks 7th out of the 17 countries surveyed, just behind the US and Swede, but ahead of Germany and Switzerland.

Australia presents an alternative strategy which is worth studying, not least because our government is considering copying its fees and loans system (HECS). Rather than concentrating resources in a few universities, Australia wants a world-class system providing excellence across diverse fields of learning and discovery, and recognising and rewarding excellence wherever it occurs. The Norwegian Commission for Higher Education, reporting in January 2008, adopted a similar stance.

How should Ireland respond?

Policy makers and the public want value for money, especially in difficult economic circumstances. Rankings appear to provide a simple way to gauge world-class status. But, measuring the wrong things can produce distortions. Using global rankings as the benchmark only makes sense if the indicators are appropriate – otherwise, we risk transforming our higher education system to conform to metrics designed for other purposes.

Instead, Ireland would be wise to develop benchmarks which reflect our societal and innovation needs and assess performance accordingly as well as bring RD investment up to a minimum of 3 per cent of GDP, in line with EU guidelines, so we can compete globally, and support and fund excellence wherever it occurs because it is the total investment that is important.

In this way, we can mobilise the potential of the whole higher education system and maximise the benefits to society at large.

Prof Ellen Hazelkorn is Director of Research and Enterprise at Dublin Institute of Technology and leads the Higher Education Policy Research Unit (HEPRU)