A Flaw in the NIRF Rankings – and a Fix
The Ministry of Human Resource Development released the fifth edition of the National Institutional Ranking Framework on June 11. Since 2017, when the first edition was introduced, engineering institutions have dominated the list. This year, seven of the top 10 places are occupied by the IITs. And in all five instalments, IIT Madras has never been dislodged from the top spot in the engineering rankings. Strangely, the other international ranking exercises, the Shanghai ARWU, the Times Higher Education and the Quacquarelli Symonds rankings, have rarely given IIT-M the topIIT Kharagpur and IIT Delhi now rank ahead of IIT Madras. Note the presence of Jadavpur University ahead of IIT Kanpur, and the presence of Vellore Institute of Technology ahead of IIT (Indian School of Mines) Dhanbad and IIT Guwahati. This is because the multiplied-score procedure recognises the fact that Jadavpur University and Vellore Institute of Technology leverage lower teaching and learning resources to produce relatively higher outcomes (research and graduation) than many privileged IITs. The NIRF score in its current form is unable to make this distinction.
However, the ranking’s methods have a serious flaw (apart from the other well-known flaws that visit all ranking exercises). It builds a single score from five categories: teaching, learning and resources (TLR), research and professional practices (RPP), graduation outcomes (GO), outreach and inclusivity, and perception. These five broad heads are built up from various sub-heads, and a complex weighting and addition scheme is used to obtain the overall rating score, which can take a maximum value of 100. The institutions are finally rank-ordered based on these scores.If a ranking is really required, we can obtain a score obtained by multiplying the quality score and the output score.
The flaw is that in a university system, TLR is technically ‘input’ and RPP and GO are ‘output’, and the NIRF adds the input and output scores to obtain the final score. This violates the basic principle of performance analyses: that performance is based on the input score and that quality is based on the ratio of output to the input. (Note that outreach and inclusivity and perception relate neither to academic nor research excellence but these are added as well.)
What happens when the NIRF scores are recomputed without adding the input and output? Let’s use an alternative two-dimensional paradigm, where the input is the ‘teaching, learning and resources’ (TLR) score, and the output is the sum of the ‘research and professional practices’ (RPP) and the ‘graduation outcomes’ (GO) scores. First, I normalise the values using the totals for RPP, GO and TLR for the top 100 engineering institutions in the NIRF 2020 list, then calculate the performance and quality scores.
Gangan Prathap is an aeronautical engineer and former scientist at the National Aeronautical Laboratory, Bangalore and former VC of Cochin University of Science and Technology. He is currently a professor at the A.P.J. Abdul Kalam Technological University, Thiruvananthapuram.
Source: The Wire: 14/06/2020
