Skip to content

Grade inflation: the fire that will not be quenched

June 13, 2016

gradeinfl_fire

 

Grade inflation is a fire that simply will not be quenched, at least in so far as the Institutes of Technology are concerned, according to the latest data collected and analyzed in Paper 11 published by the Network for Irish Educational Standards.

Normally when you choke off the fuel supply, the fire dies down. Not so with grade inflation. Despite funding for the sector as a whole being reduced by 35% between 2008 and 2015, the percentage of Firsts and Distinctions has just kept on growing.

The latest round of number crunching shows a 13.5% growth in Firsts in Honours Degrees when the average of 2012 and 2013 is compared with the average of 2007 and 2008.

By the same yardstick, the rate of Distinctions has increased by 18.7% in Ordinary Degrees and  20.8% in Higher Certificates. A remarkable development is that the rate of First Class honours degrees across the 13 Institutes of Technology (DIT not included) at 18.5% averaged over 2012-2013 exceeds that of the universities’ rate of 15%. The rate at DIT was 17% for the same years.

The fires of learning in the Institutes of Technology would seem to need no oxygen.

Dream or Nightmare?

Does anyone anywhere find all that just a tad suspicious?  Much less funding, much better results. A dream come true for the penny pinching bureaucrats at the Higher Education Authority. The fires of learning in the Institutes of Technology would seem to need no oxygen.

Surely a nightmare, however, for the Teachers Union of Ireland which has been loudly protesting about the financial squeeze and even going on strike to draw attention to the parlous state of the cash strapped sector. But what’s the fuss about? Why should any government in its right mind plough extra cash into such a hyper performing system that can do so much more with so much less?

It’s difficult to quite grasp the scale of the achievement wrought in the Institutes of Technology. Think about this for a moment. The typical CAO points which accompanied the average student going through the doors of any of the 13 Institutes in 2014 was 341. The typical entrant to the university sector had 462 points.

This is not something which varies much year on year so what was true of 2014 is true for any other year you like to pick. Students who gain access to the universities are academically much stronger than those who enter the Institutes of Technology. Even if we limit our scope to only those students entering Honours Degree courses at the IOTs, we find that in 2014 the typical CAO points tally of such a student was 357, a very long way down from 462.  Yet, the Institutes of Technology managed to produce more First Class degree performances.

gardeinfl_fire2

 

Alchemy

This alchemy is not restricted to Honours Degrees. It is equally evident in the growth of Distinctions in Ordinary Degrees and Higher Certificates which showed increases over the same period of 19% and 21% respectively. Nor is it only the top grade in each case which keeps growing more and more common. The same is true in each qualification level for the next grade down. Merit 1 awards have grown by 22% in Higher Certificates and 7% in Ordinary Degrees when the averages of the 2012 and 2013 figures are compared with those of 2007 and 2008. Upper seconds or 2.1 awards at Honours Degree level have increased by 5%.

All this would be remarkable enough if the 2007-2008 baseline figures were not already elevated over what they used to be. Look at the three graphs showing the top grades at each of the main undergraduate qualifications awarded by the Institutes of Technology since 1994. There has been some choppiness along the way but the trend has been inexorably upwards.

O’Grady and Guilfoyle (2016) investigated if the rate of increase in the top grades has been speeding up, slowing down or remaining constant. They concluded that comparing the last five years of the series with the first fifteen, the rate of increase has accelerated for Higher Certificates and Ordinary Degrees but slowed down in Honours Degrees.

This slowing down, however, may be an artefact of how they calculated the rates of increase and chose the baseline years for comparison. For unknown reasons the linear climb was interrupted by a choppy period between 2005 and 2010 but thereafter, as evident in the graph, the clearly linear slope resumed even more steeply than before.

Distinctions Certificates

The latest report by O’Grady and Guilfoyle draws attention to another development in the sector, a considerable upward shift in the level of educational qualification at which students typically graduate. Between 1994 and 2013, there has been a sharp decline from 53.5% to only 13% in the proportion of all Institute of Technology graduates at NQA Level 6 (Higher Certificate).

This has been paralleled by a major increase from 13% to 47.2% over the same period in the proportion of the total accounted for by NQA Level 8 (Honours Degree) graduates. The modal qualification has shifted from Level 6 to Level 8. This is a major transformation.

The Miraculous Transformation

There is (or at least there should be) a great deal of difference between the academic standard required by a 4 year Honours Degree and that required by a two year Certificate which renders the graphs for First Classes and Distinctions shown here all the more incomprehensible.

Surely if the better students who formerly graduated with a Certificate have increasingly been setting their sights on an Honours Degree, the effect on top grades ought to have been the reverse of what has happened. The relatively stronger students, who previously would have obtained Distinctions at Level 6, will more likely have opted to enter ab initio level 8 courses instead thereby reducing the proportion able to win Distinctions at Level 6 but are unlikely to be so strong as to be able to swell the ranks of Firsts in the academically much more demanding Honours Degree courses. The effect should have been to draw down the top grades at both levels.

Of course, if there had been a dramatic and ongoing improvement across the Institute of Technology sector in the quality of students attracted, then the numbers getting the top grades would be expected to increase. The improvement in the quality of students would have to be at all three qualification levels. It would also have to have been continuous right across the two decades involved and right across the country.

There is nothing at all to suggest that the Institutes of Technology have been managing to attract more academically capable students as the years passed.

That means the average student arriving in the IOTs would need to have become academically more capable with each passing year and the improvement overall would have to be very considerable to account for a rate of 8.2% Firsts becoming a rate of 19%.  Since the majority of students each year are admitted through the CAO system and this is true of both school leavers and more mature students, any such improvement should be immediately evident in CAO points.

gradeinfl_fire4

A quick glance at the graph above makes it abundantly clear that there is nothing at all to suggest that the Institutes of Technology have been managing to attract more academically capable students as the years passed. If they had been, then the minimum points for access to courses would have climbed quite a lot since the early 1990s. Each point on the graph shows the result for that year of adding together the minimum points at which students entered each Level 6, 7 and 8 course across the sector and dividing by the number of courses involved.

Throughout the 90s when grades at graduation were improving rapidly the Institutes were reducing their points’ requirements from a mean of 284 in 1991 to 206 in 2000. They were admitting weaker students. After 2000 there was a very gradual improvement from 206 in 2001 to 228 in 2008. Then there was a sharp upturn with the mean minimum points finally surpassing the 1991 high in 2011 when the mean reached 300 points.

The Universities, however, were creaming off the pick of the available students, hence the score of 462 points for the typical entrant to the Universities in 2014 as compared with 341 for the Institutes.

The Unquenchable Fire

What the graph fails to show is that all through that period average points were constantly rising due to more and more high grades being awarded in the Leaving Certificate. Crunching the numbers on the CAO website, O’Grady and Guilfoyle estimated that the average CAO applicant in 1995 had around 250 points. By 2005 the average applicant had approximately 350 points. How is it then that the minimum points for entry to the Institutes of Technology were not carried up by this rising tide?

The answer is that while the Institutes were taking in more and more students, at the same time the Universities were doing likewise. The Universities, however, were creaming off the pick of the available students, hence the score of 462 points for the typical entrant to the Universities in 2014 as compared with 341 for the Institutes.

Instead of getting more and more of the stronger students, the Institutes were all the time losing ground to the Universities and having to make do with progressively weaker students. This renders the grade graphs shown here all the more amazing. The unremitting rise in top grades has overwhelmed the countervailing forces of both a talent and a resources drain. It really is an unquenchable fire.

It has been a characteristic of the Network for Irish Educational Standards research to search high and low for anything which might explain the upward march of grades other than the erosion of academic standards. In the latest round of analysis, O’Grady and Guilfoyle explored the possibility that at least some of the rise might be down to more adults coming into the sector and to a change in the mix of part-time and full-time students. They drew a blank on both counts. While there has been a considerable increase in mature students across the Institutes, up to about 20% of entrants, most of them are admitted through the CAO system. Detailed analysis of progression rates from first to second year by the Higher Education Authority has shown that for both mature entrants and school leavers the key predictor is the points’ score on which they enter.

The only remaining argument … is that right across the Institutes of Technology, the academic staff has become privy to some arcane wisdom whereby weaker students can with each passing year be enabled to learn more and better than their predecessors.

This means that adult success in third level is just as much a function of prior educational attainment. That is factored into the CAO points analysis featured above. Furthermore, while there does not appear to be any such comparison for the Institutes of Technology, evidence from the University sector in Ireland and from higher education sectors in the US and the UK does not support the hypothesis that mature students as a group perform better than their school leaver counterparts.

As for the mix of part-time and full-time students, O’Grady and Guilfoyle concluded from the data available that the balance of the two has not changed significantly over time. Any impact on grades will then have remained constant.

Arcane Knowledge

The only remaining argument to counter the grade inflation explanation is that right across the Institutes of Technology, the academic staff has become privy to some arcane wisdom whereby weaker students can with each passing year be enabled to learn more and better than their predecessors.

Whatever this alchemy involves, it is seemingly independent of finance or resources since turning off that tap has had no impact at all. Fewer lecturers dealing with more students on more courses in larger classes with diminished back-up services exert no restraint.

Either you believe in this alchemy or you believe that academic standards have been sliding sharply for a long time in the Institutes of Technology and that the pace of slippage has been increasing not slowing down. For anyone with a modicum of sense it’s not difficult to figure which is true. But what’s the harm? Why should it be a problem that ever more students get better grades?

The unremitting rise in top grades has overwhelmed the countervailing forces of both a talent and a resources drain.

There actually is quite a deal of harm because increasingly there is no way of differentiating the stars from the also rans and no special grade to motivate the stars, but to focus on the number getting the better grades is to miss the point. It is the process whereby better grades are being obtained that is the real problem. It can only be achieved by reducing learning demands across the board. The real reason for grade inflation is not to give more students at the top end better grades.  It is to mask the inability of students at the bottom end to achieve any reasonable standard of learning.

The HEA’s Perverse Incentives

As the Institutes dredge ever deeper into the ultimately limited pool of available talent and at the same time stream students ever more into what should be much more demanding degree courses, they face two choices. Keep dumbing down or live with growing failure rates. The Institutes are paid on a headage basis by the Higher Education Authority so it is obvious which option they are likely to choose.

 

To make matters worse, in recent years the HEA has been requiring as a condition of funding that the Institutes enter into “compacts” which specify targets to be met in a given time frame. If they fail to reach them they may face heavy financial penalties. Among the targets agreed are increases in student progression rates from first to second year. In a sector where grade inflation has run rampant for years, this is a direct and powerful incentive to dumb down.

Thus the HEA directly creates perverse incentives by pitting the maintenance of academic standards against financial well-being. When there is no chance of getting better students and improving real learning is altogether outside the control of the Institutes, what other option is there?

Students will learn more if they are able and willing. If they are not, it is impossible  for anyone to make any difference. Faced with the large-scale reductions in their budgets and with lecturers carrying weekly class loads beyond any higher educational sector in the developed world, the systemic drive is inevitably towards dumbing down.

Why attempt the impossible when the same outcome can be achieved by a long tried and well-tested method? Dumb down. If all over Ireland for two decades, the patterns described in the graphs shown here have been allowed to go unquestioned and unchecked, why stop now? Of course that is the 64 thousand dollar question. Why stop? There are very compelling reasons why grade inflation should be stopped and we will deal with those in our next blog.

 

 

Teachers’ Stance on Marking the Junior Certificate

April 14, 2015

Marking Your Own Students’ Exams

For anyone who wonders what is the problem with having teachers mark their own students in a state examination, a read of teacher Tom Galvin’s experience of grading students in Polish schools, included in his 2007 book There’s an Egg in My Soup … and other adventures of an Irishman in Poland, should prove a a salutary experience. Galvin, later editor of the Polski Herald, the Evening Herald’s Polish supplement, taught English in the 1990s in a Polish second level school. He married a Polish woman and returned to Ireland where he has since written a number of books including the above commentary on his five years living in small town Poland.  What he had to say about being responsible for awarding students their grades is best left to his own words:

“However, at the end of the exam, when we [the teachers] were conferring, I had this particular girl down for  a ‘2’ grade – a borderline pass technically, but regarded by most as a fail.  The teacher bedside me, however, had her down for a ‘4’, which is an honour. When I went to protest, I was given a sharp kick on the ankle under the table and told to shut up, as her mother had made the tea and cakes [supplied to the teachers while they carried out the oral examination process]. Meanwhile, girls who I knew had worked their arses off but may not have performed brilliantly on the day, received a ‘3’ grade, which is a straight pass. It was a total joke. I had several rows about it afterwards, but although the other teachers knew I was right, they felt there was nothing they could do about it. The same problem applied across the board. I had colleagues in other schools who had been visited by distraught parents in the middle of the night with bottles of vodka and money. It was a mess but a mess that had always been there.”

At least in Poland, they have always had the mess. It’s a lame excuse but we all know that change is difficult. Ireland plans on starting the mess at this late stage which is surely inexcusable. Is anyone so blind and gullible as to believe that pressures of this kind will not be brought to bear on teachers in Ireland who will be responsible for grading the work of their neighbours’ and even their work colleagues’ children? The pressure may be more subtle than bottles of vodka but it will be just as real and just as difficult to resist as in Poland.

Formative Exams versus Summative Exams

As for the argument that teachers regularly mark their students’ work when they set house examinations, that is simply not the same thing as a state exam. Every parent knows that. House exams are for formative, not summative, purposes. They are used to prepare students for the state examinations. No parent with even a modicum of sense  wants a teacher to mislead students by giving them inflated grades in house examinations. State examinations are another matter entirely. And, if the real agenda is to eliminate the summative function of  the Junior Certificate examination and relegate it to the position of being a formative assessment in preparation for the Leaving Certificate exam, then let’s be honest about that and see if the population is happy with such a plan. Let’s have the debate and, if it is agreed that the Junior Certificate should be scrapped, I don’t imagine teachers will object to setting house exams for their students in its place, as they do in all other “non examination” years. But, let’s not mix up summative and formative assessment. They are different beasts and need very different handling.

Nidge’s Child’s Exam

Meanwhile, as  we are still envisaging teachers grading their own students in a state examination, let’s hear a little more from Tom Galvin on his experience of performing such a task. Writing about the Roma in Poland or “Gypsies” as the Poles call them, he related this tale:

One of these Gypsies wanted to do me in once, the boyfriend of a student of mine. She was a nice wee girl who then turned into a right little cow, so I gave her bad grades. He wanted revenge for that. Somebody, somewhere, intervened on my behalf, telling him that if anything happened to me, he was dead. For a while though it looked as if I was in trouble. Stones at my window and long stares in the pub. It upset me for a while and didn’t help my sleeping patterns.”

As has been asked rhetorically before, which teacher would like to be grading Nidge’s son’s or daughter’s Junior Cert? Bias Wittingly or unwittingly, in both anecdotes, Tom Galvin revealed another all too human risk in having teachers mark their own students – bias. Whether a student is a “right cow” or worked “her arse off” should be immaterial when performance is judged and grades  being awarded. Only the performance should count but teachers are human and humans are all too fallible. Once they know who they are grading, the temptation to reward what they see as virtue (the nice student, the trier or the student who has it hard) or to punish what they see as vice (the class disrupter, the boorish teenager or the student who just won’t try) will be hard to resist. Many may well resist  this temptation but certainly not all. Under the current marking regime, every student is protected from this risk and we must not  forget that  bias in favour of one student is the same as bias against all the others not so favoured. There is no more place in marking for kind heartedness than there is for vindictiveness. An external examination process eliminates both.

The Spreading Rot

Let’s hope the teachers stick to their guns and face down the Minister. Flush her out into the open. If her objective is to abolish the Junior Certificate by stealth, then make her say so. Far better to abolish the examination than to allow it to become a monster which will rot the integrity of our second level education. And, make no mistake, any infection which takes root now in the junior cycle will in short order follow through to the senior cycle. The dangers of teachers being responsible for the summative assessment of their own students are so obvious that only the willfully blinkered can countenance such a development.

Irish Science Ethics: Poachers and Gamekeepers

June 19, 2014

For most Irish scientists, being lectured by the government’s Chief Scientific Advisor Mark Ferguson about research ethics is a decidedly uncomfortable experience. Recently Ferguson, as Director General of  Science Foundation Ireland (SFI), took to the pages of Nature to announce the launch of SFI’s new audit system to check that funded research is being conducted with integrity.

For those worried about a draconian crackdown, fear not, as the auditors will not investigate misconduct cases, nor will they have “deep” scientific knowledge of the area audited. In fact, the whole thing, according to Ferguson, will be “fairly procedural.”

In this post we ask how would the current research quality assurance system cope with a serious case of suspected research misconduct?

Read more…

Who Has Better Degrees? The Evidence

June 3, 2014

There has been much discussion of late about the comparability of degrees of different third level institutions. To quantify this one could compare the CAO points of entrants in a given year and the proportion of higher awards given to the same cohort on graduation. Thus an institution with entrants on low CAO points who subsequently graduate with a large percentage of first class honours might indicate lower standards compared to a comparable institution with high CAO point entrants but a low percentage of firsts.

In what follows we look carefully at what the Irish data says from this perspective, factoring in a number of variables, including the proportions of overseas and mature students etc. The conclusion is that Trinity College Dublin probably is maintaining higher standards on average than most other universities in Ireland, although it is arguable that standards at University of Limerick may be equivalent, and that in four year undergraduate courses in University College Cork and University College Dublin standards may also be similar.

On the other hand, it is difficult to escape the conclusion that the standard of a degree from the Institutes of Technology is well below that of the universities. This may not be true of Dublin Institute of Technology, as we do not have the data. It is hard, however, to dispute that IoT graduates, in general, should have at least a Master’s degree if they are to be considered the equivalent of your typical university graduate with a first or a 2.1 honour’s degree.

(A pdf version of this blogpost can be downloaded here).

Read more…

Technological Universities: The Shotgun Marriages

March 2, 2014

Shotgun Marriage
What with all of the hullabaloo surrounding the creation of Technological Universities, it is time to cut through the hype and look at the grim reality. In this the first piece in a series, we look at the ever-shifting proposed mergers and TU hopefuls and ask what the hell is really going on?

Read more…

More Questions for Science Foundation Ireland Director General

November 13, 2013

Minister for Jobs Richard Bruton, SFI Director General Mark Ferguson and Minister of State Sean Sherlock

How many more skeletons can Science Foundation Ireland’s Director General, Mark Ferguson, possibly have in his closet? Last year this blog revealed the facts about Renovo, the spin-off company that was founded by Science Foundations Ireland’s then newly appointed Director General, Mark Ferguson.

Now it has emerged that in 1982 Ferguson was mentioned in a New York Times article as the most visible case of academic misconduct, whereby authors simultaneously submit identical results to different journals.

In this piece we look at the latest revelations about the Renovo collapse and the controversial promotion of Ferguson to the post to Chief Scientific Adviser to the Government of Ireland as well as the details of the “duplicate publication” case.

Read more…

The Chairman’s Thesis

October 15, 2013

As summarised in our last post, a remarkable degree of controversy and media attention has focused on a dissertation for which Flan Garvey, then Chairman of the Governing Body at IT Tralee, was awarded an master of arts degree in 2008. Last year, 26 lecturers wrote to the Registrar pointing out that they had discovered that a considerable amount of the text of the thesis had appeared in a variety of previously published sources.

Flan Garvey

Flan Garvey

A QQI investigation ensued in which three external academics concluded that the thesis was plagiarised and that the MA award was obtained in a manner that was unjustified. On an appeal by Mr Garvey to a committee that was appointed by IT Tralee, the decision that the MA was unjustified was reversed on the basis that Garvey didn’t understand that plagiarism could be unintentional. QQI then announced a statutory investigation into how postgraduate research awards are made at IT Tralee.

The thesis at the centre of the whole controversy was removed from the shelves of library in IT Tralee after the initial QQI investigation into the plagiarism allegation was launched – although it is still available from Clare County Library. Before that, it was scrutinised in detail by the 26 lecturers who identified the plagiarism.

One of that number, Martin O’Grady, cofounder of the Network for Irish Educational Standards, has completed a thorough review of the thesis. He reveals that the question of plagiarism is only one of the apparent deficiencies inherent in the thesis. In this post we give a summary of that review. The full review can be downloaded by clicking here.

The Thesis

The thesis is entitled “Cabhair is Cairde is Graiste O Dhia Chugainn: A study of the Saiocht of a Parish in Co Clare.” Problems commence with the title which misleadingly suggests through the use of Irish that the thesis has some relevance to the Irish language.

In fact, it has nothing at all to do with Irish but what it is intended to be about proves very difficult to discover. “Saiocht” is never explained or defined in the thesis and, therefore, it is left to the reader to guess what exactly the study is about. Indeed, it would seem that the writer was very far from clear as to what exactly he was writing about.

There are several contradictory attempts in the thesis to explain in general terms what its objective is. They range from suggestions that it is a general history of the parish, that it is an account of the history and folklore of the parish, or an attempt to show how certain traits of the people in the parish enabled them to overcome adversity down the ages.

Ultimately, the most accurate statement made by the author is that the dissertation is an attempt “to leave to posterity all I have discovered about our place and people.” The subject of the work, it would seem, is inclusive and delimited only by the subjective interests and knowledge of the author, in so far as they apply to his parish. Read more…

Follow

Get every new post delivered to your Inbox.

Join 48 other followers