Skip to content

Gaming Examinations

April 5, 2012

What is Really Being Assessed?

Getting around an examination is a lot like getting around paying your taxes. You can cheat, which when applied to taxes is called ‘evasion,’ and risk unpleasant consequences if you are caught. A recent blog post addressed the issue of cheating in education.

Alternatively, you can use the loopholes left wittingly or unwittingly in the system by the powers that be. This, in financial management parlance, is called ‘tax avoidance‘ and it incurs no risks at all. Indeed, it may be positively encouraged.  We all know the myriad reliefs by which the rich in this country were, and still are in many instances, enabled to minimise their tax bill.

Examinations and other educational assessments can similarly facilitate avoidance – this time of work, learning and skill acquisition – while still allowing good grades to be obtained. In this post we focus on this gaming of examinations.

To understand how the demands attendant on examinations can be avoided, we must first understand how they are meant to work. An examination is supposed to cover a representative sample of the curriculum for the course. It depends on students being only able to handle the sample if they can handle the course content as a whole.

Mostly, that in turn depends on students having no way of predicting which sample of course content will appear on the examination paper. A candidate who gets an A in Leaving Certificate Honours English should be able to read, comprehend and appraise not just the texts on which he or she answered questions in the examinations but any text at all in the English language of a roughly similar standard. Likewise he or she should be able to write creatively and fluently about a wide variety of topics and not just the topics addressed in the examinations. All this would be true, if avoidance were not possible. How then does avoidance work?

Here is a quote from a 2009 British House of Commons Select Committee report of an enquiry into higher education in England. The quote is attributed to an academic who submitted evidence to the committee:

“For it is the worst-kept secret in the academic world that, for unseen examination papers, most tutors provide their students with the contents of the paper beforehand, or at least give them a list of topics from which the questions will be drawn. (p.121)”

Lecturers Letting Students in on the Exam Paper Commons' Committee told

In Irish higher education, as in England, it is the same lecturer, or tutor (the term used above), who both teaches the course and sets the examinations. There is absolutely nothing, except a sense of professional integrity, to prevent lecturers revealing in advance to students information about the content of their examinations.

External examiners are powerless to detect any such practice because as the above quotation goes on to say:

“The role of the external examiner is …….. predicated on an assumption of academic integrity which, for the most part, does not exist.”

They see the examination scripts of students but have no way of knowing how much the performance is dependent on foreknowledge about the questions or whether any such foreknowledge exists.

This is the first time, of which I am aware, that a lecturer has openly blown the whistle on this practice, one which I have long believed is common in higher education in Ireland. I believe it to be so because I have heard several private and not so private admissions of this by lecturers and because students have frequently alluded to the practice. I have always assiduously guarded the secrecy of my examination papers and adhered rigidly to the written rule of my college that “Information regarding topics which will be or will not be assessed or the content of questions on the paper should NOT be provided to students.”

Nevertheless, no examination session goes by without students asking for ‘hints’ for the examination which, when I seek clarification, reveals an expectation that I will declare specific topics or issues covered on the course as being featured in the examination and others as not. That the requests persist year after year despite my perennial refusal to oblige leads me to conclude that the hope, even expectation, of success is not simply a triumph of optimism over experience. Unrewarded efforts eventually wither. The optimism must have its roots in other lecture halls.

Such suspicions are borne out by a recent blog post by a former Irish third level student in which they claim:

“Quite a few of the lecturers practically give their students the final exam weeks before they are meant to sit it. The lecturer wins because he/she is seen to be teaching the students well. The students win because they get good grades but Ireland suffers because the quality of professionals in the country sucks.”

It takes no great imagination to see how information about ‘unseen’ examination papers revealed in advance undermines the validity of those examinations. Students are encouraged to delay their efforts, await hints, narrow down their focus and cram the revealed topics at the end. This is ‘avoidance’ in the clearest sense.

Avoidance can be an option, however, even where there is no possibility of advance access to the examination paper. Teachers do not get to see the Leaving Certificate papers before the candidates. All that is required is a predictable pattern from year to year and the course can be narrowed down accordingly. Not only can it be narrowed down but answers can be pre-prepared and then learned by rote and reproduced in the examination.

This ‘avoids’ all skill development. It is now, it would seem, a common practice to learn off English essays and with minimal effort massage them to fit one or other of the titles appearing on the Leaving Certificate paper. In this way, a respectable mark can be obtained while ’avoiding’ the development of even basic literacy skills.

The following reference to the Leaving Certificate examination is from a HEA paper on entry to higher education in Ireland:

“…many students have indicated that intense preparation and examination practice enables them to obtain high grades by learning off by heart evaluations or analyses prepared by others, and regurgitating these at the examination. (p.11)”

What of the alternatives to examinations such as projects and portfolios as are planned for the Junior Certificate? Any submitted essay, project or piece of work, which is not written under examination conditions, cannot be safely assumed to be the sole work of the candidate. Space for avoidance, once again. Now and again, work which is not the candidate’s can be detected, as in the plagiarism found last year in the Leaving Certificate History Research Study Reports (worth 20% of the examination).

Much more frequently, it can not be discovered, as in help given by parents, elder siblings or purchased from the burgeoning bespoke essay writing industry. When traditional examinations are criticised and other approaches proposed in their stead, this fundamental flaw of many alternatives is rarely addressed.

Not only is such flawed assessment perforce of dubious validity but it is also likely to be socially regressive. Those candidates with more educated and motivated parents will receive more help and do much better while those without such parents, usually those from poorer backgrounds, will more frequently be left to their own devices, further widening the opportunity gap between the rich and the poor.

Will Teachers Resist Pressure to Massage Portfolios?

Even if the assessment process is conducted under the supervision of the teacher, can we trust the outcomes? Teachers are under huge pressure to see that their charges do well. Will they all be able to resist massaging the results if they have the opportunity? Not if third level experience is anything to go by.

Witness too what is happening in the US where teachers have been ‘improving’ their students’ test scores. If there is to be any shift away from examinations, every care must be taken not to replace them with something even more open to abuse.

Of course, if the will were there, ‘avoidance’ could be prevented. For example:

  • College authorities could quite easily impress on lecturers that revelation of examination papers contents in any way would be regarded as a serious disciplinary offense,
  • The Leaving Certificate examinations could be made altogether less predictable rendering the rote learning of pre-prepared answers a non-runner,
  • Marking schemes could be devised in such a way that genuine understanding is rewarded.

The problem is that we would then have to face up to what is really being learned and, more importantly not being learned, in our schools and colleges. In this country comfortable myths always trump reality. When and where there is change, be prepared for one form of avoidance replacing another.

Advertisements
No comments yet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: