CETL-MSOR, University of Sheffield, Sheffield

By |September 13th, 2012|2012, Presentations|0 Comments

A case study of the issues arising as lecturers use computer-aided assessment in mathematics modules


CETL-MSOR Conference 2012
12-13 July 2012
University of Sheffield, Sheffield


Recent research has developed a wealth of knowledge on the development of computer-aided assessment (CAA) systems (e.g. Greenhow et al. 2003, Pidcock et al. 2004, Sangwin 2007) and CAA has become a useful tool for lecturers to assess and give immediate feedback to large cohorts of students.

This case study examines the use of a CAA system using the QuestionMark Perception software at a UK university with a strong engineering tradition. This system has been used with around 1000 mathematics and engineering first-year undergraduates each year since the start of its development in 2002.

Though it would appear that the system is firmly established as an efficient teaching and learning resource, it has not been without problem. For example, large group sizes have precluded invigilating the online tests. Also, the questions were developed to test students’ ability to carry out common mathematical procedures, so their conceptual knowledge is not tested to the same extent.

Lecturers have responded to issues such as these and, today, a diverse range of practices aimed at mitigating these issues has emerged. Questionnaires completed by nine lecturers at the target institution highlighted stark differences in approach: from non-invigilated online tests taken by students at the time and location of their choosing, to replacement paper tests that are invigilated and incorporate questions designed to assess conceptual understanding.

Six lecturers took part in follow up interviews that sought to address the following questions:

  • What issues do lecturers identify with the use of CAA?
  • How are lecturers responding to these issues?
  • What impact will this have on their future use of CAA?

The interviews suggest that these lecturers are experiencing further issues while using CAA. For example, the paper tests delay the return of feedback and require lecturer time to mark. As for the non-invigilated test, the lecturers have no means to monitor the work that students do and have gathered anecdotes of students using mathematical software, groups of friends or prepared solutions in order to gain high marks in the tests.

Further changes have been made in order to mitigate the effect of these issues too. Some lecturers have attempted to develop new CAA questions that seek students’ conceptual understanding; and some lecturers have reduced the allocation of module marks given to CAA to minimise the advantages gained by abusing the system.

This case study aims to serve as a starting point from which to discuss good practice with computer-aided assessment. Moreover it will outline future plans at this university for better addressing some of the issues concerned.


Awakened by foxes

By |September 7th, 2012|Recent news|0 Comments

As I lay awake to the sound of mating foxes and postgraduate students (please be reassured that, so far as I can discern, they are purely intra-species relations), I look back at what has happened since my blog update.

The conference season has taken me to Brighton, Sheffield and Salamanca: to the BSRLM Day Conference, CETL-MSOR Conference and SEFI MWG Seminar, respectively. At each I presented my findings on lecturers’ use of computer-aided assessment.

Since then, I have been continuing with data collection; first interviewing students and, second, interviewing lecturers that do not use CAA. In this time I have also finished a first draft of a thesis chapter, which feels like a milestone passing despite the shortcomings I see in it.

I managed to squeeze in a holiday to Rome, which was very nice. I particularly liked the Vittorio Emanuele II monument for its ostentatiousness and sheer ridiculousness. It reached a dazzling 37°C while we were there: a feat so far unmatched by the drizzling United Kingdom. We, of course, remain hopeful of a late summer surge!

CETL-MSOR Conference 2011

By |September 6th, 2011|Recent news|0 Comments

I attended a two-day conference at Coventry University arranged by CETL-MSOR on the theme of “student-led education”. I presented some of my findings from the focus groups we conducted in April.

I met some of the leading researchers in my field and they were all very friendly and approachable! They also presented some of their latest work, showing how quickly the field is advancing.

The conference dinner at Coventry Transport Museum was great and very entertaining! Our attempt at the quiz was substantially flawed by the withdrawal of some of the questions “because they were too hard”! We were also introduced to the comedy stylings of Matt Parker, which were both enlightening and funny!

Building a profile of how students engage with CAA in mathematics modules

Broughton, S.J. Hernandez-Martinez, P. and Robinson, C.L., 2012. Building a profile of how students engage with CAA in mathematics modules. In: D. Waller (Ed.), Proceedings of the CETL-MSOR Conference 2011. Coventry University, Coventry 5-6 September 2011.