Simulations: a case study of City & Guilds' newest assessment

Abstract

Purpose

The purpose of this paper is to present a case study of City & Guilds' development of simulation‐based assessments for its ICT user qualification. These simulations are authentic scenario‐based replicas of word processing and spreadsheet software, which present a series of tasks that the test‐taker completes as if they were using actual software, thus demonstrating their ICT competence.

Design/methodology/approach

The methodology for the pilot study involved running a series of small trials at five customer sites, with test‐takers completing a detailed questionnaire. It is from this questionnaire that most of the conclusions about the simulations are drawn.

Findings

The case study explains the reasons why City & Guilds wanted to pursue this development work and presents the findings of a pilot exercise in which the simulations were trailed by a group of learners and tutors. The strengths and weaknesses of the simulations are evaluated, particularly in terms of assessment validity, and opportunities for further development work are discussed.

Research limitations/implications

The main limitation of this paper is that the sample used for trialling was small; this was a necessity imposed by tight budgets and deadlines. Furthermore, the authors were required to use the existing City & Guilds test delivery platform, which had implications for the type of simulation that could be developed.

Originality/value

In terms of originality, it is the authors' opinion that the paper has value because it offers a unique insight into the challenges faced by a vocational awarding organisation in its efforts to develop new assessments. Although the technology involved is not completely new, it is believed that the candid evaluation of the simulations will be of interest to anyone working with e‐assessments, regardless of their level of familiarity with the UK education system.

Citation

Stone, A. and Dearing, M. (2009), "Simulations: a case study of City & Guilds' newest assessment", Education + Training, Vol. 51 No. 5/6, pp. 422-433. https://doi.org/10.1108/00400910910987228