Campus-Wide Information Systems

Improving self- and peer assessment processes with technology : Table of Contents

Keith Willey, Anne Gardner — Sat Nov 21 08:00:12 GMT 2009

Abstract:
Purpose – As a way of focusing curriculum development and learning outcomes universities have introduced graduate attributes, which their students should develop during their degree course. Some of these attributes are discipline-specific, others are generic to all professions. The development of these attributes can be promoted by the careful use of self- and peer assessment. The authors have previously reported using the self- and peer assessment software tool SPARK in various contexts to facilitate opportunities to practise, develop, assess and provide feedback on these attributes. This research and that of the other developers identified the need to extend the features of SPARK, to increase its flexibility and capacity to provide feedback. This paper seeks to report the results of the initial trials to investigate the potential of these new features to improve learning outcomes. Design/methodology/approach – The paper reviews some of the key literature with regard to self- and peer assessment, discusses the main aspects of the original online self- and peer assessment tool SPARK and the new version SPARKPLUS, reports and analyses the results of a series of student surveys to investigate whether the new features and applications of the tool have improved the learning outcomes in a large multi-disciplinary Engineering Design subject. Findings – It was found that using self- and peer assessment in conjunction with collaborative peer learning activities increased the benefits to students and improved engagement. Furthermore it was found that the new features available in SPARKPLUS facilitated efficient implementation of additional self- and peer assessment processes (assessment of individual work and benchmarking exercises) and improved learning outcomes. The trials demonstrated that the tool assisted in improving students' engagement with and learning from peer learning exercises, the collection and distribution of feedback and helping them to identify their individual strengths and weaknesses. Practical implications – SPARKPLUS facilitates the efficient management of self- and peer assessment processes even in large classes, allowing assessments to be run multiple times a semester without an excessive burden for the coordinating academic. While SPARKPLUS has enormous potential to provide significant benefits to both students and academics, it is necessary to caution that, although a powerful tool, its successful use requires thoughtful and reflective application combined with good assessment design. Originality/value – It was found that the new features available in SPARKPLUS efficiently facilitated the development of new self- and peer assessment processes (assessment of individual work and benchmarking exercises) and improved learning outcomes.

Developing team skills with self- and peer assessment: Are benefits inversely related to team function? : Table of Contents

Keith Willey, Anne Gardner — Sat Nov 21 08:00:12 GMT 2009

Abstract:
Purpose – Self- and peer assessment has proved effective in promoting the development of teamwork and other professional skills in undergraduate students. However, in previous research approximately 30 percent of students reported that its use produced no perceived improvement in their teamwork experience. It was hypothesised that a significant number of these students were probably members of a team that would have functioned well without self- and peer assessment and hence the process did not improve their teamwork experience. This paper aims to report the testing of this hypothesis. Design/methodology/approach – The paper reviews some of the literature on self- and peer assessment, outlines the online self- and peer assessment tool SPARKPLUS, and analyses the results of a post-subject survey of students in a large multi-disciplinary engineering design subject. Findings – It was found that students who were neutral as to whether self- and peer assessment improved their teamwork experience cannot be assumed to be members of well-functioning teams. Originality/value – To increase the benefits for all students it is recommended that self- and peer assessment focuses on collaborative peer learning, not just assessment of team contributions. Furthermore, it is recommended that feedback sessions be focused on learning not just assessment outcomes and graduate attribute development should be recorded and tracked by linking development to categories required for professional accreditation.

Successful engagement in graduate attribute assessment using software : Table of Contents

Darrall Thompson — Sat Nov 21 08:00:12 GMT 2009

Abstract:
Purpose – The benefits of an educational shift to graduate attribute development have been foregrounded in the educational literature since the early 1990s. Attribute mapping in documentation with no change to assessment constitutes a surface approach. This paper aims to use as an example a deep approach that uses software to facilitate staff and student engagement. Design/methodology/approach – In 2002 the author developed online criteria-based assessment software ReView, together with a process involving the constructive alignment of assessment tasks. This was used in an academic development context to assist staff to integrate graduate attribute development through the assessment of student work. The time-saving features of the online software, colour-coded feedback about attribute development and its facilitation of students' self-assessment were significantly successful parts of this approach. Findings – A time-saving strategy using software as a facilitator can encourage change to assessment practices. The inclusion of discipline content as part of attribute-related assessment criteria assisted staff engagement with a developmental approach to attributes. Top-down directives need bottom-up processes and both are assisted by factors such as external accreditation and course reviews. Originality/value – The paper clarifies graduate attribute terminology issues and identifies problems with “top-down directives”. It describes innovative online criteria-based assessment software used to facilitate graduate attribute integration and student self-assessment. Assessment processes that give students a progressive portrait of their attribute development are few and far between. This paper offers data about one successful approach to this issue.

From little things big things grow: scaling-up assessment of experiential learning : Table of Contents

Diana Quinn, Simon Shurville — Sat Nov 21 08:00:12 GMT 2009

Abstract:
Purpose – The new economies of the twenty-first century require new approaches to learning and teaching from higher education (HE). Accordingly many universities have gradually scaled-up learner-centred approaches, including flexible delivery and technology-enhanced learning, from the domains of enthusiasts towards the institutional level. This paper seeks to argue that these new economies and styles of learning and teaching bring similar requirements for scaling of assessment practices in HE, in particular, that it is now time for many universities to consider change initiatives to scale-up the assessment of experiential learning to the institutional level. Design/methodology/approach – The need to scale-up assessment of experiential learning in the Australian and international higher HE contexts is discussed and a variety of change initiatives to scale-up assessment of experiential learning at the University of South Australia is described. These initiatives are explored in the wider context of change management in HE. Findings – Assessment of experiential learning is at a tipping point where it needs to transition from the enthusiasts towards the mainstream of academics. Support for this process is a new challenge for academic developers, educational technologists, librarians and other stakeholders, akin to other recent challenges such as mainstreaming flexible learning and technology-enhanced learning. It is argued that for change to succeed learners and academics require local or regional evidence that experiential learning and its assessment are both beneficial and manageable. Originality/value – Taking assessment of experiential learning to the institutional level is a challenge that is reminiscent of the need to scale-up flexible delivery and technology-enhanced learning over the past decade. Information that can help universities to graduate large numbers of knowledge workers with appropriate graduate attributes developed through experiential learning should be beneficial to the graduates, the institutions and society at large.

What can you learn in three minutes?: Critical reflection on an assessment task that embeds technology : Table of Contents

Natalie Ruth Brown — Sat Nov 21 08:00:12 GMT 2009

Abstract:
Purpose – The purpose of this paper is to critically examine an assessment task, undertaken by pre-service science teachers, that integrates the use of technology (in this case digital video-recorders and video-editing software) whilst scaffolding skill development. The embedding of technology into the assessment task is purposeful, aiming to address a recognised need for teachers to gain the necessary pedagogical knowledge, technical competence and, importantly, confidence to implement productive technology-based tasks in their classrooms. Design/methodology/approach – A Wisdom of Practice Scholarship approach has been adopted to critically analyse an aspect of teaching and learning in which the author has been engaged. The evaluation draws on student reflections of the task and student evaluations from two successive cohorts. These data have been analysed against the learning outcomes of the unit, broader principles of assessment design and the extent to which modelling a technology-based task was seen by the students to be transferable to their own practice. Findings – Development of technological skills emerged as the most frequently mentioned personal learning by the participating students. Others were the development of collaborative skills, communication skills, literacy, critical literacy and science content. Importantly, students believed that what was learned through the assessment project would be applicable to their future work as teachers. Originality/value – This approach to assessment, providing an opportunity for students to develop and demonstrate generic and subject-specific learning outcomes in an engaging way as well as gaining experience with appropriate technologies, would be applicable beyond the narrow scope of the study.

Assessment of online discussion in work-integrated learning : Table of Contents

Judith McNamara, Catherine Brown — Sat Nov 21 08:00:12 GMT 2009

Abstract:
Purpose – The purpose of this paper is to examine how online discussion can be used in work-integrated learning as a vehicle for students to demonstrate their learning in the workplace and to facilitate collaborative learning where face-to-face classes are not feasible. Design/methodology/approach – The paper evaluates the use of assessable online discussion in facilitating collaborative learning and scaffolding reflection in work placement subjects. It reviews the literature regarding the use of online discussion, particularly for work placement subjects, and evaluates the use of an online discussion forum in a case study subject in the Queensland University of Technology (QUT) undergraduate law course. Findings – The paper suggests that assessable online discussion forums are appropriate to facilitate student collaboration and collaborative learning in work placement subjects. Originality/value – The paper is original in its examination of the assessment of online discussion in a work-integrated learning context.

Automating formative and summative feedback for individualised assignments : Table of Contents

Ian Robert Hamilton — Sat Nov 21 08:00:12 GMT 2009

Abstract:
Purpose – The purpose of this paper is to report on the rationale behind the use of a unique paper-based individualised accounting assignment, which automated the provision to students of immediate formative and timely summative feedback. Design/methodology/approach – As students worked towards completing their assignment, the package provided immediate formative feedback with sufficient indicators for the student to identify that they had made an error and provided encouragement for the student to take appropriate corrective action until they were satisfied that their answer was correct. Once the student was satisfied with their work, their submission was automatically marked by the assessment package according to the pre-set marking scheme. Findings – The assignment experience reported here has shown the potential for interactive spreadsheets to be created for a wide range of rule-based accounting problems to provide formative and summative feedback. The package can be (and has been) easily extended beyond financial accounting to many other accounting- and non-accounting-related fields, e.g. tax, audit, economics and statistics. In fact, individualised assignments utilising interactive spreadsheets appear to have broad applicability in any educational context (including mathematics, engineering and science) where a correct numerical answer is required. Originality/value – The unique automated feedback and marking package was developed by the author utilising Microsoft's Excel spreadsheeting tool. The marking package generated a detailed summary providing important feedback to students clearly identifying correct answers and specific problem areas, enabling the student to review these aspects. The package also provided feedback to the teacher identifying the cohort's performance on each item in the assessment.

Using wikis and blogs for assessment in first-year engineering : Table of Contents

Elizabeth Joy Smith, Julie Evelyn Mills, Baden Myers — Sat Nov 21 08:00:12 GMT 2009

Abstract:
Purpose – This paper aims to examine some of the strengths and weaknesses of the use of online tools such as wikis and blogs for assessment purposes, with the aim of proposing future developments and improvements. Design/methodology/approach – The paper utilises a case study approach by examining the outcomes of a new first-year course for all engineering students at the Institution Name that was introduced in 2008. The course, Sustainable Engineering Practice (SEP), gives students an insight into the disciplines of engineering and emphasises the skills required for working in multi-disciplinary teams. It introduces students to the profession of engineering and how it is practised within a sustainable context. Findings – The major assessment task for the course is the Engineers Without Borders (EWB) challenge and, for the first time in the engineering program at the university, wikis and blogs were used as assessment tools to evaluate student progress in meeting the course objectives. Originality/value – The evidence of student reflections in their course blogs, and the discussions of the staff teaching team in course meetings, have been utilised to consider the strengths and weaknesses of the pedagogy adopted. These sources indicated that the use of these tools for assessment was effective, but that some modifications were required to improve outcomes for both staff and students.