Formative evaluation in health education: An exploratory study of the usefulness of the cognitive response method

Abstract

Purpose

The purpose of this study was to explore the usefulness of a simplified version of cognitive response analysis (CRA) as a formative evaluation tool for health education materials developed for small, unfunded projects.

Design/methodology/approach

A group of women (n=15), aged between 25 and 50 years, provided cognitive responses (CRs) that communicated their ongoing thoughts after viewing each slide in a narrated, online slide presentation that focused on nutrient content claims. Participants were assigned in a systematic random fashion to one of three CRA variations. In all three variations, participants simultaneously saw a slide and heard its narration and then provided their CRs while still viewing the slide. In Variation 1, a researcher wrote participants' CRs on a log sheet (n=5). Variation 2 participants' audiotaped their CRs which were later transcribed by a researcher (n=5). Variation 3 participants wrote their CRs on a log sheet. To assess the quality and usefulness of the CRs generated by each variation, the researchers categorized each CR into content‐related categories.

Findings

The time and effort required by the researcher and participants for all three variations of collecting CRs was similar. However, transcribing the audiotaped CRs presented a greater time burden to the researcher. Analysis of variance revealed that Variation 3 generated significantly fewer CRs than the other two variations. In addition, CRs from those in Variation 3 tended to be shorter and less specific and were less useful in refining the presentation because they provided limited guidance on needed improvements. In contrast, the CRs generated by Variations 1 and 2 were judged to be more useful in identifying improvements that could optimize the value of the presentation.

Originality/value

The results suggest that a simplified version of CRA is a valuable, efficient, and low‐cost tool for formative evaluation of health education materials.

Citation

Schwartz, J. and Byrd‐Bredbenner, C. (2006), "Formative evaluation in health education: An exploratory study of the usefulness of the cognitive response method", Health Education, Vol. 106 No. 1, pp. 32-41. https://doi.org/10.1108/09654280610637184