Analyzing data from a pretest-posttest control group design: The importance of statistical assumptions
European Journal of Training and Development
ISSN: 2046-9012
Article publication date: 6 September 2016
Abstract
Purpose
Among the gold standards in human resource development (HRD) research are studies that test theoretically developed hypotheses and use experimental designs. A somewhat typical experimental design would involve collecting pretest and posttest data on individuals assigned to a control or experimental group. Data from such a design that considered if training made a difference in knowledge, skills or attitudes, for example, could help advance practice. Using simulated datasets, situated in the example of a scenario-planning intervention, this paper aims to show that choosing a data analysis path that does not consider the associated assumptions can misrepresent findings and resulting conclusions. A review of HRD articles in a select set of journals indicated that some researchers reporting on pretest-posttest designs with two groups were not reporting associated statistical assumptions and reported results from repeated-measures analysis of variance that are considered of minimal utility.
Design/methodology/approach
Using heuristic datasets, situated in the example of a scenario-planning intervention, this paper will show that choosing a data analysis path that does not consider the associated assumptions can misrepresent findings and resulting conclusions. Journals in the HRD field that conducted pretest-posttest control group designs were coded.
Findings
The authors' illustrations provide evidence for the importance of testing assumptions and the need for researchers to consider alternate analyses when assumptions fail, particularly the homogeneity of regression slopes assumption.
Originality/value
This paper provides guidance to researchers faced with analyzing data from a pretest-posttest control group experimental design, so that they may select the most parsimonious solution that honors the ecological validity of the data.
Keywords
Citation
Zientek, L., Nimon, K. and Hammack-Brown, B. (2016), "Analyzing data from a pretest-posttest control group design: The importance of statistical assumptions", European Journal of Training and Development, Vol. 40 No. 8/9, pp. 638-659. https://doi.org/10.1108/EJTD-08-2015-0066
Publisher
:Emerald Group Publishing Limited
Copyright © 2016, Emerald Group Publishing Limited