Designs and methods in school improvement research: a systematic review

Abstract

Purpose

The purpose of this paper is to focus on challenges faced by longitudinal quantitative analyses of school improvement processes and offers a systematic literature review of current papers that use longitudinal analyses. In this context, the authors assessed designs and methods that are used to analyze the relation between school improvement processes and student outcomes. Based on this the authors point out to what extent the papers consider different aspects of the complex nature of school improvement (e.g. multilevel structure, indirect and nonlinear effects, reciprocity). The choice of study designs and methods of analysis substantially determines which aspects of this complexity are taken into account.

Design/methodology/approach

The authors searched in four international high-impact journals and in ERIC for articles reporting longitudinal school improvement studies. The database of the review consisted of a total of 428 journal articles. In total, 13 of the 428 papers met the selection criteria and were analyzed in detail.

Findings

The analyzed papers use a wide range of designs and methodological approaches. They support the assumption that sophisticated quantitative longitudinal designs and methods can be applied effectively in school improvement research. However, considering the complexity of school improvement is accompanied by high demands on designs and methods. Due to this none of the papers met the standards applied in this review completely.

Research limitations/implications

In particular, further research is needed to consider a long period of observation, reciprocal indirect and nonlinear processes in a multilevel structure. Moreover, research is required for a better and unambiguous theoretical foundation and empirical validation of the number of and intervals between measurement points.

Practical implications

If more consideration is given to the complex nature of school improvement in future studies, the broader knowledge base will allow a better understanding of the dynamic relation of school improvement and student learning. It would thus be possible to make more appropriate recommendations for the support of school improvement practice.

Originality/value

The original contribution of the paper is to show which aspects of the complexity of school improvement processes – and to what extent – are currently addressed in designs and methods of analysis applied in quantitative longitudinal studies that investigate the relation between schools’ capacity to managing change and student outcomes. Additionally the authors aim at deriving need for further research and giving guidelines how designs and methods in further studies can reflect the complexity appropriately. It is highly important to consider all aspects of this complexity to describe and understand the dynamic relation of school improvement processes and student outcomes.

Citation

Feldhoff, T., Radisch, F. and Bischof, L.M. (2016), "Designs and methods in school improvement research: a systematic review", Journal of Educational Administration, Vol. 54 No. 2, pp. 209-240. https://doi.org/10.1108/JEA-07-2014-0083