How effective are social engineering interventions? A meta-analysis

Abstract

Purpose

Social engineering is a prominent aspect of online crime. Various interventions have been developed to reduce the success of this type of attacks. This paper aims to investigate if interventions can help to decrease the vulnerability to social engineering attacks. If they help, the authors investigate which forms of interventions and specific elements constitute success.

Design/methodology/approach

The authors selected studies which had an experimental design and rigorously tested at least one intervention that aimed to reduce the vulnerability to social engineering. The studies were primarily identified from querying the Scopus database. The authors identified 19 studies which lead to the identification of 37 effect sizes, based on a total sample of N = 23,146 subjects. The available training, intervention materials and effect sizes were analysed. The authors collected information on the context of the intervention, the characteristics of the intervention and the characteristics of the research methodology. All analyses were performed using random-effects models, and heterogeneity was quantified.

Findings

The authors find substantial differences in effect size for the different interventions. Some interventions are highly effective; others have no effect at all. Highly intensive interventions are more effective than those that are low on intensity. Furthermore, interventions with a narrow focus are more effective than those with a broad focus.

Practical implications

The results of this study show differences in effect for different elements of interventions. This allows practitioners to review their awareness campaigns and tailor them to increase their success.

Originality/value

The authors believe that this is the first study that compares the impact of social engineering interventions systematically.

Citation

Bullee, J.-W. and Junger, M. (2020), "How effective are social engineering interventions? A meta-analysis", Information and Computer Security, Vol. 28 No. 5, pp. 801-830. https://doi.org/10.1108/ICS-07-2019-0078