Causal inference in the age of big data: blind faith in data and technology

Abstract

Purpose

In the era of big data, there is doubt about the significance of causal inference as a paramount scientific task in the social sciences. Meanwhile, data-mining techniques rooted in big data and artificial intelligence (AI) have infiltrated numerous aspects of social science research. This study aims to expound the criticality of discerning causal relationships – beyond mere correlations – and scrutinizes the ramifications of big data and AI in the identification of causality.

Design/methodology/approach

This study discusses the challenges and opportunities for causality identification in the era of big data under the framework of potential outcomes model and structural causal model.

Findings

First, even in the age of big data, correlations that lack interpretability, robustness and feasibility cannot substitute causality. Second, the richness of the sample size does not help solve the problem of systematic bias in the process of causal inference. Furthermore, current AI research targets correlations rather than causality, thus creating difficulties in advancing from observations to counterfactuals.

Originality/value

This study provides insights into the impact of big data era on causal inference in the social sciences, with a view toward enhancing the pool of theoretical concepts available to researchers in relevant fields and accurately guiding the direction of scientific research in these fields.

Citation

Chao, F., Wang, W. and Yu, G. (2023), "Causal inference in the age of big data: blind faith in data and technology", Kybernetes, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/K-06-2023-1026