To read this content please select one of the options below:

Demographic research with non-representative internet data

Emilio Zagheni (Department of Sociology, University of Washington, Seattle, WA, USA)
Ingmar Weber (Department of Social Computing, Qatar Computing Research Institute, Doha, Qatar)

International Journal of Manpower

ISSN: 0143-7720

Article publication date: 7 April 2015

1322

Abstract

Purpose

Internet data hold many promises for demographic research, but come with severe drawbacks due to several types of bias. The purpose of this paper is to review the literature that uses internet data for demographic studies and presents a general framework for addressing the problem of selection bias in non-representative samples.

Design/methodology/approach

The authors propose two main approaches to reduce bias. When ground truth data are available, the authors suggest a method that relies on calibration of the online data against reliable official statistics. When no ground truth data are available, the authors propose a difference in differences approach to evaluate relative trends.

Findings

The authors offer a generalization of existing techniques. Although there is not a definite answer to the question of whether statistical inference can be made from non-representative samples, the authors show that, when certain assumptions are met, the authors can extract signal from noisy and biased data.

Research limitations/implications

The methods are sensitive to a number of assumptions. These include some regularities in the way the bias changes across different locations, different demographic groups and between time steps. The assumptions that we discuss might not always hold. In particular, the scenario where bias varies in an unpredictable manner and, at the same time, there is no “ground truth” available to continuously calibrate the model, remains challenging and beyond the scope of this paper.

Originality/value

The paper combines a critical review of existing substantive and methodological literature with a generalization of prior techniques. It intends to provide a fresh perspective on the issue and to stimulate the methodological discussion among social scientists.

Keywords

Acknowledgements

The authors would like to thank Klaus Zimmermann, Nikos Askitas and two anonymous reviewers, for their very helpful comments and suggestions.

Citation

Zagheni, E. and Weber, I. (2015), "Demographic research with non-representative internet data", International Journal of Manpower, Vol. 36 No. 1, pp. 13-25. https://doi.org/10.1108/IJM-12-2014-0261

Publisher

:

Emerald Group Publishing Limited

Copyright © 2015, Emerald Group Publishing Limited

Related articles