Examination of work task and criteria choices for the relevance judgment process

Abstract

Purpose

The purpose of this paper is to better understand the dynamic nature of the relevance judgment process and the influence of work task on that process.

Design/methodology/approach

The empirical study reported here examined the information seeking behavior of a group of undergraduate college students assigned a set of research assignments (work tasks). Subjects recorded their selection of documents used for an assignment and the criteria used to judge those documents relevant. Statistical analysis was used to associate relevance judgments and the criteria used to make those judgments with work tasks.

Findings

Findings indicate a strong statistical association between work task and criteria used to judge relevance. Findings also include identification of specific criteria used to judge relevance and the relative importance of those criteria based on frequency of selection of criteria for a work task.

Research limitations/implications

Findings provide additional insights into the dynamic nature of the relevance judgment process. Relevance judgment influences revealed in these findings in the form of criteria used to make relevance judgments further explicate the relevance judgment process and provide suggestions for the improvement of information retrieval systems and information literacy efforts.

Originality/value

Understanding the relevance judgment process is critical to understanding information behavior in general. Few studies have examined relevance criteria selections as part of the relevance judgment process and fewer still have studied these selections in relation to work tasks. A better understanding of this relationship is an essential part of understanding the dynamic nature of the relevance judgment process and its influences.

Citation

Taylor, A. (2013), "Examination of work task and criteria choices for the relevance judgment process", Journal of Documentation, Vol. 69 No. 4, pp. 523-544. https://doi.org/10.1108/JD-12-2011-0054