Maximizing agreement on diverse ontologies with “wisdom of crowds” relation classification

Abstract

Purpose

Ontologies are defined as consensual formal conceptualisation of shared knowledge. However, the explicit overlap between diverse ontologies is usually very low since they are typically constructed by different experts. Hence, the purpose of this paper is to suggest to exploit “wisdom of crowds” to assess the maximal potential for inter-ontology agreement on controversial domains.

Design/methodology/approach

The authors propose a scheme where independent ontology users can explicitly express their opinions on the specified set of ontologies. The collected user opinions are further employed as features for machine classification algorithm to distinguish between the consensual ontological relations and the controversial ones. In addition, the authors devised new evaluation methods to measure the reliability and accuracy of the presented scheme.

Findings

The accuracy of the relation classification (90 per cent) and the reliability of user agreement annotations were quite high (over 90 per cent). These results indicate a fair ability of the scheme to learn the maximal set of consensual relations out of the specified set of diverse ontologies.

Research limitations/implications

The data sets and the group of participants in our experiments were of limited size and thus the presented results are promising but cannot be generalised at this stage of research.

Practical implications

A diversity of opinions expressed by different ontologies has to be resolved in order to digitise many domains of knowledge (e.g. cultural heritage, folklore, medicine, economy, religion, history, art). This work presents a methodology to formally represent this diverse knowledge in a rich semantic scheme where there is a need to distinguish between the commonly shared and the controversial relations.

Originality/value

To the best of the knowledge this is a first proposal to consider crowd-based evaluation and classification of ontological relations to maximise the inter-ontology agreement.

Citation

Zhitomirsky-Geffet, M. and Shalom Erez, E. (2014), "Maximizing agreement on diverse ontologies with “wisdom of crowds” relation classification", Online Information Review, Vol. 38 No. 5, pp. 616-633. https://doi.org/10.1108/OIR-03-2014-0073