Usability testing for an open-source integrated library system: A task-based study of the Koha OPAC interface

The aim of this paper is to empirically examine the usability of the Koha OPAC from a user perspective.

A series of usability tests with Koha were carried out at a private university in Bangladesh. Both experienced and novice users participated in these tests. Experienced users participated only once, whereas novices took part in three successive sessions. At first, novices’ initial performance was recorded. Then, they performed the same tasks after a short training tutorial. Novices again participated in the retention experiment with the same tasks after four weeks. A set of seven tasks was given to the users to see their performance in terms of time taken, number of errors made and success scores. Performance data were captured through a computer screen recording software, and satisfaction scores were obtained using a modified version of Questionnaire on User Interface Satisfaction (QUIS). Comparisons of performance and satisfaction with Koha OPAC were made between experienced users and novices’ initial, learning and retention experiments and amongst novices’ three test sessions.

The results showed significant performance difference between experienced and novices’ initial session. Novices could easily pick up the functionality of Koha OPAC when a brief training was provided. The comparative analysis of performance between experienced users and novices’ learning showed no significant difference between these sessions. There was a significant difference between experienced and novices’ retention in terms of success scores. The comparison amongst novices’ initial, learning and retention sessions showed significant performance differences in time taken and errors made. The QUIS results also showed significant differences in subjective satisfaction for several items between experienced users and naïve sessions, and for one item amongst novices’ three experiments.

This is a pioneering study of the task-based usability of Koha OPAC. The findings from this study will encourage researchers to empirically examine the usability of other open-source ILSs, which might result in improved user performance and satisfaction with these systems.