Information Resource, Interface, and Tasks As User Interaction Components for Digital Library Evaluation
Yuelin Li,Chang Liu
DOI: https://doi.org/10.1016/j.ipm.2018.10.012
IF: 7.466
2019-01-01
Information Processing & Management
Abstract:This study aims to explore the relationships between user interaction and digital libraries (DLs) evaluation. User interaction is a multi-dimensional construct and recognized as three dimensions in this study, as user interaction with: information resource; interface; and, tasks. DL evaluation is considered from the user's perspective and defined as users’ perception of DL performance from different perspectives, including the support of DL's interaction design to user interaction (labeled as interaction-design-based (IDB) evaluation), the support of task completion (labeled as task-based evaluation), and a DL's overall performance (labeled as overall evaluation). An experiment with 48 participants was conducted using the China National Knowledge Infrastructure (CNKI (http://cnki.net/), the most widely used digital library in China). Participants searched for four simulated work tasks and one real work task during the experiment, subsequently evaluating their interaction with information resource, interface, and tasks, and DL performance from different perspectives before or after the search. Correlation analysis and stepwise regression analysis were conducted to examine the relationships. The results indicate that a list of factors related to different dimensions of user interaction can significantly predict or be correlated to users’ evaluation of DL performance from different perspectives, including appropriateness, rich and valid links, reasonable page layout, salience of topics, search task difficulty, well-organized web site, easy to learn, accessibility, usefulness, familiarity with task procedure, etc. These factors surface as the most critical criteria for DL evaluation. Based on the results, an integrated DL evaluation framework is developed. The study adds new knowledge about how tasks affect DL evaluation. It has implications for improving the efficiency of DL evaluation and helping DL developers design DLs to better support users’ interaction, task completion, and their overall experience with DLs.