What impact do healthcare digital libraries have? An evaluation of national resource of infection control at the point of care using the Impact-ED framework

Patty Kostkova,Gemma Madle

DOI: https://doi.org/10.1007/s00799-012-0101-4

2012-12-21

International Journal on Digital Libraries

Abstract:Over the last decade billions of dollars’ worth of investments have been directed into ICT solutions for healthcare. In particular, new evidence-based digital libraries and web portals designed to keep busy clinicians up to date with the latest evidence were created in the UK and US. While usability and performance of digital libraries were widely researched, evaluation of impact did not seem to be sufficiently addressed. This is of major concern for healthcare digital libraries as their success or failure has a direct impact on patients’ health, clinical practice, government policies and funding initiatives. In order to fill this gap, we developed the Impact-ED evaluation framework measuring impact on four dimensions of digital libraries—content, community, services and technology. Applying a triangulation technique we analysed pre- and post-visit questionnaires to assess the clinical query or aim of the visit and subsequent satisfaction with each visit, mapped it against weblogs analysis for each session and triangulated with data from semi-structured interviews. In this paper, we present the complete description of the Impact-ED framework, a definition of the comparative Impact score and application of the framework to a real-world medical digital library, the National Resource of Infection Control (NRIC, http://www.nric.org.uk), to evaluate its impact at the point of care and demonstrate the generalisability of this novel methodology. We analysed the data from a cohort of 53 users who completed the registration questionnaire, of which 32 completed pre- and post-visit questionnaires of which 72 sets were matched for analysis and five users out of these were interviewed using Dervin’s method. NRIC is generally perceived to be a useful resource with 93 % of users reporting it provides relevant information regularly or occasionally ($$n=28$$) and provided relevant information in over 65 % of visits ($$n=47$$). NRIC has a positive impact on user knowledge in over half of visits to the library (52.8 %), NRIC actual impact score$$I_{\text{ a}} = 0.65$$ and the study revealed several areas for potential development to increase its impact.