Abstract:Purpose: To present the lessons learned from an evaluation of a comprehensive telehealth project regarding success factors and evaluation methodology for such projects. Methods: A recent experience with the evaluation of new telehealth services in BC, Canada, is summarized. Two domains of clinical applications, as well as educational and administrative uses, and the project environment were evaluated. In order to contribute to the success of the project, the evaluation included formative and summative approaches employing qualitative and quantitative methods with data collection from telehealth events, participants and existing databases. The evaluation had to be carried out under severe budgetary and time constraints. We therefore deliberately chose a broad ranging exploratory approach within a framework provided, and generated questions to be answered on the basis of initial observations and participant driven interviews with progressively more focused and detailed data gathering, including perusal of a variety of existing data sources. A unique feature was an economic evaluation using static simulation models. Results: The evaluation yielded rich and detailed data, which were able to explain a number of unanticipated findings. One clinical application domain was cancelled after 6 months, the other continues. The factors contributing to success include: Focus on chronic conditions which require visual information for proper management. Involvement of established teams in regular scheduled visits or in sessions scheduled well in advance. Problems arose with: Ad hoc applications, in particular under emergency conditions. Applications that disregard established referral patterns. Applications that support only part of a unit's services. The latter leads to the service mismatch dilemma (SMMD) with the end result that even those e-health services provided are not used. The problems encountered were compounded by issues arising from the manner in which the telehealth services had been introduced, in particular the lack of time for preparation and establishment of routine use. Educational applications had significant clinical benefits. Administrative applications generated savings which exceeded the substantial capital investment and made educational and clinical applications available at variable cost. Conclusion: Evaluation under severe constraints can yield rich information. The identified success factors, including provision of an overarching architecture and infrastructure, strong program management, thorough needs analysis and detailing applications to match the identified needs should improve the sustainability of e-health projects. Insights gained: Existing assumptions before the study was conducted: Evaluation has to proceed from identified questions according to a rigorous experimental design. Emergency and trauma services in remote regions can and should be supported via telehealth based on video-conferencing. Educational applications of telehealth directed at providers are beneficial for recruitment and retention of providers in remote areas. Insights gained by the study: An exploratory approach to evaluation using a multiplicity of methods can yield rich and detailed information even under severe constraints. Ad hoc and emergency clinical applications of telehealth can present problems unless they are based on thorough, detailed analyses of environment and need, conform to established practice patterns and rely on established trusting collaborative relationships. Less difficult applications should be introduced before attempting to support use under emergency conditions. Educational applications are of interest beyond the provider community to patients, family and community members, and have clinical value. In large, sparsely populated areas with difficult travel conditions administrative applications by themselves generate savings that compensate for the substantial capital investment for telehealth required for clinical applications.

Essential telemedicine elements (tele-ments) for connecting the academic health center and remote community providers to enhance patient care

Important Administrative Aspects of Critical Care Telemedicine Programs

Overview for Implementation of Telemedicine Services in a Large Integrated Multispecialty Health Care System

Telemedicine: a Primer

Development and Evolution of a Statewide Outpatient Consultation Service: Leveraging Telemedicine to Improve Access to Specialty Care

Developing next-generation telehealth tools and technologies: patients, systems, and data perspectives

Telemedicine Technology: a Review of Services, Equipment, and Other Aspects

Success factors for telehealth--a case study

Defining the needs of a telemedicine service

Delivering Urgent Care Using Telemedicine: Insights from Experienced Clinicians at Academic Medical Centers

Telemedicine in practice

Challenges And Opportunities Faced By Large Health Systems Implementing Telehealth

Telemedicine: Where It Is and Where It's Going

Telemedicine Training in Undergraduate Medical Education: Mixed-Methods Review

A framework for telepsychiatric training and e-health: Competency-based education, evaluation and implications

Finding Our Lanes: A Roadmap for Collaboration Between Academic Medical Centers and Behavioral Telehealth Companies

Telehealth Policy, Practice, and Education: a Position Statement of the Society of General Internal Medicine

The Development of a Telemedicine Planning Framework Based on Needs Assessment

Project Initiation for Telemedicine Services

Ethical dimensions in telemedicine - balancing technology, responsible care, and patient protection

Telemedicine and teleradiology