Abstract:Purpose: To present the lessons learned from an evaluation of a comprehensive telehealth project regarding success factors and evaluation methodology for such projects. Methods: A recent experience with the evaluation of new telehealth services in BC, Canada, is summarized. Two domains of clinical applications, as well as educational and administrative uses, and the project environment were evaluated. In order to contribute to the success of the project, the evaluation included formative and summative approaches employing qualitative and quantitative methods with data collection from telehealth events, participants and existing databases. The evaluation had to be carried out under severe budgetary and time constraints. We therefore deliberately chose a broad ranging exploratory approach within a framework provided, and generated questions to be answered on the basis of initial observations and participant driven interviews with progressively more focused and detailed data gathering, including perusal of a variety of existing data sources. A unique feature was an economic evaluation using static simulation models. Results: The evaluation yielded rich and detailed data, which were able to explain a number of unanticipated findings. One clinical application domain was cancelled after 6 months, the other continues. The factors contributing to success include: Focus on chronic conditions which require visual information for proper management. Involvement of established teams in regular scheduled visits or in sessions scheduled well in advance. Problems arose with: Ad hoc applications, in particular under emergency conditions. Applications that disregard established referral patterns. Applications that support only part of a unit's services. The latter leads to the service mismatch dilemma (SMMD) with the end result that even those e-health services provided are not used. The problems encountered were compounded by issues arising from the manner in which the telehealth services had been introduced, in particular the lack of time for preparation and establishment of routine use. Educational applications had significant clinical benefits. Administrative applications generated savings which exceeded the substantial capital investment and made educational and clinical applications available at variable cost. Conclusion: Evaluation under severe constraints can yield rich information. The identified success factors, including provision of an overarching architecture and infrastructure, strong program management, thorough needs analysis and detailing applications to match the identified needs should improve the sustainability of e-health projects. Insights gained: Existing assumptions before the study was conducted: Evaluation has to proceed from identified questions according to a rigorous experimental design. Emergency and trauma services in remote regions can and should be supported via telehealth based on video-conferencing. Educational applications of telehealth directed at providers are beneficial for recruitment and retention of providers in remote areas. Insights gained by the study: An exploratory approach to evaluation using a multiplicity of methods can yield rich and detailed information even under severe constraints. Ad hoc and emergency clinical applications of telehealth can present problems unless they are based on thorough, detailed analyses of environment and need, conform to established practice patterns and rely on established trusting collaborative relationships. Less difficult applications should be introduced before attempting to support use under emergency conditions. Educational applications are of interest beyond the provider community to patients, family and community members, and have clinical value. In large, sparsely populated areas with difficult travel conditions administrative applications by themselves generate savings that compensate for the substantial capital investment for telehealth required for clinical applications.

Ten Years of Bipolar Telehealth: Program Evaluation of a Team-Based Telemental Health Clinic

Seeing the Value of Video: A Qualitative Study on Patient Preference for Using Video in a Veteran Affairs Telemental Health Program Evaluation

Telemental health: A status update

Evaluating the benefits of a live, simulation-based telebehavioral health training for a deploying army reserve unit

Older Veterans' Experiences of a Multicomponent Telehealth Program: Qualitative Program Evaluation Study

Rapid implementation of Veterans Health Administration telehealth creative arts therapies: survey evaluation of adoption and adaptation

Evaluation of a multi-site health services psychology training program for telehealth and integrated behavioral health

Telehealth-delivered recovery-orientated well-being plan group program for bipolar disorder: a pilot randomised feasibility and acceptability study

Telehealth Treatment of Patients with Bipolar Depression during the COVID-19 Pandemic: Comparative Safety, Patient Satisfaction, and Effectiveness to Prepandemic In-person Treatment

Evaluating A Chronic Pain Telehealth Program

Telehealth Collaborative Care Led by Clinical Pharmacists for People With Psychosis or Bipolar Disorder

Impact and Sustainability of Tele-Geriatric Mental Services for Rural Veterans and Other Stakeholders

Transitioning to Telehealth for COVID-19 and Beyond: Perspectives of Community Mental Health Clinicians

Telehealth Collaborative Care Led by Clinical Pharmacists for People With Psychosis or Bipolar Disorder: A Propensity Weighted Comparison With Usual Psychiatric Care

Patient experiences receiving rehabilitation care via telehealth: Identifying opportunities for remote care

Building Resilience and Improving Neurocognition (BRAIN): Feasibility and acceptability of a novel, multimodal telehealth cognitive rehabilitation

Success factors for telehealth--a case study

TeleMental Health: Standards, Reimbursement, and Interstate Practice

Factors Associated with Acute Telemental Health Consultations in Older Veterans

Telehealth videoconferencing psychotherapy in rural primary care.

Evaluation of a Telehealth-Enabled Pilot Program to Address Intensive Care Unit Health Care Worker Mental Health Distress