Abstract:Background: The National Institute for Health and Care Excellence (NICE) commissioned a 'mock technology appraisal' to assess whether changes to its methods and processes are needed. This report presents the findings of independent research commissioned to inform this appraisal and the deliberations of a panel convened by NICE to evaluate the mock appraisal. Methods: Our research included reviews to identify issues, analysis methods and conceptual differences and the relevance of alternative decision frameworks, alongside the development of an exemplar case study of chimeric antigen receptor (CAR) T-cell therapy for treating acute lymphoblastic leukaemia. Results: An assessment of previous evaluations of regenerative medicines found that, although there were a number of evidential challenges, none was unique to regenerative medicines or was beyond the scope of existing methods used to conceptualise decision uncertainty. Regarding the clinical evidence for regenerative medicines, the issues were those associated with a limited evidence base but were not unique to regenerative medicines: small non-randomised studies, high variation in response and the intervention subject to continuing development. The relative treatment effects generated from single-arm trials are likely to be optimistic unless it is certain that the historical data have accurately estimated the efficacy of the control agent. Pivotal trials may use surrogate end points, which, on average, overestimate treatment effects. To reduce overall uncertainty, multivariate meta-analysis of all available data should be considered. Incorporating indirectly relevant but more reliable (more mature) data into the analysis can also be considered; such data may become available as a result of the evolving regulatory pathways being developed by the European Medicines Agency. For the exemplar case of CAR T-cell therapy, target product profiles (TPPs) were developed, which considered the 'curative' and 'bridging to stem-cell transplantation' treatment approaches separately. Within each TPP, three 'hypothetical' evidence sets (minimum, intermediate and mature) were generated to simulate the impact of alternative levels of precision and maturity in the clinical evidence. Subsequent assessments of cost-effectiveness were undertaken, employing the existing NICE reference case alongside additional analyses suggested within alternative frameworks. The additional exploratory analyses were undertaken to demonstrate how assessments of cost-effectiveness and uncertainty could be impacted by alternative managed entry agreements (MEAs), including price discounts, performance-related schemes and technology leasing. The panel deliberated on the range of TPPs, evidence sets and MEAs, commenting on the likely recommendations for each scenario. The panel discussed the challenges associated with the exemplar and regenerative medicines more broadly, focusing on the need for a robust quantification of the level of uncertainty in the cost-effective estimates and the potential value of MEAs in limiting the exposure of the NHS to high upfront costs and loss associated with a wrong decision. Conclusions: It is to be expected that there will be a significant level of uncertainty in determining the clinical effectiveness of regenerative medicines and their long-term costs and benefits, but the existing methods available to estimate the implications of this uncertainty are sufficient. The use of risk sharing and MEAs between the NHS and manufacturers of regenerative medicines should be investigated further. Funding: The National Institute for Health Research Health Technology Assessment programme.

Evaluating high-cost technologies - no need to throw the baby out with the bathwater

Navigating the unknown: how to best ‘reflect’ standard of care in indications without a dedicated treatment pathway in health technology assessment submissions

Algorithms and heuristics of health technology assessments: A retrospective analysis of factors associated with HTA outcomes for new drugs across seven OECD countries

Ethical challenges for Health Technology Assessment (HTA) in the evolving evidence landscape

Health technology assessment and innovation: here to help or hinder?

Evolving Use of Health Technology Assessment in Medical Device Procurement-Global Systematic Review: An ISPOR Special Interest Group Report

The Future of Health Technology Assessment in Low- and Middle-Income Countries

HTA - Algorithm or Process? Comment on "Expanded HTA: Enhancing Fairness and Legitimacy"

Challenges of Health Technology Assessment in Pluralistic Healthcare Systems: An ISPOR Council Report

Health technology assessment (HTA) and economic evaluation: efficiency or fairness first

Influencing Factors of Health Technology Assessment to Orphan Drugs: Empirical Evidence in England, Scotland, Canada, and Australia

Assessment of quality of data submitted for NICE technology appraisals over two decades

Funding the unfundable: mechanisms for managing uncertainty in decisions on the introduction of new and innovative technologies into healthcare systems

Bring Out Your Dead: A Review of the Cost Minimisation Approach in Health Technology Assessment Submissions to the Australian Pharmaceutical Benefits Advisory Committee

Health Technology Assessment of Medical Devices: Current Landscape, Challenges, and a Way Forward

The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal

A Scoping Review of Different Methods of Assessing the Impact of New Medical Technologies at Early Stages of Development

Acceptability of using Real-World Data to Estimate Relative Treatment Effects in Health Technology Assessments: Barriers and Future Steps

Economic Evaluation as a Tool in Emerging Technology Assessment

Generating Appropriate and Reliable Evidence for Value Assessment of Medical Devices: An ISPOR Medical Devices and Diagnostics Special Interest Group Report

Institute for Clinical and Economic Review – Peterson Health Technology Institute value assessment framework for digital health technologies