Review of evaluation challenges for novel AI technologies (‘Frontier AI’)

Off

HealthTech (diagnostics, medical devices and digital products) assessed by NICE includes technologies based on AI. Typically, these have focused on fixed algorithms, such as for assisting in image processing. However, in the near future, the types of AI NICE will produce guidance on will likely cover “frontier AI”, that is, highly capable general-purpose AI models that can perform a wide variety of tasks that match or exceed the capabilities present in today’s most advanced models.

The DTSU will conduct a comprehensive literature and case study review to define Frontier AI in healthcare and identify the specific clinical and economic challenges these technologies pose for Health Technology Assessment (HTA). By examining both academic and grey literature, the unit aims to determine whether frontier AI introduces unique technical or procedural hurdles, such as evolving evidence requirements or the need for more frequent monitoring, that differ significantly from traditional digital health evaluations. Ultimately, this work will clarify if current HTA methods are sufficient or if new frameworks are required to effectively assess the cost-effectiveness and impact of these advanced AI tools within NICE’s remit.
 

DSU Report

Frontier artificial intelligence and health technology assessment

This report examines how NICE’s health technology assessment methods, processes and evidence requirements may need to evolve to assess frontier artificial intelligence used in health and care. It draws on a range of literature: horizon scanning of AI health technology documents, evaluation frameworks, health economic methods, adoption and implementation evidence, and regulatory-adjacent guidance to propose practical, proportionate adaptations that can be embedded within NICE’s current processes. Throughout, we focus on distinguishing where frontier AI creates genuinely different HTA challenges compared with other technologies, and where it may amplify challenges NICE already faces.