Published 20 June 2024

Beyond volume metrics: towards meaningful impact measurement for digital HCP engagement

What does ‘impact’ look like for digital HCP engagement and medical education?

Within the pharmaceutical industry, many continue to rely on ‘volume’ metrics; basic headline statistics like total reach, email open rates or page views. But truthfully, these offer minimal insight into the real impact of an activity and whether core objectives are being met.

It has proven far more difficult for Industry to demonstrate truly meaningful outcomes for HCP education, such as improvements in knowledge and clinical behaviour that implicate better patient care. New approaches are required to improve how this is achieved in the digital space.

In this article, we will assess the current landscape for impact measurement in digital medical education and explore how data insights can be leveraged to drive deeper HCP engagement and unlock meaningful metrics.



As the curtains drew back, Dr. Alex Rutherford stepped out to a congress auditorium filled with healthcare professionals (HCPs) – a sea of faces from all corners of the medical community.

It was a moment the key opinion leader had been thoroughly preparing for. A leading expert in their specialty area, each word of their presentation was delivered with precision; the data was compelling and the implications for medical practice significant.

But as they reached the important conclusions of their presentation, a glance into the audience revealed that half had already left the room. Another 40% were engrossed in their own digital worlds. Only a handful, a mere 10%, were truly present, engaged and actively learning from Dr. Rutherford’s words.

The presentation concluded, not with the roar of applause and enthusiastic peppering of questions they had envisioned, but with a quiet acknowledgment from the few who truly listened. As the stage lights dimmed, Dr. Rutherford was left with a profound realisation: success in medical communication cannot be measured by the quantity of listeners, but rather by the quality of engagement.

This analogy illustrates the common over-emphasis on volume metrics for demonstrating HCP engagement impact within the pharmaceutical and life science industry. The event organisers were too focused on filling the room, rather than ensuring the right HCPs, at the right stage of their educational journey were engaged and taking away learnings aligned with their clinical needs.

While the latter is challenging to achieve, the growing availability of deep data insights into the needs and behaviours of HCPs is now unlocking the ability to deliver education in an increasingly targeted and personalised way, online.



In digital medical education too, attention has tended to focus on the ‘vanity’ metrics. But placing too much emphasis on such top-line KPIs may result in educational programmes that are poorly optimised to deliver on more meaningful engagement and impact objectives.

Instead, a focus on measuring outcomes that show impact on physicians’ knowledge, confidence and clinical decision-making is needed; outcomes that get as close as possible to signalling progress against the Industry’s overarching objective of improving patient care and ultimately, health.

Designed in the context of traditional, in-person learning, this is what Donald Moore et al. sought to address in 2009 with their framework for outcomes measurement in continuing medical education (CME). It responded to the that HCPs engaging with in-person CME activities only needed to demonstrate attendance to receive certification, with no evidence of learning required.

Providers of digital medical education have subsequently adopted ‘Moore’s Level’ assessments employing pre- and post-learning surveys in an attempt to better demonstrate impact for online programmes. However, with significant shifts in HCP engagement behaviour observed over the last 15 years, it has become clear that new approaches are required in the digital space.

More than ever, healthcare professionals are consuming educational content and gaining CME points through digital channels that offer greater flexibility of engagement than traditional CME. For example, over 90% of HCPs engaging in learning on EPG Health’s independent medical education platform Medthority ( follow a unique educational journey which, on average, spans more than 4 sessions.

Preferring to consume bite-sized educational material in 5-10 minute chunks, HCPs are engaging in an incremental style of learning online that Moore’s framework was never designed to address.



Currently, Moore’s Level assessments as applied to non-linear digital programmes lack something that was taken for granted in the traditional CME setting: transparency of individual HCP engagement and content consumption.

Instead, many providers of online medical education simply assume that active engagement has taken place. However, it has been demonstrably possible for HCPs to complete post-learning assessments and gain certification, without any evidence of meaningful learning time with which to confidently validate the outcomes measured.

Recognising this challenge, EPG Health sought to adapt Moore’s framework to enable more robust impact measurement for digital medical education, while continuing to support HCPs with the flexibility of online learning they have become accustomed to. This culminated in the development of a unique digital Impact Outcomes Framework.

The new framework – which is applied to educational programmes hosted on the Medthority platform – considers all 4 core areas covered by Moore et al (Stage of learning, educational design, Outcomes and Assessment), and takes additional steps to fit these concepts into a digital setting.

Underpinned by deep, technology-driven insight into HCP behaviour, the approach sees HCPs tracked through 5 stages of learning as they engage with the content, usually over the course of multiple sessions. By first demonstrating engaged learning, key ‘Impact Outcomes’ for resulting changes in knowledge, confidence, competence and clinical behaviour can be confidently assessed (see diagram).

IOF Funnel


The Impact Outcomes Framework is enabled by the vast wealth of data that we can now collect and utilise to measure all aspects of HCP behaviour and engagement within a Medthority educational programme.

By tapping into this data, including correlating the specific content HCPs have consumed to their learning assessment results, it is possible to build a full picture of an HCP’s learning journey. Aggregated together, this provides valuable insights; the data reporting – revealing how engagement has influenced HCP knowledge and competence – but also has an instrumental role to play in how learning objectives are addressed through programme design. It contributes to ever-evolving understanding of how HCPs learn best, and how engagement is driven through communication outreach to them.

With the level of data available at our fingertips, it is possible to do what the event organisers from the earlier scenario did not; target HCPs based on their recent learning activity, develop audience segmentation based on digital behaviour (not just demographics) to adapt communication, and react promptly to new and emerging market trends.

All of this means we can create relevant and increasingly-personalised ‘teachable moments’ that will resonate with HCPs, building their awareness to potential gaps in their knowledge and helping them to address these.

Combining an effective communication strategy and hyper-relevant content makes delivering the right educational message to the right HCP easier, resulting in higher levels of engagement and reducing the need to rely on vanity metrics. Instead, the focus can be put onto building powerful Impact Outcomes.

EPG Health helps our customers drive deep engagement and demonstrate meaningful impact through pharma-funded HCP education on Medthority. If you would like to find out more, let’s have a conversation – book a meeting

About the author

Matthew Hellyer

Head of Insight

Matt generates meaningful insights and performance tracking aligned to healthcare professional engagement and behaviour for Medthority educational programmes. He joined EPG Health in 2022 with over 10 years’ experience within data insight and visualisation.

News & Views