Measuring the impact of ICH E9(R1) on UK research protocols

The final International Conference on Harmonisation E9 Addendum on Estimands and Sensitivity Analysis in Clinical Trials (ICH E9(R1) Addendum) was adopted in November 2019. The aim is to ensure that clinical trials clearly define what treatment effects are being measured, so that the study design, data collection, and analysis all align with the trial’s objectives.. Tim Clark, Vice President Clinical Sciences in ICON Drug Development Solutions recently co-authored a peer-reviewed study of estimands published in Trials Journal. In this article we briefly outline the study which examined whether estimands were being correctly used in UK research protocols. The study also looked at how the use of estimands has changed since the adoption of the ICH E9(R1) Addendum.

What is an estimand?

Estimands are definitions of the treatment effect under investigation in a clinical trial. The ICH E9(R1) Addendum proscribes five attributes that must be defined:

Treatment condition
Population
Endpoint (outcome measure)
Population-level summary measure
How intercurrent events (ICEs) are managed

ICEs occur after the study participant has received treatment and can impact the interpretation or the measurements related to the clinical question of interest. They can include discontinuation of the treatment, use of rescue medication or mortality.

Review of estimands in UK research protocols

Dr Clark and the study co-authors extracted data from research protocols submitted to the UK Health Research Authority (HRA). The HRA overseas health research conducted within the National Health Service (NHS). Each year around 1,000 applications to conduct randomised clinical trials for medicinal products, medical devices and surgical interventions are submitted to the UK HRA. The study authors looked at submissions for the period immediately before and after the adoption of the addendum. Where trial application forms used the term “estimand” or “estimands” they were included in the study survey. From this group of protocols, the authors looked at whether the protocols defined at least one of the five proscribed attributes.

The study identified 122 protocols with the term “estimand” or “estimands”. This amounts to less than 1% of the HRA database of trials. The number of protocols which attempted to define an attribute of the estimand was 81. Between January 2011 and August 2017, prior to the publication of the Addendum, the use of estimands was approximately 3 per year. During the two-year consultation period before the adoption of the Addendum, this figure rose to 18 protocols per year, 0.6%. In the year after the adoption of the final Addendum 23 (2.1%) protocols included estimands.

Protocol definitions of estimand attributes and ICEs

Many protocols failed to include information on one or more of the estimands attributes in their description. The study found the following common errors:

43% used an incorrect description of the estimand population
29.5% did not define the endpoint within the estimand
45.7% did not attempt to define the summary measure

The study also looked at how the protocols defined ICEs and found that almost 1 in 10 protocols (8/81 [10%]) of the protocols listed these incorrectly. One surprising finding was that only 9% defined discontinuation due to an adverse event, despite this being a frequent occurrence in clinical trials.

Treatment policy (35/62 [57%]) and hypothetical (15/62 [24%]) were the most common strategies to handle treatment non-adherence with no reason specified. Composite (11/24 [46%]), treatment policy (8/24 [33%]) and hypothetical (5/24 [21%]) were the strategies used to handle use of rescue therapy.

Room for improvement to achieve clarity and transparency

The review suggests that while estimands are being used more widely, the proportion of protocols that correctly defined the primary estimand did not change appreciably over time. Hence, the clarity that estimands provide in terms of the treatment effect being estimated was often lost.

The authors conclude that if the estimand framework is going to deliver on its promise of greater transparency then clinical investigators need to undertake further training . Protocol templates from both ICH MH11 and TransCelerate provide guidance on the use of estimands in clinical trials and include structured sections for their definition and integration. Regulatory authorities must also ensure that estimands are defined correctly and applied consistently in clinical trials.

Read the peer-reviewed paper.

What is an estimand?

Review of estimands in UK research protocols

Protocol definitions of estimand attributes and ICEs

Room for improvement to achieve clarity and transparency

In this section

Connect with us