Statistics from Altmetric.com
The importance of outcomes in research
‘Clinical trials are only as credible as their outcomes.’1
Clinical trials guide clinical practice. They do this by demonstrating beneficial or detrimental effects of an intervention on patients: the outcomes of the trial. The reliability of trial findings can be reduced by poor research practices such as inadequate allocation concealment or lack of blinding.2 The CONSORT standards were developed to alleviate the problems caused by poor trial reporting.3 Problems can also be caused by the selection, measurement and reporting of outcomes for clinical trials:
Heterogeneity: When trials addressing similar clinical questions measure different outcomes, or the same outcome at different points or using different tools, it is difficult to combine results in systematic reviews or meta-analyses. A recent study of neonatal Cochrane reviews demonstrated that half were inconclusive, with heterogeneity of outcomes an important contributor.4
Selective reporting: Selective reporting of outcomes on the basis of the results is widespread.5 This can introduce bias to such a degree that after selective reporting is addressed, the conclusions of Cochrane reviews have been shown to change from evidence of benefit to no evidence of benefit.6 Preregistration of trials, and of the outcomes to be reported, is increasingly practised to minimise this.7
Relevance: When the outcomes of a trial are not relevant to research users (patients, parents and clinicians) statistically significant results may be clinically meaningless and such trials will not translate into improved clinical care. In paediatrics, outcomes in clinical trials are often chosen to meet the needs of researchers,8 with children rarely involved in outcome selection.9
A solution to these problems is the development of Core Outcome Sets.
Core Outcome Sets
A Core Outcome Set is an agreed, standardised group of outcomes to be reported by all trials within a research field.10 There are a number of benefits from Core Outcome Sets:
They reduce heterogeneity and facilitate meta-analysis.
They reduce the risk of reporting bias, and thus ensure that all trials contribute outcome data to meta-analyses.
By involving a wide range of stakeholders, such as patients, parents and health professionals, it is more likely that clinically relevant outcomes are identified.
Core Outcome Sets should not stifle innovation. Rather than prescribing outcome selection, they simply represent a minimum group of outcomes that should be reported, commonly as secondary outcomes. An example of a recent paediatric Core Outcome Set is shown in box 1. In this case, the core outcomes have been identified and further work is in progress to determine how they should be measured in a standardised fashion.
Box 1 An example of a paediatric Core Outcome Set
An example of a Core Outcome Set :
The recommended therapeutic core outcome measures for paediatric outpatients with acute diarrhoea include:
degree of dehydration
need for hospitalisation
proportion of patients recovered by 48 hours
adverse effects associated with therapy.
Identifying a Core Outcome Set
The first step in developing a Core Outcome Set is to define the scope: population, condition and types of intervention.10 This information can be used to search existing databases to ensure a suitable Core Outcome Set does not already exist. The COMET (Core Outcome Measures in Effectiveness Trials) initiative maintains a database of Core Outcome Sets.12 If no appropriate Core Outcome Set is found, developing one involves two steps:
identifying potential outcomes
ranking outcomes to determine a ‘core’ set.
Developing a Core Outcome Set should involve stakeholders including patients, parents, family members, clinicians, allied healthcare professionals and researchers. Patients and public involvement is increasingly common: 89% of current and ongoing Core Outcome Set registered on the COMET database involved them.13 Meaningful involvement of stakeholders is commonly done through a multiprofessional steering group.
Identifying potential outcomes
Identification of outcomes used in clinical research is commonly by systematically reviewing clinical trials within the relevant clinical field to extract primary and secondary outcomes. Such reviews often identify considerable outcome heterogeneity14 within a specialty, further supporting the need for a Core Outcome Set.
A variety of methods have been used to identify outcomes considered important by other stakeholders (patients, parents, children and other health professionals). These include primary qualitative research—in-depth interviews or focus groups15—undertaken for the purpose of the Core Outcome Set, and systematic reviews of published qualitative research.16 Such approaches are required because clinical trials have been repeatedly shown to miss outcomes considered important by these other stakeholder groups.17 18
Ranking outcomes to determine a ‘core’ set
Different techniques have been used to reach consensus regarding which outcomes are ‘core.’ These range from relatively informal methods, such as semistructured group discussion at a workshop or meeting, to more formal group work, such as use of the Nominal Group technique.15 Currently the most common method used is modified Delphi technique13 ; this allows the views of a large number of geographically diverse stakeholders to be integrated quickly and efficiently.
In a Delphi process,19 all potential outcomes that have been identified are sent to a panel of representatives from identified stakeholder groups, such as parents, patients and health professionals, usually as an online survey. Each individual then has the opportunity to rank each outcome as important or unimportant to them. After each round, any outcomes that are universally felt to be unimportant are removed. In the following round, participants are given feedback on how other stakeholders ranked the remaining outcomes and have the opportunity to alter their ratings on the basis of this feedback. The aim of the Delphi technique is that after several rounds consensus will be reached on a set of outcomes that all stakeholders agree are important. This process is illustrated in figure 1 (Additional File 1).
Core Outcome Sets are being developed in paediatrics (box 2) and more widely to improve outcome reporting in clinical trials. Where high-quality Core Outcome Sets have been developed, their use is increasingly mandated by research funders20 and journal editors.21 Within paediatrics and neonatology, Core Outcome Sets provide a mechanism to increase research involvement and participation by groups not traditionally represented (such as parents and patients), reduce research bias and strengthen the evidence base for treatments.
Box 2 Core Outcome Sets in paediatrics currently in development (according to COMET database)
Core Outcomes in Neonatology (COIN project)
Outcomes in clinical trials of bronchiolitis
Development of a Core Outcome Set for use in infants with Hirschsprung’s disease
Systematic review of tools to measure outcomes for young children with autism spectrum disorder
Contributors CG conceived of this article. JW, CG and IS drafted and created the final manuscript.
Funding This article was funded as part of aClinician Scientist Fellowship awarded to CG by the MRC (MR/N008405/1)
Competing interests None declared.
Provenance and peer review Commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.