The importance of outcomes in research

‘Clinical trials are only as credible as their outcomes.’1

Clinical trials guide clinical practice. They do this by demonstrating beneficial or detrimental effects of an intervention on patients: the outcomes of the trial. The reliability of trial findings can be reduced by poor research practices such as inadequate allocation concealment or lack of blinding.2 The CONSORT standards were developed to alleviate the problems caused by poor trial reporting.3 Problems can also be caused by the selection, measurement and reporting of outcomes for clinical trials:

• Heterogeneity: When trials addressing similar clinical questions measure different outcomes, or the same outcome at different points or using different tools, it is difficult to combine results in systematic reviews or meta-analyses. A recent study of neonatal Cochrane reviews demonstrated that half were inconclusive, with heterogeneity of outcomes an important contributor.4

• Selective reporting: Selective reporting of outcomes on the basis of the results is widespread.5 This can introduce bias to such a degree that after selective reporting is addressed, the conclusions of Cochrane reviews have been shown to change from evidence of benefit to no evidence of benefit.6 Preregistration of trials, and of the outcomes to be reported, is increasingly practised to minimise this.7

• Relevance: When the outcomes of a trial are not relevant to research users (patients, parents and clinicians) statistically significant results may be clinically meaningless and such trials will not translate into improved clinical care. In paediatrics, outcomes in clinical trials are often chosen to meet the needs of researchers,8 with children rarely involved in outcome selection.9

A solution to these problems is the development of Core Outcome Sets.

Core Outcome Sets

A Core Outcome Set is an agreed, standardised group of outcomes to be reported by all trials within a research field.10 There are a number of benefits from Core Outcome Sets:

• They reduce heterogeneity and facilitate meta-analysis.

• They reduce the risk of reporting bias, and thus ensure that all trials contribute outcome data to meta-analyses.

• By involving a wide range of stakeholders, such as patients, parents and health professionals, it is more likely that clinically relevant outcomes are identified.

Core Outcome Sets should not stifle innovation. Rather than prescribing outcome selection, they simply represent a minimum group of outcomes that should be reported, commonly as secondary outcomes. An example of a recent paediatric Core Outcome Set is shown in box 1. In this case, the core outcomes have been identified and further work is in progress to determine how they should be measured in a standardised fashion.

Identifying a Core Outcome Set

The first step in developing a Core Outcome Set is to define the scope: population, condition and types of intervention.10 This information can be used to search existing databases to ensure a suitable Core Outcome Set does not already exist. The COMET (Core Outcome Measures in Effectiveness Trials) initiative maintains a database of Core Outcome Sets.12 If no appropriate Core Outcome Set is found, developing one involves two steps:

1. identifying potential outcomes

2. ranking outcomes to determine a ‘core’ set.

Developing a Core Outcome Set should involve stakeholders including patients, parents, family members, clinicians, allied healthcare professionals and researchers. Patients and public involvement is increasingly common: 89% of current and ongoing Core Outcome Set registered on the COMET database involved them.13 Meaningful involvement of stakeholders is commonly done through a multiprofessional steering group.

Ranking outcomes to determine a ‘core’ set

Different techniques have been used to reach consensus regarding which outcomes are ‘core.’ These range from relatively informal methods, such as semistructured group discussion at a workshop or meeting, to more formal group work, such as use of the Nominal Group technique.15 Currently the most common method used is modified Delphi technique13 ; this allows the views of a large number of geographically diverse stakeholders to be integrated quickly and efficiently.

In a Delphi process,19 all potential outcomes that have been identified are sent to a panel of representatives from identified stakeholder groups, such as parents, patients and health professionals, usually as an online survey. Each individual then has the opportunity to rank each outcome as important or unimportant to them. After each round, any outcomes that are universally felt to be unimportant are removed. In the following round, participants are given feedback on how other stakeholders ranked the remaining outcomes and have the opportunity to alter their ratings on the basis of this feedback. The aim of the Delphi technique is that after several rounds consensus will be reached on a set of outcomes that all stakeholders agree are important. This process is illustrated in figure 1 (Additional File 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Illustration of the steps in the Delphi process. (Figure courtesy of the COMET Initiative).