Article Text
PDF
Abstract
Objective Core outcome sets are the baseline for what should be measured in clinical research and, thus, should serve as a guide for what should be collected and reported. The Consensus Group on Outcome Measures Made in Pediatric Enteral Nutrition Clinical Trials, established in 2012, agreed that consensus on a core set of outcomes with agreed-upon definitions that should be measured and reported in clinical trials was needed. To achieve this goal, six working groups (WGs) were setup, including WG on acute diarrhoea, whose main goal was to develop a core outcome set for trials in acute diarrhoea.
Methods The first step identified how published outcomes related to acute diarrhoea were reported. The second focused on the methodology for determining which outcomes to measure in clinical trials. The third employed a two-phase questionnaire study using the Delphi technique to define clinically important outcomes to clinicians and parents.
Results For therapeutic studies, the five most important outcome measures were diarrhoea duration, degree of dehydration, need for hospitalisation (or duration of hospitalisation for inpatients), the proportion of patients recovered by 48 h and adverse effects. The prophylactic core outcome set included prevention of diarrhoea, prevention of dehydration, prevention of hospitalisation and adverse effects.
Conclusions The outcome sets for therapy and prevention can be recommended for use in future trials of patients with gastroenteritis. Their envisioned goal is to decrease study heterogeneity and to ease the comparability of studies. WG's next step is to determine how to measure the outcomes included in the core set.
- General Paediatrics
- Gastroenterology
- Infectious Diseases
- Nutrition
Statistics from Altmetric.com
What is already known on this topic
-
Differences in outcome definitions and measurements make it difficult, sometimes impossible, to synthesise the results of trials and apply them in a meaningful way.
-
Consensus on a core set of outcomes with agreed-upon definitions on what should be measured and reported in clinical trials is needed.
What this study adds
-
Using the Delphi technique, a list of core outcome sets for clinical trials in acute diarrhoea from different perspectives was developed.
-
Therapeutic studies core outcome measures: diarrhoea duration, degree of dehydration, need for hospitalisation (or duration of hospitalisation for inpatients), the proportion of patients recovered by 48 h and adverse effects.
-
Prophylactic core outcome set: prevention of diarrhoea, prevention of dehydration, prevention of hospitalisation and adverse effects.
Background
Differences in outcomes and how outcomes are defined and measured make it difficult, sometimes impossible, to synthesise the results of nutritional trials and apply them in a meaningful way. Non-uniform outcome selection and reporting leads to a difficulty in synthesising results and potentially to outcome reporting bias. Additionally, important outcomes may not be included, resulting in lower validity of the results. Recognising the problem, the Consensus Group on Outcome Measures Made in Pediatric Enteral Nutrition Clinical Trials (COMMENT) was established in 2012.1 COMMENT agreed that consensus on a core set of outcomes, with agreed-upon definitions on what should be measured and reported in nutritional trials, was needed. To achieve this goal, six working groups (WGs) were setup, including WG on acute diarrhoea. The main goal of this WG is to develop a core outcome set for clinical trials in acute diarrhoea.
Methods
In order to achieve the main goal, four steps have been planned. The first step involved identifying how outcomes related to acute diarrhoea are reported. By searching MEDLINE, EMBASE and the Cochrane Library (search date: February 2009), we found one systematic review2 that identified 138 randomised controlled trials (RCTs) reporting on ≥1 primary outcomes related to paediatric acute diarrhoea/diseases. The included trials used 64 unique definitions of diarrhoea, 69 unique definitions of diarrhoea resolution and 46 unique primary outcomes. Overall, this systematic review documented substantial heterogeneity in acute diarrhoea outcomes. Furthermore, even in what would be considered methodologically sound clinical trials, definitions of diarrhoea, primary outcomes and measurement instruments employed in RCTs of paediatric acute diarrhoea lacked evidence of validity and focused on indices that may not be important to participants.
The second step of the project focused on the methodology for determining which outcomes to measure in clinical trials. A review of studies that address the process of selecting outcomes to measure in clinical trials revealed that the best strategy for selecting outcomes for clinical trials in children is currently not known.3 When deciding on our own methods, WG had the choice of the Delphi technique, the nominal group technique, a semistructured discussion or a questionnaire-based survey.3 The Delphi technique was eventually chosen for its merits in comparison to the other options. This is a structured method for reaching consensus in which opinions are sought from individuals and the collated results are fed back to the group as a whole to generate further discussion and finally reach an agreement.4 The Delphi method's main advantage is anonymity, which allows for freedom of expression and also protects from any individual dominating a discussion, as can happen during a discussion or face-to-face debate.
The main aim of the third step of the project, which we are reporting here, was to identify outcomes of highest relevance in clinical trials on acute diarrhoea from different perspectives (ie, clinicians/researchers, patients or their families, representatives from industry and regulatory people). Previously, Sinha et al5 developed a pilot method for identifying outcomes of particular relevance when evaluating the effects of regular therapies for chronic childhood asthma from the perspectives of clinicians involved in the outpatient management of children with asthma, parents of children younger than 18 years and young people aged between 13 and 18 years. For this, paediatricians and specialist nurses, identified through the British Paediatric Respiratory Society, completed a two-round Delphi survey. It was recommended that others adopt this approach.
In May 2013, during the London meeting of the European Society for Paediatric Gastroenterology, Hepatology and Nutrition (ESPGHAN), WG decided to adopt the methodology of the pilot study developed by Sinha et al.5 Thus, we performed a two-step questionnaire study using the Delphi technique. The process first involved identifying a long list of potential outcomes (phase 1) and then defining a short list of clinically important outcomes (phase 2). The study flowchart is given in figure 1. The 'facilitator’, as he is known in a Delphi survey, a member of WG, was responsible for collecting and analysing the results of phase 1.
Study flowchart. ESPGHAN, European Society for Paediatric Gastroenterology, Hepatology and Nutrition; ESPID, European Society of Paediatric Infectious Diseases.
Prior to the questionnaires being sent out, a predefined domain/subdomain and outcome classification was created by WG. It was based on the most common pre-existing outcomes and results from Johnston et al2 and was analysed and selected by one of the WG members. It was then discussed and accepted by WG. It was decided that after the questionnaires were filled out and analysed, the results could be classified into the categories or, if needed, new categories could be created. The 'facilitator' was responsible for this as well, and acceptance and agreement was sought from WG. The predefined categories are summarised in online supplementary table A.
Phase 1
Clinical questionnaire
In phase 1, an electronic questionnaire with two open-ended questions to identify potential outcomes was created for clinicians/researchers, industry representatives and members of regulatory bodies (see online supplementary figure A). Members of ESPGHAN and the European Society of Paediatric Infectious Diseases (ESPID) were invited to participate in the electronic Delphi survey, along with representatives from industry, regulatory people and researchers. The responders were asked to consider which outcomes should be measured in clinical trials related to acute diarrhoea in both inpatient and outpatient settings.
Parents’ questionnaire
A questionnaire for parents was also created (see online supplementary figure B). Parents from Belgium, Italy, Israel and Poland were invited to participate in the first round of the Delphi survey of parents. The parents selected for the study were those of children admitted to the hospital due to acute diarrhoea. The purpose of the study was explained to them, and they were asked to answer based on their own personal view. Open questions were asked in order to identify a long list of outcomes that could be relevant.
Phase 2
Clinical questionnaire
This step included defining a short list of clinically important outcomes. Thus, the outcomes listed by at least 10% of participants in phase 1 were forwarded to phase 2. Questions in the phase 2 clinical questionnaire were based on the updated domain/subdomain and outcome classification (see online supplementary table A). In comparison with the predefined classification, subdomains such as the need for antibiotics, the need for other medication (eg, antimotility agents) and the use of intravenous rehydration therapy were added. Additionally, some minor phrasing changes were made and 'weight change' was divided into 'weight loss' and 'weight gain'. Importantly, although 'probiotics', 'functional status' and 'parental cooperation' were mentioned by >10% of responders, WG decided not to include them as they are not actually 'outcomes' and thus do not merit inclusion. The reason for not including 'functional status' was the ambiguity of this term, which could create confusion among responders.
The responders were asked to consider which outcomes should be measured in clinical trials related to acute diarrhoea in both inpatient and outpatient settings. Two types of question were created for this part—one ranking each outcome on a scale of 0 (unimportant) to 4 (very important) and the other asking responders to select the five most important outcomes in their opinion (see online supplementary figure C). The same group of individuals who had participated in phase 1 and had also agreed to take part in phase 2 was invited. At no point did the individuals know the results of phase 1.
Parents’ questionnaire
Outcomes listed by at least 10% of participants were carried forward to phase 2. In phase 2, just as for clinicians, closed-ended questions would be employed to identify the most important outcomes (see online supplementary figure D). Questions were divided into two kinds: ranking each aspect of treatment from 0 to 4 and choosing the five most important aspects of treatment. This would be done for both comforting and worrying aspects of treatment. The parents were not aware of the clinician's responses to phase 1, nor were clinicians aware of the parent's responses.
Results
Phase 1
Clinical questionnaire results
The questionnaire was initially sent to all ESPGHAN and ESPID members, which amounted to around 700 people, resulting in an extremely poor response rate (see online supplementary table B). As a result, a more targeted email was sent out to around 100 people, who included ESPGHAN and ESPID members, researchers, members of regulatory bodies and industry representatives. In the end 70 responses were received, with 64 responders agreeing to take part in phase 2. Though this response rate may be viewed as low, it is similar to the 16% response rate to the clinician's questionnaire in the Sinha et al5 study. Of the 70 responders, 46 identified themselves as clinicians, 16 as clinicians/researchers, 4 as researchers, 1 as a regulatory body member/clinician and 3 as members of industry. After receiving responses from 70 responders, we analysed and collated them. Some answers were very clear and straightforward (eg, diarrhoea duration) while others were unclear and even ambiguous (eg, diet) and it was up to us to either classify the answers into the predefined categories or create new categories. Often, we felt it was appropriate to combine responses into one outcome as well. After collecting and allocating all responses from phase 1 of the clinical questionnaire, the predefined domain/subdomain and outcome classification proved accurate and only a few outcomes needed to be added (see online supplementary table A).
Parents’ questionnaire results
Once we received responses from each of the five participating countries, we collated the results and attempted to find those that were most prevalent. A summary of all the results is presented in online supplementary table C.
Phase 2
Clinical questionnaire results
A total of 64 responders took part in phase 2 from among ESPGHAN members, ESPID members, researchers, regulatory body members and industry representatives (see online supplementary table D). In the outpatient setting, the need for hospitalisation, diarrhoea duration and dehydration were clearly considered to be the most important outcomes of treatment. In the inpatient setting, on the other hand, hospitalisation duration, diarrhoea duration, dehydration and the use of intravenous rehydration therapy were seen as the most crucial.
Parents’ questionnaire results
A total of 32 parents from Belgium, Israel, Italy and Poland were asked to take part in phase 2 (see online supplementary table E). It was found that the most comforting aspects of treatment included the child behaving normally, seeming healthy and being willing to eat and drink, diarrhoea improving and the physician seeing the child and being helpful and informative. The most worrisome aspects of treatment included bloody diarrhoea, fever and the child's worsening condition.
Discussion
Main findings
This study aimed to identify outcomes of highest relevance in clinical trials on acute diarrhoea from different perspectives. The first group of responders was represented mainly by clinicians/researchers, but also by the representatives from industry and regulatory people. WG took into account the results of the questionnaires and after a thorough discussion decided on three core outcome sets: two therapeutic core outcome sets for inpatient and outpatient scenarios and a prophylactic core outcome set.
The recommended therapeutic core outcome measures for outpatients include diarrhoea duration, degree of dehydration, need for hospitalisation, proportion of patients recovered by 48 h and adverse effects associated with therapy. The recommended therapeutic core outcome set for inpatients includes diarrhoea duration, degree of dehydration, duration of hospitalisation, proportion of patients recovered by 48 h and adverse effects associated with therapy.
The recommended prophylactic core outcome set includes prevention of diarrhoea, prevention of dehydration, prevention of hospitalisation and adverse effects associated with therapy.
WG also discussed that it would be valuable for the core outcome set to include social/life impact outcomes (eg, days of work or kindergarten missed) and economic impact outcomes. However, due to the difficulty in measuring such outcomes, it was decided to delay the introduction of social/life impact outcomes until further evaluation of these outcomes becomes available. In the meanwhile, these impact outcomes may also be recommended to researchers as a possible additional outcome in nutritional trials.
In contrast to the clinician's questionnaire, parents had a very different approach. They felt that the child's health status, including their appearance and behaviour, were of importance. Additionally, the worsening of symptoms and additional worrying symptoms such as bloody diarrhoea or fever were of importance as well. On top of this, staff behaviour was of great importance. The attention, care and approach to the patient by the medical staff (doctors and nurses) were seen as factors that could make the parents very comfortable or very worried, when lacking. It is the opinion of this WG that these results can serve as a best practice guideline to keep in mind when dealing with patients and their families in order to alleviate stress and increase doctor–patient/family cooperation.
Comparison with other studies
Core outcome sets have been implemented in other fields, most notably in rheumatology with Outcome Measures in Rheumatology. This collaboration has designed core outcome sets by reaching consensus among clinicians and researchers on which outcomes merit measuring6 and also asking patients which outcomes they feel are most important.7 Core Outcome Measures in Effectiveness Trials (COMET),8 on the other hand, has brought together researchers interested in developing core outcomes and has collected over 120 published or ongoing studies related to core outcomes, which can be found in the COMET repository (http://www.comet-initiative.org). Although many core outcome sets have been published, there have been none on acute diarrhoea until now.
Study limitations
Our study has some limitations. The Delphi technique in itself is flawed and allows for bias by the collector.9 However, it is also probably the best method to collect and define such a core outcome set and has been used so in the past.
Low response rate is another potential limitation of our study. Among clinicians, very few initially answered the invitation to phase 1. Those who finally did were asked more directly. This potentially could have resulted in bias.
While we did question parents and clinicians for our study, we did not include children. The reason for this is that diarrhoea in children under the age of 5 is a much more worrying condition than in those over 5. As a result, it was this group we focused on. Additionally, it would be very difficult, if not impossible, to gather information on what the <5-year-old patient felt about his treatment, its effects and the outcomes of his treatment.
Moreover, there is always a problem with questionnaires when it comes to comprehension. There is no certainty that those parents who completed the questionnaires in English fully understood what was being asked of them. On top of that, those who answered the translated questionnaires may have been answering questions that were slightly differently translated and thus biased the results. We tried our best to avoid this however.
Finally, a point of concern during preparation for the phase 2 questionnaire was the fact that responses in phase 1 were open to interpretation. They were often unclear, ambiguous or incomprehensible. As a result, they had to be allocated into groups or new groups needed to be created. It cannot be forgotten that this is an area of the study open to bias, mistakes and misinterpretations.
Conclusions and further research
Developing a core outcome set for use in clinical trials is an urgent task. COMMENT1 and other similar initiatives bring together parties interested in developing standardised sets of outcomes. So far, the COMMENT WG on acute diarrhoea has completed three out of four steps of the project. These included identifying how outcomes related to acute diarrhoea are reported, deciding on the methodology for determining which outcomes to measure in clinical trials and developing a core outcome set for clinical trials performed in subjects with acute diarrhoea. The latter step is being reported in the current paper. To complete the project, WG needs to determine how to measure the outcomes included in the core set. The methodology for this is still under discussion, and this will be the final part of our WG's task. It would additionally be very useful to ascertain the impact of this core outcome set creation and monitor its implementation in future trials. However, this shall be possible after the completion of step 4 and publishing of the final and complete core outcome set with the core outcome measure set. Of note, the outcome sets are the baseline for what should be measured and reported in clinical research. As clearly stated by COMET,10 the outcomes in trials do not need to be restricted to the outcomes in the core set. Instead, the core set should serve as a guide for what should be collected and reported. Thus, core sets make it easier for the results of different trials to be compared, contrasted and combined as is found necessary and useful.
Acknowledgments
Participants who agreed to take part in the Delphi process (for the list, see online supplementary table F; shown with consent from each person).
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Collaborators COMMENT Group (in alphabetical order): P Aggett; S Ashkenazi, M Ashwell, K Baerlocher, Ch Braegger, P Calder, C Campoy Folgoso, V Colomb, T Decsi, M Domellöf, Ch Dupont, M Fewtrell, A Guarino, B Koletzko, S Koletzko, W Mihatsch, K Michaelsen, P Noakes, J Rigo, R Shamir, H Szajewska, J Taminiau, D Turck, JB van Goudoever.
-
Contributors All authors contributed to study design and data collection. JK and HS developed the first draft of the manuscript. All authors contributed to the development of the final draft of the manuscript. All authors approved the manuscript.
-
Funding The workshop that initiated the activity of this working group was supported by the Early Nutrition Academy.
-
Ethics approval In order to maintain an ethical approach, no details about patients or families were collected. At the same time all clinicians were asked for their willingness and permission in taking part in the study by answering our questionnaire questions. The Ethics Committee was contacted; no special permission was deemed to be required.
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.