Article Text
PDF
Abstract
Objective Acute infection is the most common presentation of children to hospital. A minority of these infections are serious, but early recognition and adequate management are essential. We aimed to develop improved tools to assess children attending ambulatory hospital care, integrating clinical features with point-of-care C reactive protein (CRP).
Design Prospective observational diagnostic study.
Setting and patients 5517 acutely ill children (1 month–16 years) presenting to 106 paediatricians at six outpatient clinics and six emergency departments in Belgium.
Index test Point-of-care CRP alongside vital signs and objective symptoms measurements.
Main outcome Hospital admission for >24 hours with a serious infection <5 days after presentation.
Results An algorithm was developed consisting of clinical features and CRP. This achieved 97.1% (95% CI 94.3% to 98.7%) sensitivity and 99.6% (95% CI 99.2% to 99.8%) negative predictive value, excluding serious infections in 36.4% of children. It stratifies patients into three groups based on CRP level: high-risk group with CRP >75 mg/L (26.8% risk of infection), intermediate-risk group with CRP 20–75 mg/L and at least one of seven clinical features (8.1%), and lower risk group with CRP <20 mg/L with at least one of the 11 features (3.8%). Children in intermediate-risk or low-risk groups with normal clinical assessment have 0.6% and 0.4% risk of serious infections, respectively.
Conclusions Conducting a CRP test may first enable children to be stratified into three risk groups, guiding assessment of clinical features that could be performed by junior doctors or nurses. In one-third of acutely ill children, the algorithm could exclude serious infection. Prospective validation of the algorithm is needed.
Clinical trial registration NCT02024282 (post-results).
- general paediatrics
- infectious diseases
- technology
- data collection
Statistics from Altmetric.com
What is already known on this topic?
Early recognition of serious infection in acutely ill children presenting to hospital is crucial to improve outcome.
Laboratory-based C reactive protein (CRP) testing is used in specialist settings to support diagnosis and management decisions of such children.
What this study adds?
A point-of-care CRP test preceding clinical assessment can help stratify children into three risk groups.
High-risk (CRP >75 mg/L) children could be prioritised immediately for further assessment and management by senior staff.
7 features in the intermediate and 11 features in the lower risk group further classify children as at risk of a serious infection or not.
Introduction
Most acutely ill children suffer from self-limiting illnesses requiring little medical intervention.1 However, prompt recognition of the few children with a serious infection (SI), such as sepsis, meningitis and pyelonephritis, is essential to avoid complications or death2: the mortality of meningococcal disease can be as high as 14%, and up to 19% of children who survive bacterial meningitis suffer from hearing loss, amputations or neurodevelopmental disorders.3–6
The care for acutely sick children has traditionally been delivered by ambulatory care,7 but diagnostic uncertainty has driven care upwards, from primary to secondary care. In the UK, there has been a 40% rise in the number of children attending the emergency department (ED) over the last decade, with 14% presenting with febrile illness.8 In the same period, emergency admission rates have increased by 28%, primarily for acute infections,1 with up to 23 per 1000 admissions annually for a condition that could be safely managed out-of-hospital.1 9 In the USA in 2011, 1 out of every 10 children with an infectious disease visited the ED.10
The clinician must decide whether a child needs immediate admission to hospital or can be discharged safely. However recognising SI in children can be challenging, especially at disease onset when the severity of the infection is unclear. Nearly half of the children with meningococcal disease are not identified at first contact.3 Some clinical features have been shown to be ‘red flags’, increasing the likelihood of SI, but none appears to have ‘rule out’ value, making SI less likely in their absence.2 11 12
Combinations of signs and symptoms may rule out SI,11 13 14 with only one such algorithm developed in primary care, involving features such as clinician gut feeling and temperature >40°C.13–15
Evidence suggests that inflammatory markers such as C reactive protein (CRP) or procalcitonin provide the most diagnostic value in hospital settings, with CRP <20 mg/L and procalcitonin <0.5 ng/mL significantly reducing the risk of missing an SI in children.16 A recent study on point-of-care (POC) CRP in primary care found an even lower threshold of 5 mg/L to rule out SI in those children.17
To date, these tests play a minor role in hospital ambulatory care, requiring venous blood sampling, which can be bothersome when triaging acutely ill children. POC tests from a finger prick are now available in ambulatory care, testing for CRP within 4 min.18 19
We aimed to develop improved tools to assess children attending ambulatory hospital care, thus at higher risk than those presenting to primary care, by integrating clinical features with POC CRP.
Methods
Setting and patients
This study was conducted in Belgium, from 15 February 2013 to 28 February 2014.14 Children aged 1 month–16 years presenting with an acute illness to ambulatory care were recruited to have POC CRP testing.20 Results in primary care have been published elsewhere17; this paper reports the findings of children presenting to six paediatric outpatient clinics (offering same-day appointments and urgent ambulatory care) and six EDs, assessed by paediatricians (n=106) (figure 1). Children were excluded if the acute illness was caused by merely traumatic or neurological conditions, intoxication, psychiatric problems, exacerbations of a known chronic condition or if the child was referred by their general practitioners.
Flow chart of recruited illness episodes in acutely ill children. GP, general practitioner.
Index tests
Vital signs and symptoms
We asked physicians to assess 61 clinical features, selected based on previous research and consensus of an international team of clinicians and researchers,13 including the National Institute for Health and Care Excellence (NICE) traffic light system, Yale Observation Scale, temperature, heart rate, breathing rate, oxygen saturation and capillary refill.13 21 22 Parents were also asked whether the illness was different from previous illnesses and whether fever improved with antipyretics2 (online supplementary file 1).
Supplementary file 1
Vital signs—temperature, respiratory rate, heart rate, oxygen saturation and capillary refill—were measured, each according to their respective standardised method.23
‘Body temperature’ was defined as the highest temperature measured by parents or physician. Before analysis 0.5°C was added to temperatures measured under the axilla, or with a tympanic thermometer.24 25
POC CRP test
We used the Afinion CRP Test Cartridge, which has a measuring range for CRP of 5–200 mg/L19 and requires 1.5 µL of blood obtained by finger prick, providing a result within 4 min. We trained all physicians to perform the CRP test. Internal quality control was performed per manufacturer’s instructions.
Outcomes
The primary outcome of the study was hospital admission (>24 hours) for SI within 5 days after initial presentation,14 which was verified by a search of the electronic medical records of all paediatric clinics and EDs.
SIs were defined as the following:
Septicaemia (including bacteraemia): pathogenic bacteria isolated from blood culture.
Meningitis: pleocytosis or identification of bacteria or virus in cerebrospinal fluid.
Appendicitis: histology.
Pneumonia: infiltrate on chest X-ray.
Osteomyelitis: pathogens from bone aspirate, or MRI or bone scan suggestive of osteomyelitis.
Cellulitis: acute suppurative inflammation of subcutaneous tissues.
Bacterial gastroenteritis: pathogen isolated from stool culture.
Complicated urinary tract infection: >105/mL pathogens of a single species isolated from urine culture and systemic effects such as fever.
In cases where no definitive adjudication could be made based on the above, a committee consisting of clinicians with expertise in acute paediatric care assigned outcome by consensus, using all available information.
Sample size
To effectively rule out SI, the algorithm should have high sensitivity (>97%), as shown in previous studies from different settings.14 15 26
Assuming a prevalence in our population of 5%,26 27 we estimated that recruiting 5500 children would provide a maximum margin of error of 2% around our estimate of sensitivity.28
We thus aimed to include 5500 acute illness episodes in children over a 12-month period.
Statistical analysis
We calculated the area under the receiver operating characteristics curve (AUC) as a measure of CRP’s ability to discriminate between children with and without SI. We calculated the accuracy for the following predetermined cut-offs: 5 and 200 mg/L (lower and upper limit of the POC CRP test), and 20 and 80 mg/L (identified as ruling out and ruling in thresholds, respectively, in earlier studies conducted in secondary care).2 We reported diagnostic accuracy in sensitivity, specificity, positive and negative diagnostic likelihood ratios (LR+ and LR−), and positive and negative predictive values (PPV and NPV) with their 95% CIs.
We developed a new algorithm, deliberately only feeding clinical features that could also be assessed by trained triage nurses and junior doctors into the algorithm. Analyses were thus limited to the POC CRP test, all items from history taking and observation, and vital signs (online supplementary file 1).29
From the observation features, we excluded ‘clinician concern’ and ‘clinical impression child is seriously ill’ considering they are holistic features encompassing all available information from clinical examination and their assessment might be influenced by experience.29
We used Classification and Regression Tree (CART) analyses to develop this algorithm. CART is a decision tree building method used to classify patients in different classes, based on non-parametric tests. It creates a tree by exhaustively searching all possible splits recursively until the desired fit is reached. It uses the most discriminative test at every split, dichotomising continuous variables.30 CART deals effectively with missing data through surrogate splits. These splits use the available information in the other predicting variables with similar outcome patterns whenever the primary variable is missing for an individual observation to decide which side of the split that observation should go to,31 using the best available information when variables are missing, in contrast to other methods such as multivariate regression modelling.32
Sensitivity and NPV were maximised by introducing a weighing factor of 100 to misclassification of SI in a data-driven way, balancing maximum sensitivity with the complexity of the tree. The minimum number of observations for a split to be attempted was set at 20 to obtain accurate predictions. We applied a 50-fold cross-validation procedure, validating our results internally.
We conducted sensitivity analyses to explore the effect of multiple illness episodes of the same child (ie, clustering from a predisposition for abnormal clinical findings such as high temperature or health-seeking behaviour) by restricting the analyses to only the first illness episodes of all children. We conducted subgroup analyses by setting, that is, children seen at the paediatric clinics or the EDs, and by age, selecting children up to 36 months of age, to compare our results with previous research in that same age group.7–9
All analyses were performed with Stata V.13.2 software and JMP Statistical Discovery (Pro V.12.1.0; SAS Institute, Cary, North Carolina, USA).
Results
Baseline characteristics
We included 5517 new illness episodes (figure 1) in 4608 children14; 944 children had two separate illness episodes and 611 had three or more. The children’s median age was 1.5 years (age range: 1 month–15.8 years) and 2951 were boys (53.5%) (table 1). The baseline characteristics of children seen at outpatient clinics versus EDs are shown in online supplementary file 2.
Supplementary file 2
Baseline characteristics of children with or without a serious infection
The median number of missing data for all features was 4.3% (IQR 3.6%–4.7%), which was higher for POC CRP (11.3%), in part due to technical failure (2%) or parental refusal (3%).
The overall prevalence of SI was 4.9% (95% CI 4.4 to 5.5), however higher in ED than in outpatient clinics (7.5% vs 2.6%). Pneumonia and complicated urinary tract infection were the most common SIs. Of the 27 cases of sepsis and meningitis, 16 children had viral meningitis (mostly enterovirus or herpes simplex meningitis), 1 bacterial meningeal infection (group B Streptococcus), 5 Streptococcus pneumoniae sepsis, 1 Haemophilus influenzae type b sepsis (despite evidence of prior immunisation), 1 Neisseria meningitidis sepsis and 3 uropathogenic sepsis (eg, Escherichia coli). No child died during the study.
Accuracy of individual features
The overall sensitivity of individual clinical features was <65%, with only fever >1 day having high sensitivity (98.7%; 95% CI 96.3% to 99.7%), however at an LR− of 2.3 (95% CI 0.8 to 6.9).
Red flags (features substantially raising the probability of infection with LR+ >5) included the clinical impression the child is seriously ill, moaning, bloody diarrhoea, abnormal behaviour, abnormal fontanelle tension, reduced peripheral circulation, meningeal irritation, cyanosis and peritoneal irritation (LR+ from 5 to 14) (see online supplementary file 3).
Supplementary file 3
The AUC for CRP was 0.76 (95% CI 0.73 to 0.79), with diagnostic accuracy of CRP shown at different thresholds in table 2 (decreasing sensitivity and increasing specificity as threshold increases).
Accuracy of POC CRP at different thresholds to diagnose serious infection
Algorithm
The CART analyses resulted in an algorithm with POC CRP and 18 clinical features (online supplementary file 4).
Supplementary file 4
It starts with CRP, classifying children with a CRP >75 mg/L as at risk of SI (figure 2). This results in 378 children classified as at risk, of whom 100 had SI (PPV 26.8%, 95% CI 22.4% to 31.6%).
POC CRP algorithm; number of all illness episodes (n=5517) and number of serious infections (n=272) given in each classification group. POC CRP, point-of-care C reactive protein testing.
Children with a CRP between 20 and 75 mg/L should be assessed on seven features: whether the illness was different from previous illness according to parents, no improvement with antipyretics, age <6 months, fever duration <1 day, vomiting, excessive crying and reduced intake. Any positive feature classifies a further 1065 children as at risk, of whom 86 had SI (PPV 8.1%, 95% CI 6.5% to 9.9%).
In children with a CRP >20 mg/L, four vital signs (temperature, respiratory rate, oxygen saturation and capillary refill) and seven features (moaning, belly ache, pale skin, neck pain, petechial rash, inconsolable and drowsiness) should be assessed. Any positive feature classifies 2071 children as at high risk, of whom 78 had SI (PPV 3.8%, 95% CI 3.0% to 4.7%).
The algorithm classifies 36.4% of the total population as low risk for SI, with 99.6% NPV (95% CI 99.2% to 99.8%) and 97.1% sensitivity (95% CI 94.3% to 98.7%) (table 3). Eight children with SI were misclassified as low risk (with CRP ranging from <5 to 31 mg/L): six children with bronchopneumonia (chest X-ray: peribronchitis and pulmonary infiltrate) and two children diagnosed with complicated urinary tract infection (negative dimercaptosuccinic acid scan) (figure 2).
Sensitivity analyses of algorithm
Subgroup analyses by setting and age and sensitivity analysis in all first acute illness episodes indicated similar diagnostic outcome measures in all groups (table 3).
Discussion
While low CRP levels in isolation cannot rule out SIs, a CRP level >75 mg/L identified 37% of all SIs, regardless of the outcome of clinical assessment. In children with an intermediate CRP level, seven easy-to-assess clinical features identified a further 32% of all SIs. Another third of SI occurs in children with CRP <20 mg/L (contrary to guidelines in adults with cough in whom CRP <20 mg/L is used to withhold antibiotics).33 Six children with SI were missed (NPV 99.6%), although these had either a urinary tract infection or bronchopneumonia.
This was a prospective, multicentre study involving a large representative sample of children from hospital-based ambulatory care. A long list of clinical features were assessed, selected based on a systematic review of the available evidence.2 We acknowledge potential variability among paediatricians of assessing these features, which could lead to differential interpretation of the suggested algorithm.
Although this new algorithm was also built for junior doctors and triage nurses, the study was conducted by paediatricians with different levels of clinical experience.
Verification of our target condition relied on the quality of hospital records. We believe it to be robust by using both objective criteria and independent adjudicators. Considering our outcome included SIs and not all serious illnesses are an infection, a normal CRP level does not necessarily mean a child is well.
CRP and procalcitonin were identified as the best inflammatory markers for SIs in children to date in a systematic review,16 which only identified studies from hospital settings. CRP <20 mg/L was found to provide the best rule-out value for SIs in children, identical to the threshold used in our analyses, although still six children with SI had a CRP level below this threshold, underlining the importance of clinical assessment alongside the use of a single biomarker.16 We previously published the results from the primary care arm of this study and found that CRP levels <5 mg/L in children at increased risk (with breathlessness, temperature ≥40°C, diarrhoea in children aged 12–30 months or clinician’s gut feeling) rule out SI, potentially avoiding unnecessary referrals to secondary care.17 The lower CRP threshold of 5 mg/L is probably due to the early presentation in primary care, when the inflammatory response is still developing.
Most studies on this topic quantified the value of blood tests in isolation, ignoring the relation with clinical signs and symptoms.16 34–36 In order to assess the value of CRP over and above clinical features while limiting the risk of selection bias,37–40 we instructed our physicians to test CRP in every child, regardless of the clinical presentation.
Most previously reported clinical prediction rules on febrile children combining clinical features and laboratory markers either had poor external validity41 42 or targeted young infants.35 One clinical prediction model including POC CRP37 had good diagnostic performance on external validation after updating the model coefficients in a recent single-centre diagnostic study, discriminating well between pneumonia, other bacterial SI and absence of SI.43
Our algorithm contains several predictors (and thresholds) similar to the NICE guideline on feverish illness in children.23 However, we advise future updates should also include POC CRP testing, as well as parental concern and effect of antipyretics.
The Cochrane review on POC testing to guide antibiotic prescribing44 showed that POC CRP in primary care decreases antibiotic prescribing up to 30%, now recommended in NICE guidance and Dutch guideline for acute cough in adults.33 45 However, it remains to be seen whether recommendations on antibiotic prescribing should be linked to CRP levels in children, considering the fairly high prevalence of SIs in the low ranges and the fact that many SIs are potentially of viral origin.
Our results support the use of POC CRP-guided clinical assessment to help identify SI and the need for hospital admission. It allows a redesign of patient flow, with triage staff performing the CRP POC test at the beginning, and stratification of subsequent assessment based on the test result. Children at high risk of SI (CRP >75 mg/L) can be prioritised for further testing or treatment, under the supervision of senior paediatric staff; children with an intermediate (20–75 mg/L) or low CRP level (<20 mg/L) could be assessed clinically by more junior staff. Since this last category also included most missed cases (0.4% of the total population), appropriate safety netting measures ought to be put in place when their assessment proves to be reassuring (figure 3). Children at high risk should be assessed per usual care, considering the additional information of the POC test, which in most cases would be routinely available after venous blood sampling.
Suggested interpretation of algorithm. n/N, number of serious infections/number of children testing ‘no to all’ or ‘yes to any’; O2 sat, oxygen saturation; POC CRP, point-of-care C reactive protein testing; sec, seconds; t°, temperature; %, percentage of all children within this category. Reading guide: (1) perform POC CRP on all children; (2) depending on CRP level (<20; 20–75; >75 mg/L); (3) check additional signs and symptoms; and (4) if the child scores ‘yes to any’, have the child reviewed by a consultant paediatrician, if ‘no to all’, apply appropriate safety netting.
Although our newly developed algorithm is one of the very few based on recent research, rather than expert consensus,22 23 46–48 it should be validated prospectively in a new, similar paediatric population to ensure generalisability.
To make our algorithm as applicable as possible, we deliberately selected features that require very little clinical expertise. A prospective validation in which the clinical assessment is performed by junior doctors and nurses should confirm whether diagnostic accuracy is maintained.
Further research should also evaluate the cost-effectiveness of applying our algorithm in terms of healthcare expenditures (eg, avoidable admissions, costs per test and quality control of the POC devices).
Other biomarkers (eg, procalcitonin) might be valuable as well, and novel technologies, such as wearables for vital signs measurements, will substantially change the focus of future research.
Although identifying SIs in children is crucial, improving rational prescribing of antibiotics is also important. Balancing these complementary outcomes in the design of future studies will hopefully improve overall management of sick children.
Acknowledgments
We would like to thank all participating paediatricians. We would also like to thank Frederick Albert, Greet Delvou and Annelien Poppe for daily follow-up during the study. And last but not least, we would like to thank all children and parents who took part in this study.
References
Footnotes
Contributors JYV, MBL, ADS, AVdB and FB conceived the study. JYV, MBL and TDB supervised data collection and performed data follow-up and data cleaning. JYV performed the analyses, which were discussed with MBL, ADS, BA, BS, DB, AVdB and FB. JYV drafted this report, and MBL, TDB, ADS, BA, BS, DB, AVdB and FB codrafted and commented on the final version. All authors had full access to all of the data (including statistical reports and tables) in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. JYV affirms that the manuscript is an honest, accurate and transparent account of the study being reported, and that no important aspects of the study have been omitted. All authors have read and approved the final manuscript.
Funding This study was funded by the National Institute for Health and Disability Insurance (RIZIV, Belgium) under reference CGV n° 2012/235 and the Research Foundation Flanders (FWO) under reference n° G067509N. AVdB was funded by the NIHR Diagnostic Evidence Co-operative Oxford and BS by the NIHR Diagnostic Evidence Co-operative Leeds. DB is a recipient of a senior clinical investigator fellowship from the Research Foundation Flanders (FWO).
Competing interests None declared.
Patient consent Obtained from the parents/guardian.
Ethics approval The protocol of this study was approved by the Ethical Review Board of the University Hospitals/KU Leuven under reference ML8601.
Provenance and peer review Not commissioned; externally peer reviewed.