Article Text
PDF
Abstract
Evidence based medicine (EBM) consists of four separate processes.
Asking a clinical question
Locating the evidence
Appraising and synthesizing the evidence
Applying the evidence
Most people are familiar with the third step but not so familiar with the first two.
Well-designed clinical questions can help clarify in a clinicians mind exactly what information is being sought and also help with the search. This clinical question can be framed around the PICO format.
With a well-structured PICO information can then be located in a timely manner using the concept of Haynes's 5 S's and a range of websites including trial registers and meta-search engines.
Statistics from Altmetric.com
Jo Bloggs the ST6 on-call for PICU is faced with a 2-month-old infant (ex-prem with a VSD) who has been admitted with RSV+ bronchiolitis. His SHO, Alec Smart has heard that infants should be given surfactant in this situation but cannot remember more detail. Unsure, Jo pulls the standard textbook off the PICU shelf, it makes no mention of what to do (being published in 2001). Instead, Jo turns to a Google Scholar search. He types in the words ‘bronchiolitis’ and ‘surfactant’ and gets 29 000 ‘hits’. It should only take a couple of hours to go through them all but luckily the first citation seems to be relevant – a Cochrane review. Jo feels that if Google Scholar is so easy to use why bother with all those other databases ever again?
What is evidence based medicine?
Evidence based medicine (EBM) is “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients”.1 It helps support the practice of making clinical decisions which are informed by the best evidence ‘out there’ and by a patient's beliefs and values. It also offers the potential to partially solve one of the big issues of the information age, which is how to handle an exponential increase in the amount of medical information ‘out there’ without inventing a 28-hour day.
The four stages
EBM consists of four separate stages.2
Asking a clinical question
Locating the evidence
Appraising and synthesising the evidence
Applying the evidence
Most paediatricians are familiar with step 3, however you can only appraise the evidence that is before you, and unfortunately the best evidence will not always land on your doorstep or your email inbox (although you can improve the chances of this happening … read on). So when a question arises, the chances are that you will not have the answer on your desktop: you will need to search for the evidence.
In order to look for something you must first define what it is you are looking for; and so the process of EBM begins by formulating a clear question. This has two purposes. First if done properly it can help in locating the evidence. Second, formulating an unambiguous question can help distinguish what you already know from what you do not know, and what you really need to know from what would just be an interesting piece of trivia.
The second stage of EBM requires a workable and practical strategy to hunting for clinical answers (above and beyond The Hobbit's Thorin Oakenshield's witty maxim “there is nothing like looking, if you want to find something”).3 This paper concentrates on these first two steps – how to ask a question and acquire good quality evidence that may answer that question.
Asking a clinical question
Re-read the clinical scenario described above featuring Jo Bloggs and Alec Smart. What exactly do our PICU doctors want to know? Does surfactant work in all intubated babies with bronchiolitis? Or just the really sick ones with pre-existing morbidities? Or only those who are RSV+? What about the treatment itself? Which surfactant; porcine or bovine? What dose? More than one treatment? And finally what outcome are we interested in? In other words when we ask “does surfactant work” how will we know it has worked? What we might be getting at is does surfactant reduce mortality? But how clinically relevant is this outcome in a condition where overall mortality is<1% and is probably confined to those with a serious co-morbidity?4 Perhaps what really matters is how quickly we can get our patient off the ventilator and out of that expensive and very busy PICU bed.
The answers to each of these questions is possibly different, so getting it right requires you to ask the right question, hence the emphasis on getting this first step done properly.
The best, most answerable clinical questions can be built around the framework of the acronym PICO5 (Richardson 1995):
PICO
where P refers to patient, I intervention, C comparison (not always needed), O Outcome.
For example, returning to the above scenario involving a child with bronchiolitis our question would be:
In infants with bronchiolitis (Patient) does surfactant (Intervention) compared to placebo (Comparison) lead to a reduced PICU length of stay (Outcome)?
Generally speaking the PICO format works wonderfully well for questions looking at the effectiveness of therapy, where you can ask “In something does this compared with that lead to some other thing”. Thus:
In children with appendicitis (P) does a key-hole approach (I) compared to laparotomy (C) lead to reduced complication rates (O)?
In neonates with sepsis induced neutropenia (P) does GM-CSF and routine management (I) compared to routine therapy (C) reduce associated mortality and complications (O)?
But it can be more difficult to fit other sorts of questions into a PICO format, where there is no ‘intervention’.
If you take a diagnostic question (like “Is neck stiffness pathognomonic for meningitis?”), you can relatively easily extract the patient (“Child with sepsis”) and outcome (“Meningitis”). The diagnostic sign of neck stiffness can become your ‘thing of Interest’ and the acronym can be helpful again. An alternative diagnostic example might be “In a neonate with bradycardias and desaturations [P] does the presence of bounding pulses [I] confirm a diagnosis of patent ductus arteriosus [O]?”
The same approach may be employed to ask questions of risk factors (“In a child with Duchenne Muscular Dystrophy [P] does echocardiogram ventricular size [I] predict mortality [O]?”).
Questions that involve aetiology do not fit PICO like a glove either, but PICO is not meant to be a framework that must be used, it is merely a device to help frame questions and as such can be adapted. For example:
In neonates (P) who have developed Intra-uterine Growth retardation (I) what is the risk of developmental delay at 2 yrs of age (O)?
Or
In infants (P) who receive a head injury (I) what is the risk subsequent risk of ADHD (O)?
Searching for the evidence
“my students are dismayed when I say to them “half of what you are taught as medical students will in 10 years have been shown to be wrong. And the trouble is, none of your teachers knows which half.” ”
Dr Sydney Burwell, Dean of Harvard Medical School.6 (Burwell 1947)
Let us return to the question about surfactant in bronchiolitis.
The problem with searching in text books for an answer to this question is that by the time a book hits the shelves it is already out of date. It may even be that the bigger and more prestigious the book the more out of date it is because of the enormous length of time it takes to get published.7 There are also real problems with relying primarily on a search engine such as Google Scholar. True, it has several benefits; it is free, easy to use, and brings up a very wide range of sources of information including the grey literature, but its potential weaknesses (less accurate than PubMed, less advanced options for fine-tuning searches, non-specificity of results, insufficient coverage of Cochrane Systematic Reviews, time delay in coverage8,–,10) should make any user cautious about using it as the main source for answering clinical questions.
The solution when faced with a clinical question is to use one of a number of reliable (ie, evidence based) medical resources.
You could go to Medline and search for ‘RCTs’, collate the evidence and arrive at an answer … but why do this if you can read a recent systematic review? But why stop there, why use a systematic review if you can use a highly synthesised (and shorter) EBM review/synopsis. Not doing so could be compared to prospecting for diamonds for your girlfriends engagement ring in some shady smuggler infested banana republic when you could just go to Beaverbrooks and get it without the hassle (and without it literally costing an arm and a leg).
Haynes's 5 S'S
The concept of finding the low-hanging easily plucked fruit when searching for evidence has been conceptualised by Haynes11 (figure 1) to reflect the different sources of EBM available to clinicians and how they relate to each other hierarchically.
Haynes's 5 S's. Illustrates the hierarchical relationship between different sources of evidence available to the clinician. Each tier up will generally represent a collation of the evidence below, and thus searching for an answer on the highest tier probably results in the most efficient search.
Information gathering strategies: push, pull, just-in-case, just-in-time
We can divide how we search for information from these sources into two strategies1: the more passive ‘just in case’ and the more active ‘just in time’. The former relates to processes like scanning journals ‘just in case’ there is something there relevant to you (we will talk more about this later on). The latter is checking up on the dose of gentamicin in the BNF ‘just in time’ to stop the nurse giving that overdose prescribed by the new FY1. Searching for answers to a clinical question is a classic mode of ‘just-in-time’ information recovery. Both strategies can be informed by sources (or sauces) which come in two ‘flavours’ – ‘push’ and ‘pull’. With ‘pull’ the user is actively rooting out the information whereas with ‘push’ the information reaches out to the user (usually based on prespecified criterion). How these concepts relate to each is other shown in the Push/Pull/Case/Time Matrix (table 1).
Push/Pull/Case/Time Matrix
Push technologies are potentially more time efficient and the ultimate example of this is Computerised Decision Support where evidence is delivered to a doctor at some point during a clinical interaction (eg, while entering case details into a computer). There is evidence that the use of such systems can improve clinical care12 and lead to evidence based cost effective drug prescribing.13 Unfortunately the days when we all have the benefit of a working and workable CDSS in every paediatric department is some way off but the use of summaries and synopses offers the benefits of using rapid access sources of appraised high-quality evidence.
Switching to a more formal ‘pull’ approach, exactly how far you have to go to get your question answered depends on a number of things:
The time you have. (The limiting factor for all searches.)
The importance of your question.
The purpose of your search.
Timing in relation to questions is crucial. Obviously your bedside search about the efficacy of repeated intravenous hydrocortisone to prevent rebound anaphylaxis will come to a screeching halt if your patient begins complaining of shortness of breath, and a rash begins to appear before your eyes. The importance of your search will also be a valid consideration. How clinically important the question is may relate to factors such as mortality/morbidity but also frequency – that is, “is this a life and death question” or “is it something I will keep on asking week in week out until I get an answer”. For example if it is October and the first week of Bronchiolitis ‘season’ and you leave that nagging question about saline ‘nebs’ unanswered it WILL come back to haunt you before the end of January so you might as well address it now. Whereas that question about the best treatment for pneumonoultramicroscopicsilicovolcanoconiosis may never arise again and so probably does not need 6 hours to be spent online searching through obscure material.
Finally, why you need the information will also determine how doggedly you try to forage for something meaningful. If you are preparing for Grand Rounds or adapting international guidelines for local use you will want to go further than the first few ‘hits’ on Google Scholar and PubMed.
If you do not have a CDSS to ‘push’ you the answer, you will have to go ‘on the pull’. But before you type in scholar.google.com how about using a search engine that will structure your answers, starting with guidelines and working through synopses, then systematic reviews, then finally individual studies.
Two good places to start “pulling” from
Two important resources to be aware of are SUMSearch and TRIPdatabase.
SUMSearch (available from www.sumsearch.uthscsa.edu) is a metasearch engine run by the University of Texas. It functions by searching several databases (Medline, DARE, NGC among others) for papers relevant to a user defined search strategy (eg, asthmatic AND steroids) and then refining and re-running the search if too many or too few ‘hits’ occur. In addition to this SUMSearch can give basic advice about the search strategy (eg, dropping the ‘tic’ in asthmatic and the ‘s’ in steroids and replacing the dropped letters with ‘*’) and allows the user to focus the search for papers on therapeutics or diagnosis, etc.
TRIPdatabase (Turning Research Into Practice) is a search engine (available from www.tripdatabase.com) which as well as searching for systematic reviews will also identify primary research studies on a given topic as well as synopses and ‘clinical questions’ such as our very own Archimedes. (An example of how to use TRIPdatabase is shown in figure 2.)
Shows the output screen for TRIPdatabase. The top box shows the search terms. In this case “ADHD AND methyphenidate” have been entered. However a more advanced search can be carried out by clicking on the closely located blue “advanced search” bar. The listing beneath shows the results from the search in order of relevance while the box on the extreme right headed “Filter by” breaks down the search into systematic reviews, synopses, guidelines, clinical questions, primary research, textbooks and “more”. This allows the user to cherry pick a tier from Haynes 5 S's.
What if the searches from these sources draw a blank? We are at the stage of searching for syntheses and individual studies from other sources. The bad news is that while we have to broaden out our search, the good news is that (as the pyramid shape suggests) there are going to be more syntheses (eg, systematic reviews) and studies than there are summaries and synopses.
And although we are getting closer to the stage where words like ‘Boolean’ ‘MESH’ and ‘NAND’ are beginning to trouble our minds we are not at the stage of doing a Medline search yet. There are still other places we can go and look for answers.
Other important resources
You can look for systematic reviews through the Cochrane Library or DARE (Database of Abstracts of Reviews of Effectiveness). These are relatively small databases, so straightforward searches (like ‘bronchiolitis’) are not unmanageable. Alternatively larger sites for guidelines like the US National guidelines clearinghouse, NICE (or the Scottish equivalent SIGN) can be searched (see table 2 on useful sites).
Useful websites for finding evidence based resources
If we are unable to find a synthesis we then ask “can we then find an individual study which will help us” by again looking on a specialist website/database. This is where PubMed comes in, although it is a part of PubMed that you may never have seen.
Medline is a general medical database developed by the US National Library of Medicine, and can be accessed freely over the internet via the PubMed front end. It references over 5000 journals and has a notable US bias when it comes to journal coverage. It is similar to EMBASE (which has more of a European slant) although the overlap between the two databases can vary considerably but has been estimated to be approximately 34%.14
There are broadly two search approaches that can be used. These are the self-designed strategies (learning carpentry to make a table) versus the predesigned strategies (going to Ikea to buy the table).
The easiest and most practical route is to use the pre-designed search strategy employing ‘clinical queries’. The more difficult and tortuous route is to design your own clinical search strategy using the whole raft of suffixes and Boolean terms (AND OR etc) – which needs fairly extensive training and course attendance.
The predesigned strategy can be used via PubMed by accessing the clinical queries section from the main PubMed page. (The clinical queries page is shown in figure 3).
Shows the clinical queries page of PubMed. This allows the user to search for studies by category (eg, ‘therapy’ equates to RCTs). A specific systematic review page is also shown.
The user can then enter the search term(s) of interest. For example, “bronchiolitis” or “bronchiolitis AND surfactant”. In addition, boxes dealing with themes, for example, “therapy” or “diagnosis” can be ticked and a sensitive or specific search conducted. Alternatively the section specifically hunting for systematic reviews can be used first to identify whether such reviews exist.
PubMed will then carry out a search using predefined search terms which have been shown to be effective at identifying relevant citations.
For example by typing in “bronchiolitis AND surfactant” the search strategy employed will be a combination of Boolean terms (AND) and terms which are effective at identifying randomised controlled trials.
The results of the search are shown in figure 4.
Shows the results from a clinical queries search using the terms ‘bronchiolitis’ and ‘surfactant’. Two relevant randomised controlled trials are identified as well as a follow up study from a neonatal trial.
The search has identified three citations as opposed to thousands, and by retrieving the abstracts we will see that two of them are relevant. The system is not perfect – there is another RCT out there which has not been picked up. (The reason for this is that this third paper while dealing with the use of surfactant in bronchiolitis does not use the term bronchiolitis in the title or abstract, it employs the somewhat synonymous term RSV induced respiratory failure instead.) This highlights one of the limitations of using clinical queries – there may be gaps in the information obtained which may matter for a comprehensive search; however to obtain a quick answer to a clinical query it is a relatively efficient way to get the ball rolling.
This relatively quick method of using Haynes 5 S's for finding clinically relevant material works well for answering day to day questions when time is not on your side. However a more thorough search strategy is needed when:
The above databases do not yield useful results.
There is doubt about the information obtained.
A more thorough search for a dissertation, systematic review or guideline is needed.
In these instances (all of which are beyond the scope of this paper) you will need to learn how to search bibliographic databases (like Medline) properly by attending courses dealing with the above or find a friendly librarian to help you do this.
Now although we have spent a while explaining how to get your evidence, do not forget that EBM is more than just finding a paper and reading the abstract: you have got to do the appraisal and application too.
We promised at the start about ways of making answers magically appear, and thus save you valuable effort in trying to ‘keep up to date’. It is in maximising the efficiency of your ‘just in case’ brain space that will hopefully let you feel less stressed about knowing things. The ideal systems are ‘push’ – high-quality information is sent to your inbox based on your chosen criterion (eg,. all high-quality stuff published last month on ‘bronchiolitis’, or the contents of an EBM journal).
The McMaster/BMJ-Updates-Plus service provides a free ‘push’ of email summaries which are selected on the basis of a clinicians predefined interests (broad categories, eg, general paeds, neonates, emergency paeds). These can be received on a user-defined frequency (daily to weekly). The system works by searching 150 ‘premier’ journals (this is done by research staff according to predefined criterion, eg, RCTs evaluated on the basis on allocation concealment and completeness of follow-up). Eligible papers are then evaluated via the McMaster Online Rating of Evidence (MORE) international network of 4000 clinicians. Ratings are done by four reviewers who rate the paper on relevance and newsworthiness. An individual user can then set the limit on how relevant or newsworthy the articles he/she receives are. The results display the article's ratings alongside access to an abstract and reviewer comments.
Journals with ‘evidence synopses’ are also available. These include ACP Journal Club and Evidence Based Medicine (which use the MORE system) which present an independently structured abstract with a clinicians commentary. The Cochrane Journal: Evidence Based Child Health presents papers that pool together evidence from different Cochrane systematic reviews on a single topic (eg,. the efficacy of pharmacological interventions in bronchiolitis). Add to this ‘guidelines alerts’ services from NICE and the Royal College of Paediatrics and Child Health's ‘President's Letter’ (which always contains a bit tucked away about recent guidelines) and your inbox will be topped up with the highest quality, most relevant new news in paediatrics and child health. For some paper-based browsing, you should probably look out for Archimedes in the Archives ‘blue’ edition too. With all that, you may find your need to go looking for answers diminished – but if not; Happy Hunting!
Footnotes
-
Competing interests None.
-
Ethics approval This study was conducted with the approval of the Ethics committee approval was not obtained as this paper does not report the results of any research but was a review on conducting evidence based medicine commissioned by the journal.
-
Provenance and peer review Commissioned; externally peer reviewed.