Article Text
PDF
Abstract
Aim To assess the reliability and accuracy of medical advice, over a range of types of websites, found using the Google search engine, thus simulating a patient's experience.
Design Advice was sought for five common paediatric questions using the Google search engine. The first 100 results of each question were classified as either being consistent or inconsistent with current recommendations or as ‘no answer given’. Record of the type of site and its visibility was noted.
Results 39% of the 500 sites searched gave correct information; 11% were incorrect and 49% failed to answer the question. Where an answer was available, 78% of sites gave the correct information. The accuracy of information varied depending on the topic and ranged from 51% (mumps, measles and rubella and autism) to 100% (breast feeding with mastitis/the sleeping position of a baby). Governmental sites gave uniformly accurate advice. News sites gave correct advice in 55% of cases. No sponsored sites were encountered that gave the correct advice.
Implications The authors have shown that the advice on the internet is very variable. Patients are known to use the internet for their own research and as such the authors encourage healthcare workers to recommend government or NHS websites.
Statistics from Altmetric.com
Introduction
Internet usage is now at an all time high with an estimated 70% of households in the UK having access to the internet in 2009.1 Parents are known to utilise a wide variety of resources in their search for health-related information2 and numerous studies have shown that the internet is a popular means of doing this.2,–,5 Interestingly, parents are also using the internet prior to contact with a healthcare provider4 5 and so in some cases the internet may be the sole source of healthcare advice. There is evidence to suggest that this is a trend that continues into adulthood when patients will continue to use it as a resource to answer their medical questions.6
However, the validity and quality of the information on the internet is an area that has not been well documented. Gagliadi and Jadad7 have shown on two occasions that while numerous methods exist to assess the accuracy of internet sites, none are validated. Several small studies have looked at paediatric topics. Haddow and Watts8 noted that only three out of the 22 websites concerning fever in children gave information consistent with best practice guidelines, a finding repeated by others.9 In addition, Nahas and Evans10 and Roshan et al11 looked at the medical advice regarding cough and tonsillectomy, respectively, and concluded that the information is variable and frequently of poor quality. These studies have however only looked at isolated topics and have not explored where on the internet reliable advice can be found.
What is already known on this topic
▶. Parents are known to use the internet to gather health-related information.
▶. Internet-sourced health information has been shown to be of poor quality and that which is correct may not be readily understandable to a non-clinician.
What this study adds
▶. The reliability and accuracy of health information on the internet ranges from very poor to excellent, depending on the topic.
▶. The internet does have a place in providing health-related advice but given the variation in quality, healthcare professionals should be recommending government or NHS sites to patients.
The small number of adult studies conducted have shown similar results. Soot et al12 noted that a third of websites searched contain incorrect information regarding peripheral vascular disease. Similar findings were reported when internet information for scoliosis13 and lumbar disc herniation (fewer than 10% of sites were felt to be high quality)14 were assessed.
The behaviour of users of the internet has been well documented.15 People spend an average of only 69 s on a website and most will only ever look at the first page of results. In England 16% of adults have a reading age equivalent to that expected of an 11 year old16 and so where correct information is available it may not be presented in a way that is understandable. An analysis of 114 websites found that only two were aimed at the recommended reading level.17
Google is now one of the most popular search engines available. It was the most commonly used search engine in a survey of parents attending paediatric outpatients18 and is also the most frequent way that doctors access some academic sites.19 Because of this dominance we aimed to use the Google search engine to simulate a patient's experience.
Given the prominent position the internet now holds in providing the public with medical information, the key question formulated by our group was: how reliable is the internet when used to answer a range of common paediatric questions?
Methods
We searched for advice on five common questions relating to paediatrics, limiting our search to the ‘pages from the UK’ option from www.google.co.uk.
Five questions were chosen to reflect common questions clinicians are asked in practice. Questions were selected that had clear, discrete, and evidenced answers.
A pilot Google search was performed and a record of the types of sites encountered recorded. From this, seven categories (governmental/NHS, educational establishment, news site, company, interest group, individual person's site, and sponsored link) were chosen to encompass all the types of site that had arisen.
For each question, two keywords (table 1) were used to perform a Google search. From this, sites were categorised by their type and a record made of the accuracy of the information provided. The first 100 websites for each of the five searches was assessed. This covered approximately the first 8–10 pages of results. It was felt that users would be unlikely to continue searching beyond this. Two researchers independently performed the searches over a 1-month period. A short protocol with strict definitions regarding the correct answer and allocation of a sites category was written to limit the potential for observational bias. An arbitrator was available for where uncertainty arose but was not required. Each appearance of a site was counted: so a site with higher visibility may have had more than one ‘hit’ per question. The quality of information found was classified as consistent with current recommendations (‘correct’), against current recommendations (‘incorrect’), or did not answer the question, either by not giving advice or by being unrelated to the question. Assessment of the type of site was performed by analysis of ‘about us’ sections, or the website's http address. The ‘correct’ answer for the question was based on current UK gold standard advice.
Topics and keywords
Results
The quality of the advice found varied widely depending on the question asked. Overall 197 (39%) of the 500 sites searched gave correct information; 57 (11%) gave incorrect information while 246 (49%) did not answer the question or were unrelated to the question. However, when the sites that did not answer the question were discounted, 78% of sites gave the correct information.
Questions concerning mumps, measles and rubella (MMR) and autism and HIV and breast feeding were answered correctly in 65% and 51%, respectively. The remaining questions were answered far more accurately (range 94–100%) (table 2).
Question searched and advice accuracy
Analysis of each type of website showed that governmental sites (usually ending in .gov.uk or .nhs.uk) presented accurate information every time. Educational establishments (usually .ac.uk), companies, interest groups, and individual persons performed well, with accuracies of around 80%. News sites gave correct advice in only 55%. No sponsored site was found that gave the correct advice (figure 1).
Accuracy of advice by type of website.
Restricting the search to the first page of results (table 3) showed that when an answer was available, 87% of sites were accurate. However, 35% of sites on the first page of results did not provide an appropriate response to the question.
Accuracy of information found on the first page of searching
Discussion
Previous studies8,–,14 that have looked at medical advice on the internet have only looked at isolated topics and all have been very negative concerning the quality of information available. We have shown that the quality is very variable depending on the topic but can range from excellent to very poor.
Three of the five topics investigated gave the correct information in almost all cases. However, the questions regarding MMR and autism and HIV and breast feeding were answered very poorly. A possible cause for the poor result concerning breast feeding is that some of the sites may have contained non-UK advice where breast feeding is advised regardless of HIV status. However, we had aimed to target our search to UK-specific sites by using the ‘pages from the UK’ option.
Previous research has also excluded sites from the study that do not provide an answer. We have shown that because of the large number of irrelevant sites some effort has to be invested to find specific answers (be they correct or incorrect). Search engines are designed to provide the most popular and relevant sites first. However, as can be seen from table 3, even on the first page of results many websites do not provide the requested information.
We have looked in more detail than previous studies at the type of websites that are available. Governmental websites (eg, ending in .nhs.uk or .gov.uk) all gave factually correct advice and should be promoted as the first port of call for parents.
Educational websites (for instance affiliated to a university), special interest group websites (for instance run by a patient support group), company websites, and those run by an individual all had similar accuracy ratings of around 80%. There is sometimes cynicism of the motivation behind individuals having the time and incentive to run a website unless they have a polarised view, but it seems that the information provided can be as reliable as that found on an educational institution's website.
Sponsored sites universally gave poor information, and were often unrelated to the subject. Sponsored sites are sites that have paid a premium to appear prominently in the results list on a search engine. They often reappeared on every page of searching, increasing their visibility. Many sponsored sites had a potential conflict of interest with many offering products or services. We noted that the nine sponsored sites relating to the MMR/autism controversy were all offering single vaccinations. In addition, lots of sponsored sites relating to a baby's sleeping position offered books or products aimed at improving a newborn's sleep pattern. News sites were also unreliable and often appeared on the first few pages of searching. Like sponsored sites, they have a potential conflict of interest with the maintenance of controversy.
The possibility of observer bias is a limitation of this study. We have tried to control this as far as possible with the selection of questions with evidence-based or best practice answers, strict definitions for researchers, and taking a consensus view where there was disagreement. We accept that by looking at only the first 100 sites for each search question, we have taken only a small sample but it was felt that many users would not search past this. We were unable to find any data regarding the use or popularity of google.co.uk (UK) compared to google.com (Worldwide). We have looked at topics with answers based on UK best practice and so have only searched the UK option of google.co.uk. When the same search is performed in google.com, although many results are identical the order and content does vary. This means that our results may not be generalisable to international searches.
Explanation and patient education are a vital part in the provision of holistic healthcare. A variety of techniques are currently used to do so, each with their own merits and disadvantages. The provision of locally generated, written information to patients is a common and attractive method but even this has limitations and is not always accurate or understandable.20
Given the increasing dominance of the internet in providing medical information, directing patients to an approved website, either in clinic or in their copy of the consultation letter may further enhance their education and experience and would serve to promote the high-quality websites that otherwise may be difficult to find.
Conclusions
Healthcare professionals should continue to strive to be the main source of information for patients but we should be aware that most will continue to use the internet to gather information. We suggest that in addition to verbal and written information, patients and parents should be signposted to NHS, governmental or other preapproved websites.
References
Footnotes
-
Funding None.
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Detail has been removed from this case description/these case descriptions to ensure anonymity. The editors and reviewers have seen the detailed information available and are satisfied that the information backs up the case the authors are making.