Aug 06 2012
Health Information on the Internet
Most Americans have used the internet to look for health information (a recent survey reports that 59% of adults have searched for health information on the net). Yet there are serious concerns about the accuracy and reliability of that information. There have therefore been many studies looking at the accuracy of health information, and not surprisingly the results are concerning.
Most of these studies pick a specific topic and then have one or more experts on that topic review websites obtained through specific search terms. For example, a British study looking at the treatment of fever in children concluded:
Only a few web sites provided complete and accurate information for this common and widely discussed condition. This suggests an urgent need to check public oriented healthcare information on the internet for accuracy, completeness, and consistency.
Only 4 of 41 websites they reviewed gave complete and accurate information. Another study looking at information for carpal tunnel syndrome found:
Thirty-three percent of the sites sold commercial products for the evaluation or treatment of carpal tunnel syndrome. An additional 30 percent were commercial web sites that did not sell products. Only 23 percent of the sites were authored by a physician or an academic organization. Fewer than half of the sites offered conventional information. Twenty-three percent of the sites offered unconventional or misleading information. The mean informational value of the web sites was 28.4 of a possible 100 points.
There has also been a systematic review of studies that review health information on the internet. This review found:
Most frequently used quality criteria used include accuracy, completeness, readability, design, disclosures, and references provided. Fifty-five studies (70%) concluded that quality is a problem on the Web, 17 (22%) remained neutral, and 7 studies (9%) came to a positive conclusion. Positive studies scored significantly lower in search (P =.02) and evaluation (P =.04) methods.
Therefore most studies concluded that the quality of health information on the internet is poor, and those that had a positive evaluation were less rigorous. From existing research it seems fairly clear that the quality of health information on the internet is variable and generally poor. This is not surprising given the “wild west” nature of the internet. Anyone can buy a domain name, set up a reasonably impressive website, and then start spewing whatever nonsense they wish. Other sites may intend to provide accurate information, but it takes a lot of work to provide accurate and up to date health information, so therefore high volume sites, especially those that are monetized, likely take many shortcuts that compromise quality.
There are many efforts to improve the quality of health information on the net, or at least provide consumers a way to judge the likely accuracy of the information they find. Several studies have looked at predictors of accuracy of information. A recent study of information about infant sleeping, for example, found:
Government and organizational websites had the highest percentage of accurate information (80.1% and 72.5%, respectively). Blogs, retail product reviews, and individuals’ websites had the highest percentage of inaccurate information regarding infant sleep safety (30.9%, 36.2%, and 45.5%, respectively). News websites were accurate only one-half of the time.
This is in line with previous studies that gave similar results. The most reliable sites are institutional and education sites (whether government, university, or patient-advocacy). These are sites that likely have the resources and access to dedicated professionals to ensure high quality information. They also have another thing in common – their professional reputation is their most treasured asset, and so they care about their public face on the internet. Having wrong or misleading information branded by their institution for the world to see is something to be avoided at all costs.
Individual websites are highly variable – they run the full gamut from an individual professional blogger (journalist, scientist, or other expert) to cranks or misguided crusaders of all stripes. Even the best individual sites, however, still represent the knowledge and opinions of an individual. This by necessity (and yes, of course this includes my own humble blog) is limited to an individual’s perspective and biases. It’s always a good idea, therefore, to balance that with the opinions of others, especially institutions that can represent a consensus of expert opinion.
Commercial sites generally should be looked at with skepticism. The bottom line is that they are trying to sell you something, so their motivation is very different from an academic institution. They may be directly trying to sell you a product, and all of the information they are providing is essentially a commercial (perhaps disguised deliberately as news or an information resource). Or they may simply be driving traffic to their website, and providing provocative opinions and information is a good way to do that.
Another study also looked at indicators of accuracy and found slightly different results:
Three indicators correlated with accuracy: displaying the HONcode logo, having an organization domain, and displaying a copyright. Many proposed indicators taken from published guidelines did not correlate with accuracy (e.g., the author being identified and the author having medical credentials) or inaccuracy (e.g., lack of currency and advertising).
The HON code stands for “Health on the Net” and is a seal of approval, meaning that the HON organization has looked at the material and found it to be reliable. It makes sense that this should be a predictor, because this review can weed out blatantly inaccurate, pseudoscientific, or commercial sites. Displaying a copyright is interesting – I wonder if that is just a marker for having institutional resources.
It is interesting but not surprising that having a medical degree did not predict accuracy. In my experience many crank or commercial websites use degrees and credential as a mark of legitimacy – it’s a marketing strategy. It apparently is very easy to find someone with an MD or PhD to endorse your product, and often times they are part owners of the company. Also, having a degree is no guarantee that one is not a crank.
Also interesting is the fact that having advertising on the website is not a negative predictor. At times I have run Google ads on this site, for example, and I often get comments from people that use this fact as an indicator that the site is somehow less legitimate. Placing a strip of ads on a website with decent traffic, however, is an easy way to support the site. In my case it generates enough money to pay for bandwidth, and that is about it, but it’s nice for a site to at least pay for itself.
Having generic ads on a site to cover costs, however, is different from a highly monetized or commercial site – making millions off of selling products, and gearing their editorial policy toward maximizing sales.
I also found a number of studies looking at the process of evaluating websites itself. The internet (despite the fact that many of us take it for granted now) is a relatively new phenomenon in our culture. Methods for evaluating information on the net are still evolving. One study looked at inter-rater reliability among the experts that are evaluating the information for accuracy. They found:
The medical experts showed a low agreement when rating the postings from the newsgroup. Hence, it is important to test inter-rater reliability in research assessing the accuracy and quality of health-related information on the Internet. A discussion of the different measures of agreement that could be used reveals that the choice of statistic can be problematic. It is therefore important to consider the assumptions underlying a measure of reliability before using it. Often, more than one measure will be needed for “triangulation” purposes.
In other words, when more than one expert is used to evaluate information, they often do not agree with each other about the accuracy of the information. Further, there are various statistical methods that can be used to measure how much they agree, and these produce different results. The authors therefore conclude that multiple experts should be used, and multiple statistical methods should be used to measure their agreement.
The same study also found that research that uses a single expert to evaluate health information may be unreliable itself. The knowledge and opinions of an individual, even an expert, are quirky. In general it is more reliable to use a consensus of multiple experts than any individual.
Conclusion
We are still living in the wild west of the internet. The net is becoming a critical resource, and health information is near the top of the list. It is difficult, however, to create and maintain accurate resources of health information. The topics are complex and the information is constantly changing. In my experience the more controversial the topic also the more misinformation there is out there clogging up Google searches.
There is no single guarantee of information accuracy. The features I discussed above, however, are a useful guide, at least to a first approximation of which information is likely to be accurate. There are some additional rules you can follow also.
- Don’t rely on any one site. If the information is important, verify it on multiple reliable sites.
- Always seek out the counter opinion for anything that seems controversial. Until you have heard both sides (or multiple sides) to a controversy, do not think you understand it or which side is likely to be correct.
- Don’t be afraid to get help from those with more knowledge and expertise than you (including your personal doctor). Even when information on the net is accurate it may be difficult for the non-expert to understand and put into proper context. I see patients every day with information they got from the internet which is not inaccurate, they are just unable to put the information into a proper context and apply it properly to their situation. In many cases I see patients or family members very confused by accurate information, but they don’t know how to plug that information into the specific situation they are dealing with.
And – as always – be skeptical.
13 Responses to “Health Information on the Internet”
Leave a Reply
You must be logged in to post a comment.
Hmmm… Like, say, my new twitter follower? http://www.drguberman.com/
Excellent post.
I know you must experience the ever increasing frequency of patients who have self diagnosed and are trying to get you to agree to the treatment they believe they need. I try to take internet questions a a critical thinking exercise. Working in a spine specialty I get a great deal of internet marketing and information questions. As a critical teaching moment I often say “Google strep throat and back pain independently”. You will see that strep throat will have a 7 million hits verses 280 million for back pain. I will tell them that strep throat has simple, easy cure, that is effective. So the there is much less BS online. Back pain has no easy answers so there are tons of scams. I always steer my patients toward “quackwatch” and “science based medicine” websites. Another thing I always say as a general rule is” Google the title of the webpage and add in scam and see what you get”. I find giving them their own Ah-Ha moment helps.
I had an experience this last winter with online medicine. I had pain in my hip for over 3 months. It wasn’t to bad and came and went. I’m 27, used to play football and rugby in college so thought it was just old injuries. After a while (and one trip to a therapeutic touch therapist at the insistence of my mother, but that’s a different story) it got worse so i made a doctors appointment. I also looked the symptoms up on webmd and other websites. It appeared that i had hip bursitis, caused by overuse or an old injury. I learned that bursitis could also be caused by an infection in very rare cases. I began to worry that i had this, but told myself that i was just being a hypochondriac. I new it was foolish to self diagnose so put the thought aside.
At the time i also had (what i thought was) a bad cold or the flu. The doc diagnosed me with bursitis and a cold, gave me a steroid shot in the hip that completely releaved my symptoms….. for about a week, then the pain came back worse. I went to bed on night in pain, and feeling like my cold or flu had come back. The next morning there was a black spot on my hip. i made an appointment with the doc. By the time i got there the black spot had grown in size and had a white spot in the center. The doc freaked out when he saw this and scheduled me to see an orthopedic surgeon. It had been bursitis caused by infection, specifically a osteomialitis staff infection deep in my pelvis that i had had since birth! 5 surgeries, 1 flight for life hospital transfer, 2 letters of drained puss, removal of 15% of the mussel in my leg and 3 months later i was fine.
My point in shearing this is that diagnosis is a complicated and imprecise method, while i may have been right in my fears about my condition, if i fallowed up on every such thought i would be in the ER every other day. A psychology prof once started an entry level clinical psychology class by saying “as this class progresses you will think you have schizophrenia, depression, multiple personality disorder, that you are a psychopath, sociopath, and all other manners of mental afflictions, you do not and are not, and this is why..” Internet medicine looks like it will have a role in healthcare, but it may put too much information in the hands of the those who do not understand their own limitations and can lead to poor healthcare decisions. On the other hand readily available ACCURATE information in the hands of a somewhat educated public can take stress off of the overburdened healthcare system.
The internet has accended to the main delivery system of entertainment.That medium having built a staggering raport with users now brings all content togather that previously was the domain of tv,newspapers,encyclopedias,classrooms and books.The internet has primed us to inhibit common sense where it concerns health,appearence and performance.Web pages are sparkling billboards,bedazzled with seals,badges and emblems that are often eye candy.
These are all useful and interesting rules for evaluating sources, but what’s all this “wild west” stuff? Compared to what? I was at a book store yesterday (hoping to find a used copy of Barrett’s insanely-priced Consumer Health), and whoa boy is the medicine section a sad monument to the seweriness of the printed word.
In my opinion, this is the best piece ever written on this blog. Education on how to interrupt data and/or those providing such data is the only long-term solution to helping people avoid quacks.
Thadius,
Ignore this comment if you like, it is just my rambling thoughts based on incomplete data on the following:
“osteomialitis staff infection deep in my pelvis that i had had since birth”
I assume you mean osteomyelitis infection caused by the bacterium, Staphyococcus aureus.
(My brother had such an infection in his tibia and nearly lost his leg as a result)
Somehow, and I’m not saying this with any confidence, I think the more likely explanation is that your doctor accidentally introduced this infection into your bursa when he injected the cortisone. Staph aureus is a organism often found on the skin which could have been the source of the infection. And the cortisone could have aggravated that infection because it is an immune suppressant. The time sequence would seem to suggest this. It would seem very unlikely to me that a staph aureus infection would lie dormant for 20 years. Perhaps there are well documented examples, but even then I think the chance of that being the explanation would seem to me to be very unlikely.
As I say, this is just my interpretation based on incomplete data.
An interesting coincidence here is that my brother was going to be called Thadius (Thaddeus actually) till his older siblings vehemently objected (;
BillyJoe7,
Yes i did mean osteomyelitis, and i think it was Staph aureus. The doctors did think, at first, that it was introduced from the shot, but there was a bone cyst that had grown over a large part of the infection far from the shot location, leading them to believe that it was quite old. As i had never had a puncture injury to that part of my body, they concluded that it happened in utero. They weren’t positive of this and were quite stumped by the whole ordeal. It was apparent that the steroid shot stopped the inflammation but also allowed the infection to grow rapidly. My point was, however that self diagnoses is probably not a good idea.
Point taken
…and be wary of sites copying from a single source.
As someone on the Internet side I think surveys looking at “number of sites” are probably missing a key point.
Sure there are a lot of bad sites for medicine, but there are a lot of bad sites on pay day loans, car insurance, and any other topic with significant market interest. Most of these sites exist for link farming, or affiliate revenue, and generally rank poorly in the search results because – well Google and Bing folk see enough of the net to spot this sort of rubbish.
Tim’s advice to skeptics applies equally to doctors, use Wikipedia – and there are groups of doctors organized to bring professional expertise across a wide range of wiki articles.
Most people go to one of the first 4 results from google. Try “Ear Candling”
1) Wikipedia “Medical research has shown that the practice is both dangerous and ineffective”
2) Quackwatch.com “Why ear candling is not a good idea”.
3) altmedicine.about.com is not a fan
4) earcandling.com – okay pro-ear candling but fairly upfront about risks.
We are done the other search results fall off my screen, even folk with larger monitors don’t read that far.
I get a lot of medical information online, I often start with Google Scholar (although – really “Dose-Response” as a journal – well probably it ticks the required boxes to get included), I may add searches with “site:nhs.uk” or “site:gov” or “site:gov.uk” or even just ask google to search only cdc.gov or site:nih.gov, I have a wikipedia search option in my browser. Okay I’m a geek, and a former scientist, I grok this stuff, but so do a lot of kids. My biggest gripe is Google shuffled Google Scholar off to the sides, but I guess most people (like me) without subscriptions to the big science published get a very poor experience. Maybe with the rise of open access academic publishing they’ll bring back a new version?
So the point is not “how much bad information is out there”, but “how good is the information found”. The knee jerk reaction is “lets ban the bad stuff” which can be a free speech issue, but you only need make sure people read the good stuff.
“The knee jerk reaction is “lets ban the bad stuff” which can be a free speech issue”
Free speech shouldn’t apply to lies and deceptions.
And it shouldn’t be a matter of banning but of prosecuting.
There was a quite interesting study done on health portals and evidence a while ago… Here we are;
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1079858/?tool=pubmed (free PMC-article).
So basically national portals do a poor job of linking people to high-quality advice, as well.
Something I´d like to do is a survey of online forums or message boards, and assessing the health-advice given by users. Of course, there might be some ethical problems that need to be solved first.