Continued from Part I

5) How to Analyze a Scientific Study

I don’t expect a non-scientist, or even a scientist far outside their area of expertise, to be able to do a detailed analysis of the strengths and weaknesses of a study. That is what peer-review is for. However, there are some basic rules of thumb that could give even a lay person a rough idea how seriously they should take a study. Always ask at least the following questions:

Is the study controlled in some way? Was the treatment group compared to a control group, or was the alleged effect compared to some baseline?

Is the study blinded? Were the primary measurements or assessments performed by someone who was blinded to whether or not the alleged effect is supposed to be present?

Are the outcomes being measured subjective or objective? How are they being measured? What do they really mean?

How large is the study? Studies with small numbers of subjects or measurements (less than 50 per group is a good rule of thumb) are considered small and unreliable.

Is the study an observation or an experiment? Are they just looking for some correlation  (in which it is difficult to make statements about cause and effect), or are they controlling for variables and isolating the one factor of interest?

What is the reaction of the scientific community to the study? Are experts generally critical or excited about the results? Continue Reading »

I had hoped that the advent of the internet would have a positive effect on public access to information, and perhaps it has. The problem is that it also facilitates access to misinformation. I also wonder to what extent people are availing themselves of this easy access to information (or are they just watching cat videos?).

I now frequently have the experience of being in a discussion with someone and arriving at a disagreement over a specific fact. Pre-internet we would not be able to resolve the difference, we would agree to look it up later, and usually would never do so. Now we can whip our our smartphones and within a minute or two find references to the correct fact.

Despite this there remains a disturbing gap between public perception and reality on many important issues. I discussed previously the recent survey showing significant differences between public attitudes towards certain scientific issues and the attitudes of science. The biggest difference was for the statement that it is, “safe to eat genetically modified food.” While 88% of scientists agreed with this statement, only 37% of the public did.

The gap is not limited to scientific issues, but spans the spectrum of civil issues as well. For example, 68% of Americans believe crime is worsening nationally, and 48% believe it is worsening locally, while crime has been steadily decreasing for the last two decades.

Continue Reading »

Wikipedia is now the most common general reference on the internet, and the 7th most popular site overall. It is, in a way, our collective online store of human knowledge.

Wikipedia is also a wiki, which means that anyone can theoretically write and edit articles. Wikipedia has editors, however, and they manage this process, evaluating the authoritativeness and apparent motives of editors. It is not the wild west like it used to be. (We discussed this recently on the SGU – look for the interview with Tim Farley and Susan Gerbic here.)

The question remains, however – how accurate and reliable is the information on Wikipedia? There have been several comparisons of Wikipedia to traditional references, such as the Encyclopedia Brittanica, and Wikipedia compares fairly well to such sources.

How does Wikipedia do when it comes to more technical, and specifically health care, information? This is a critical question as many people use the internet and Wikipedia specifically as a source of medical information, often to guide medical decisions or self-treatment. A recent study set out to test that very question.

Continue Reading »

It will probably come as no surprise that not being vaccinated, or being undervaccinated (missing a schedule routine vaccine) is associated with a higher risk of being infected with the disease that the vaccines are meant to prevent. This is just another way of saying that vaccines work – they reduce the risk of infection.

Often anti-vaccinationists deliberately look at such data wrong, however, in order to create a false impression. They will cite data showing that the majority of the infected are vaccinated, implying that the vaccines do not work. This argument is not valid, however. If, for example, 95% of a population is vaccinated, but 2% of them are non-responders and get the illness, while 5% are unvaccinated and 20% of them get the illness, the vaccinated will outnumber the unvaccinated among those infected – even though ten times as many unvaccinated as vaccinated get sick.

This represents what researchers are now calling “motivated numeracy” – losing basic math skills when ideologically motivated.

Continue Reading »

This is the fourth is a series of posts analyzing the claims of Don McLeroy, former chairman of the Texas School Board of Education and young Earth creationist. I recently interviewed Don on the SGU about his successful insertion into the Texas science textbook standards language requiring books to address stasis and suddenness in the fossil record and the complexity of the cell.

In parts 2 and 3 I addressed Don’s stasis and suddenness arguments. They are classic denialist fallacies – focusing on lower order details as if they call into question higher order patterns (they don’t). In this case, Don is arguing that the fact that many (not all) species display relative morphological stability in the fossil record, with episodes of (geologically) rapid speciation events, calls into question the bigger picture of the change of species over time in an exquisitely evolutionary pattern.

The former is a reflection of the tempo of evolutionary change and an artifact of the fossil record, while the latter is home-run unequivocal evidence for common descent and evolutionary change. Don has not provided any explanation for why the pattern of change we see in the fossil record presents any problems for evolutionary theory.

In this post I will address Don’ other main point, which he feels is the greatest weakness of evolutionary theory – the complexity of the cell. His premise seems to be that, if evolution were true, then evolutionary biologists should be able to provide detailed evidence for the specific evolutionary history of many biochemical pathways and cell structures. He argues that they cannot, and therefore the evidence for evolution is weak.

Continue Reading »

This week I am posting a discussion with Don McLeroy, a young Earth creationist and former chairman of the Texas Board of Education during the recent controversy over the science textbook standards. This is a follow up to an interview I did with him on the SGU.

Don has been traveling a bit this week, so our e-mail conversation has been slow, but we have had a few exchanges. For today’s post I want to simply reprint that exchange and then add a few thoughts, before I go onto new territory, which I will do in tomorrow’s post.

Here is Don’s response to my prior posts:

Steven,

I do have time for one reply.

First, you keep bringing up creationism while I do not; I am only discussing the evidence for evolution–the idea that all life is descended from a common ancestor as a result of unguided natural processes.

Continue Reading »

A comment on my recent post about Backdoor Creationism calls into question the premise that the US Constitution demands separation of church and state, and therefore religious beliefs cannot be taught in public schools. The comment reads:

The first amendment states that the federal government can neither (sic) or prohibit the exercise of religion. “separation of church and state” is just a propaganda term used by some to stave off religious nuts who use undue social pressures or indoctrination to push their beliefs to others.

Here’s a section of the first amendment.

“Congress shall make no law respecting an establishment of religion or prohibiting the free exercise thereof.”

And here’s the definition of the word “respecting” from a dictionary dated 5 years after the adoption of the Bill of Rights.

RESPECT’ING, ppr. Regarding; having regard to ; relating to.

Continue Reading »

Proponents of creationism have essentially been banned from the public school science classroom. A series of court decisions has created a clear precedence that doing so violates the Constitutional separation of church and state. However, no one really expected them to quietly go away. They have taken on a series of strategies to continue their efforts to teach their particular religious faith as science in the public schools.

They endlessly are seeking end-runs around the Constitution. First they tried “creation science,” and then “intelligent design,” but these were both transparently just religious faith crudely dressed up as science. Now they are still trying “teach the controversy” and “teach the strengths and weakness of evolution.” Both have had some limited success, but I predict will also eventually die a legal death.

Another strategy is to simply ignore the law and teach creationism anyway. In highly fundamentalist Christian communities there’s no one to blow the whistle, and no one to listen. A recent survey found that 13% of public schools teach creationism outright, while 60% avoid controversy by promoting neither evolution nor creationism. Only 28% teach evolution as the unifying theory of biology (as it should be taught).

Continue Reading »

“Bottom line, at the end of the day, we want our kids to be exposed to the best facts. Let’s teach them about the big bang theory, let’s teach them about evolution…”

Not a bad sentiment so far. I don’t think I would have used the term, “best facts.” It’s a bit awkward, and more importantly science is not just about facts, it’s about how we know what we know, and the interaction of facts and data with hypotheses and theories.

This is not quibbling. A public figure with responsibilities toward public science education should have a thorough and nuanced understanding of science education.

Continue Reading »

Do you think you could communicate a scientific concept to a general audience using only the 1,000 most commonly used words? A thousand words sounds like a lot, but is it? Clearly this would not allow for the use of specialized scientific jargon, which is the point. A good science communicator should be able to translate complex science into everyday language, and use accessible analogies to make those concepts understandable.

This is something I do everyday, and not just on my blog and other social media. As a physician I have to communicate sometimes complex medical information to patients and their families. To make things more challenging my patients vary from being other physicians, health care workers, scientists or academics, to lacking a high school education or even not being a native English speaker.

Communicating to the public effectively means targeting a broad range of background knowledge. An effective science communicator should be interesting to experts while being understandable to a novice. Another challenge is to make scientific concepts simple without being oversimplified.

Continue Reading »