Nov 29 2016

Civic Online Reasoning

nuclearflowersA recent study adds some empirical data to the current discussions regarding online information. This Stanford University study looked at 7,804 student responses across 12 states, divided among middle school, high school, and college students. The goal of the study was to see if these students could distinguish reliable sources of information from fake or unreliable sources.

Their conclusion?

Overall, young people’s ability to reason about the information on the Internet can be summed up in one word: bleak.

Although students grew up in the internet and social media age, and are very skilled at using online resources, they apparently have not developed the skills to critically evaluate the information they are finding online.

The authors echo what I and many others have pointed out, that while the internet is a great source of information, it is largely a source without editorial filters. As I recently discussed, this has led to a range of outlets including high quality journalism, low quality journalism, advocacy sites, biased sites, advertising, opinion, and fake sites that exist only to drive clicks. Since you no longer need a large infrastructure, or years to build up a reputation and circulation, in order to publish articles that then get shared on social media as news, every kind of information is jumbled together and it is up to the reader to discriminate.

The authors looked at five tasks for each school level that they felt was appropriate for that level. Here are some example results:

For middle school students they showed them the front page of and asked them if specific articles were an advertisement or not. The students were able to recognize a real news article as news, and a straightforward ad as an ad. However, more than 80% of students thought that a “sponsored content” article (that was clearly labeled as such) was real news and not an advertisement. Many students noted that it was sponsored content but still thought it was news, indicating they don’t know what sponsored content is.

One of the tasks for the highschool students was to evaluate a photo uploaded to Imgur. The picture of flowers shown above was offered as evidence of the effect of radiation at the Fukushima nuclear plants. Less than 20% of the students appropriately questioned the utility of the photo as evidence and the source of the photo. Obviously the photo is only a close up of flowers, and there is nothing in the photo itself to tie it to Fukushima.

As an example of the college-level tasks, they showed the students a tweet with a link to regarding a poll about gun regulations conducted by the Center for American Progress, and asked how reliable this information is. The question was whether or not the students would recognize that the two sources of information are both liberal advocacy groups with a clear political agenda regarding gun regulations.

Let me give their full explanation of the results since it is a little involved:

We piloted this task with 44 undergraduate students at three universities. Results indicated that students struggled to evaluate tweets. Only a few students noted that the tweet was based on a poll conducted by a professional polling firm and explained why this would make the tweet a stronger source of information. Similarly, less than a third of students fully explained how the political agendas of and the Center for American Progress might influence the content of the tweet. Many students made broad statements about the limitations of polling or the dangers of social media content instead of investigating the particulars of the organizations involved in this tweet.

An interesting trend that emerged from our think-aloud interviews was that more than half of students failed to click on the link provided within the tweet. Some of these students did not click on any links and simply scrolled up and down within the tweet. Other students tried to do outside web searches. However, searches for “CAP” (the Center for American Progress’s acronym, which is included in the tweet’s graphic) did not produce useful information. Together these results suggest that students need further instruction in how best to navigate social media content, particularly when that content comes from a source with a clear political agenda.

The students have some elements of critical analysis, but most did not fully recognize the roll of political bias and less than half the students clicked the link to go to the original source (remember, they were specifically asked to evaluate the credibility of the source).

Overall I agree with the authors, the results are “bleak.”

When reading the actual student responses, I was often struck by how familiar their reasoning was to what I witnessed in my daughters as they are progressing through school. I could see that the student was reverting to responses that they were probably taught in school – doing the task they thought they were supposed to do, and not truly understanding the task or thinking clearly about it.

This to me suggests that students often learn the format of scholarship in school, but don’t truly understand it, and therefore cannot apply it to the real world. They were reaching for some task they were taught in school and then applied it even when it was not appropriate.

This is an area where education can have a clear impact. Overall I think the educational system needs to do a better job of getting students to truly own key intellectual skills and concepts, rather than just go through the motions. Specifically, however, I think that students should be directly taught how to evaluate the credibility of information online and elsewhere. This is now a critical life skill, and obviously students are not just absorbing it or extrapolating these skills from other contexts.

Imagine a dedicated course that taught civic scientific and critical thinking literacy. This should be given in middle school, and again in high school. This could easily be a fun and engaging course, that focuses on applying knowledge to the real world, self-learning, evaluating information, and thinking critically. This one course could arguably be more valuable to the average student than the rest of their curriculum combined.

Clearly something like this is needed. We now have some objective data to support this conclusion.

16 responses so far

16 thoughts on “Civic Online Reasoning”

  1. Michael Finfer, MD says:

    When my daughter was, I think, in middle school, her class was given an assignment to read a web site about tree octopuses and to be prepared to discuss it in class the next day.

    The site was actually quite good, filled with photos of multicolored octopuses in trees.

    Of course, the point was to get the kids to understand that not everything you see on the internet is true. I thought that it was quite a useful assignment. Perhaps more of that should be going on in the schools.

    Here’s the link:

  2. TheGorilla says:

    I feel like this is in the same category of financial courses where students would learn basic concepts about compounding interest, loans, budgeting, whatever. Life skill that will never end up being implemented 🙁

    I went to a way-too-expensive private high school, and we had actual courses for both library use (ie how to conduct research, use databases, etc) and critically evaluating the content of articles/web pages/papers. We also had two required critical thinking courses. It was the same sort of story in university – basically every 100 level humanities course (humanities only, basically, is another interesting phenomenon) had one of the librarians give a lecture on the same sort of thing (yea, that got old VERY fast).

    The importance of this is well-understood by universities and pretentious high schools (study itself is a demonstration of this), but for the majority of school districts I just do not see it possible to integrate this kind of thing. Between lack of resources and the absurd degree to which quality is evaluated quantitatively… feels like a pipe dream.


    I wonder how long until Dr Novella plays one of those fake news tricks on his readers?

  3. Jeffrey says:

    Sadly, I believe a lot of middle aged adults also have difficulty evaluating on-line content.

  4. carbonUnit says:

    If talking to your local school board, as we all should, don’t call it “critical thinking” if the school board is too conservative. In 2012, Texas Republicans had a party plank against critical thinking!
    (Dare to be stupid! – I love that Weird Al song more and more!)

    I guess I would use the Carl Sagan terminology: everyone needs a functional baloney detector.

  5. Kabbor says:

    I almost feel that the internet at large needs some kind of peer review process. Only fuzzy ideas as to how this would be implemented. Maybe it would be a browser-based overlay that is run in a similar capacity to Wikipedia. This would be an opt-in method, but might be a means of giving context to the commonly trod sites to those who are interested and care about such things.

  6. tmac57 says:

    carbonUnit- “I guess I would use the Carl Sagan terminology: everyone needs a functional baloney detector.”
    My guess is that those same Texas Republicans would interpret that as a ‘Politically Correct’ attempt to cram veganism down the school kids throats (they really, really like that phrase too). By the way, I live in Texas, so I know from whence I speak.

  7. RickK says:

    I recently read Rebecca Skloot’s excellent “The Immortal Life of Henrieta Lacks”. Henrieta Lacks was a poor African American woman who died young of a very aggressive cancer. But a sample of her tumor, collected without her knowledge, turned out to be the origin of the first and most widespread “immortal” human cell line. Her cells (called HeLa) have been replicated by e billions and used in research all over the world for decades. Henrietta has had periods of notoriety or infamy whenever someone writes another article about her – whether in “The Lancet” or in “The National Enquirer”.

    A major theme of the book is how little Henrietta’s under-educated family understood about her cells. I was struck by how important education is in building critical thinking skills. In one poignant scene, Henrietta’s daughter pulls page after page out of a file of articles people had given her over the years, challenging Rebecca with each one as if they were all equal. An article from “National Geographic” or “Nature” about HeLa cells in research was no different than an article about HeLa-cell zombies from the “Weekly World News”. The daughter and other family members had zero ability to filter and separate the sensational from the scientific and the false from true. And these were adults, some quite clever, but who’d never benefitted from early education in critical thinking.

    And that, in no small part, explains the 2016 election results.

  8. GrahamH says:

    I make the differentiation betwen pupil and student. A pupil is handed information, a student has to seek out the information. Gradual continuum of course.
    I did a teacher training course where I made the same point in an essay. The lecturer marked the intial few sentences in red question marks, the followinf sentences (that explained the comment) repealing the q marks. In other words, the lecturer himself fidn’t bother truing to understand the comment and didn’t take the effort of reading on to see the completed argument. Not a good example of how to process information.
    I had several run ins with lecturers on that course.

    I’m currently monitoring a couple of 16 yo interns. It’s quite scary the lack of critical thinking skills, as well as basic numeracy and literacy.

  9. GrahamH says:

    I feel a lot of the root cause is due to ‘paternalism’. Children grow up followong the lead and instruction of the adults (I agree on the necessity of that) and I would suggest most don’t break away from that mode. It’s difficult.
    Speak with an air of confidence/authority more people will listen than if you ask them to think for themselves.

  10. Teaser says:

    Chris Hedges has written quite a bit about the decline of the USA educational system. Here is a quote from a 2012 article. I appreciate that he doesn’t mince words.

    A nation that destroys its systems of education, degrades its public information, guts its public libraries and turns its airwaves into vehicles for cheap, mindless amusement becomes deaf, dumb and blind. It prizes test scores above critical thinking and literacy. It celebrates rote vocational training and the singular, amoral skill of making money. It churns out stunted human products, lacking the capacity and vocabulary to challenge the assumptions and structures of the corporate state. It funnels them into a caste system of drones and systems managers. It transforms a democratic state into a feudal system of corporate masters and serfs.

    Here is Chris’s bio:

  11. steve12 says:

    “I could see that the student was reverting to responses that they were probably taught in school – doing the task they thought they were supposed to do, and not truly understanding the task or thinking clearly about it.”

    This rigidity is specifically taught for the purpose of taking standardized tests. This testing regime is horrific for critical thinking skills. Instead of preparing kids for the wide open world, we prepare them for specific and fixed-circumstance tests.

    Add to this that critical, creative, or adaptive thinking skills are, psychometrically speaking, harder to reliably test for. The solution? Don’t test for that, and therefore don’t teach that.

    And I’m not saying that we shouldn’t assess kids – we should. But that one useful goal is ALSO thwarted by teaching these kids to take tests. We can’t assess what the kids really know how to do when the schools have hopelessly conflated the objects of measurement with practice effects and specific test taking abilities. Teaching to the tests makes them almost uninterpretable, so they’re now even useless for that (save for the tail performers of course).

  12. tmac57 says:

    Steve12 – ” I could see that the student was reverting to responses that they were probably taught in school – doing the task they thought they were supposed to do, and not truly understanding the task or thinking clearly about it.”

    Since you reiterated this from the article, it triggered a thought. I worked in telecom for nearly 40 years, and had to learn new technologies constantly since it was changing rapidly during that time. Now, I am not good at rote learning, and the only way I could remember the new material was to integrate it firmly into real world models. In other words, I really had to understand how it worked, rather than the dry facts surrounding it, and that took a bit more study.
    The courses that the company taught were pretty good, but it was a lot to digest and retain, and I noticed that many of my coworker’s came out of those classes with passing grades (it was often pass/fail and you had to pass to keep your job), but almost immediately had trouble utilizing in the field what they had learned.
    A situation that more people might identify with is setting up and using home theater equipment (TV/DVD/receiver/ROKU/broadband/etc.) For people who are tech minded, they take the time to fully understand the equipment, and those base understandings can easily be generalized to newer equipment (often you don’t even need instructions) , since the setup has a logic to it. But, if you try to explain it to someone who won’t learn/follow the inputs and outputs of the cables, and what they do, and the signal flow between components, and what the remotes are telling all of those thing to do in relation to each other, then you end up with someone completely baffled and helpless when they need to move the equipment, or even if someone changes an input by pressing a button.
    That’s a long winded way of saying that understanding what information and facts mean in an integrated way (and this takes effort), is as important as merely being exposed to them. Probably more so.

  13. steve12 says:


    “That’s a long winded way of saying that understanding what information and facts mean in an integrated way (and this takes effort), is as important as merely being exposed to them. ”

    Right – what you describe is that deeper level of comprehension that is simply not valued. If you really understand the whole and the relationship of the details (‘facts’) to that whole you can use your knowledge in novel ways and adapt them to new situations.

    That seems to not be stressed in schools for the reasons I write above – but this is a great real world example of what I mean. I’d go further to guess that there’s generalization of your skill in acquiring other unrelated knowledge and expertise – that’s what I’d like to see kids doing.

  14. dcardani says:

    One thing that bothered me about their findings were that in the section on the polling question, they listed this as being “mastery” of the task:

    > The polling information which the tweet references was collected by Public Policy Polling, which appears to have a fairly strong accuracy record, though with a Democratic bent (e.g., Wall Street Journal article:

    While the information does happen to be accurate, the source they pulled it from (Rupert Murdoch’s news paper) is itself biased in the other direction. So they cited a source, but it was no better than the source they were questioning, essentially.

  15. Damon says:

    I’ve been saying this for years! It’s time a Science text book adopts a new chapter that “taught civic scientific and critical thinking literacy”; it’d come right after the scientific method chapter. I am a high school science teacher; if I put together a rough draft, will someone help edit and help push it for publication? I’m in if you’re in!

  16. MaryM says:

    I just came across this piece, which I find baffling. Any chance Steve could take a look at it?

Leave a Reply