It appears that Google has removed all Natural News content from their indexing. This means that Natural News pages will not appear in organic Google searches.

This is big news for skeptics, but it is also complicated and sure to spark vigorous discussion.

For those who may not know, Mike Adams, who runs Natural News, is a crank conspiracy theorist supreme. He hawks snake oil on his site that he markets partly by spreading the worst medical misinformation on the net. He also routinely personally attacks his critics. He has launched a smear-campaign against my colleague, David Gorski, for example.

A few years ago Adams put up a post in which he listed people who support the science of GMOs to the public, comparing them to Nazis and arguing that it would be ethical (even a moral obligation) to kill them. So he essentially made a kill-list for his conspiracy-addled followers. Mine was one of the names on that list, as were other journalists and science-communicators.

In short Adams is a dangerous loon spreading misinformation and harmful conspiracy theories in order to sell snake oil, and will smear and threaten those who call him out. He is an active menace to the health of the public.

Adams is a good example of the dark underbelly of social media. It makes it possible to build a massive empire out of click-bait and sensationalism.

Despite this, there are two ways you can look at Google’s actions. The first is that Google is a private company selling a product. That product is information – curated search results to help users find information on the vast web. Google is free to use whatever methods they deem optimal to provide the best product to their customers.

There are many criteria that Google can choose to use to determine which results will best serve the needs and desires of their customers. Popularity is not the only criterion. Google became the search-engine giant because they figured out ways to provide not just the popular sites, but the best sites, with quality information matching what the user was looking for.

It is also important to note that there is an entire industry of search engine optimization (SEO) which seeks to game Google’s algorithms, to get websites higher in search results. (Much of SEO is standard and works with search companies, but there is a dark side that goes beyond this to game the system.) SEO is essentially about decreasing the value of Google’s product by forcing a relatively lower quality site higher into the search results by exploiting what is known about Google’s engine.

Google has every right to take whatever steps they deem necessary to preserve the value of their product by frustrating SEO attempts, and by choosing the characteristics of websites that will be favored in their searches. Again, their is nothing special about popularity that says it should be the sole or dominant criterion. Sometimes that may mean manually making decisions and altering search results.

Natural News is an objectively low quality site from the perspective of the academic and scientific quality of the information it provides. If someone is searching for medical information, Google may determine that they will best serve their customers by providing legitimate medical information from trusted sources, rather than popular misinformation. They can also value academic sources over commercial sources.

Google has yet to explain its decision, and even if it was a deliberate decision (it probably was). There are many technical reasons why they would have delisted Natural News. Perhaps the site was aggressively using questionable SEO and the most effective countermeasure was to simply delist the entire domain. They may simply have felt that it was the most effective way to improve the quality of Google searches for medical information.

The other perspective views Google more like a utility. While they are not a government site (and therefore no first amendment issues), Google is by far the most popular portal to the web, which is now an invaluable general resource. Private utility companies are regulated by the government (or in some countries even nationalized) because they provide an essential service to the public. If Google is viewed as an essential utility, you can argue that they should not discriminate in this way.

There is also the slippery slope argument. If Google uses their own assessment of quality to delist entire domains from their organic searches, ideological bias can slip into their quality assessments. Some would rather deal with low quality sites coming up in their searches rather than have Google scrub the searches for them, and therefore be subject to the ideological biases of a private company not subject to any oversight.

While I might agree with their decision regarding Natural News, I might not agree with decisions they make in the future. The company itself might be subject to a take-over by someone with an ideological agenda.

Of course this is no different than any media company.

Where do I come down on this issue? I am a bit conflicted. I can see the arguments on both sides. I do not see any absolute rights here, and take a mostly risk vs benefit approach.

In general I favor legitimate attempts at quality control for information, and am not persuaded by arguments of censorship when the real goal is quality. But I also recognize then when you are dealing with a company and service as large as Google, you have to be especially careful that quality control does not slip into ideological bias. We have to then ask – what are the checks and balances?

Transparency is one check that can be extremely effective. Google, however, as a private company in a competitive market tends to be rather opaque regarding its algorithms. Its algorithms are a legitimate corporate secret, carefully guarded.

Still, with big decisions like this, a frank explanation would be nice. It is also possible that the courts may get involved, which tends to create transparency through discovery. That is not an ideal solution, because it favors those with money.

Google does appear to be experimenting with various methods to maintain the quality of its product in the current world of fake news, misinformation, and false equivalency. I applaud them for recognizing the problem and being willing to take dramatic steps to improve them. I don’t think we have arrived at the optimal solution, but we can get there if we are willing to experiment, and be open and transparent about the results.

Update: Official word from Google:

• “We don’t comment on individual sites, but if we find that a site violates one or more of our Webmaster Guidelines we may take manual action against it. For webmasters who have questions about their own sites, our Webmaster team provides support through platforms such as the Webmaster Forums. Once a site has remedied the problem, the webmaster can submit the site for reconsideration.”

It seems that Natural News violated a webmaster guideline and manual action was taken against it.