Tuesday, July 27, 2010

The Fact Handlers: Side Note!

While I still need to write up an entry about the bizarre gulf between the research and the reporting as mentioned in this entry, in the meantime I decided to have a look at the wikipedia entry for confirmation bias.

I could make comments all day about it, but this part stood out to me right away:

Oddly, the section devoted to confirmation bias in science is rather small:

A distinguishing feature of scientific thinking is the search for falsifying as well as confirming evidence.[96] However, many times in the history of science, scientists have resisted new discoveries by selectively interpreting or ignoring unfavorable data.[96] In the context of scientific research, confirmation biases can sustain theories or research programs in the face of inadequate or even contradictory evidence;[57][97] the field of parapsychology has been particularly affected.[98] An experimenter's confirmation bias can potentially affect which data are reported. Data that conflict with the experimenter's expectations may be more readily discarded as unreliable, producing the so-called file drawer effect. To combat this tendency, scientific training teaches ways to avoid bias.[99] Experimental designs involving randomization and double blind trials, along with the social process of peer review, mitigate the effect of individual scientists' bias.

Some notes.

* Searching for falsifying evidence is a "distinguishing feature" of "scientific thinking"? But confirmation bias is described as "is a tendency for people to favor information that confirms their preconceptions or hypotheses whether or not it is true." A person engaged in confirmation bias would certainly be seeking falsifying evidence. They'd just be seeking it for views other than what they privilege.

* It's noted that there has been a history of confirmation bias by some scientists. Fair enough - as Vox Day as pointed out, scientists are not "golems animated by the spirit of the scientific method". The field of parapsychology has been singled out, and while I'm skeptical of parapsychology, I'm also skeptical of the skeptics of it. Going back to the previous point, wouldn't determined skeptics of parapsychology be open to charges of confirmation bias? And they as a group would marvelously illustrate that someone engaged in confirmation bias could be dedicated to finding "falsifying evidence" - again, so long as what was being falsified was the view they were biased against.

* It's noted that steps are taken to mitigate against confirmation bias on the part of scientists. Of course, is there scientific evidence these steps do the job they're advertised as doing? How could there be? And couldn't peer review in particular end up exaggerating the problem of confirmation bias? Peers can be biased, both individually and in a group sense.

I'm not making or trying to make any great or deep point here, really. But it seems to me that the mere acknowledgment of confirmation bias as a widespread (call it "natural") phenomenon opens up a can of worms that isn't easily closed.

1 comment:

The Phantom Blogger said...

I seen this piece on Wikipedia a while back, on the work of Drew Westen.

http://en.wikipedia.org/wiki/Drew_Westen

Political bias study

In January 2006 a group of scientists led by Westen announced at the annual Society for Personality and Social Psychology conference in Palm Springs, California the results of a study in which functional magnetic resonance imaging (fMRI) showed that self-described Democrats and Republicans responded to negative remarks about their political candidate of choice in systematically biased ways.

Specifically, when Republican test subjects were shown self-contradictory quotes by George W. Bush and when Democratic test subjects were shown self-contradictory quotes by John Kerry, both groups tended to explain away the apparent contradictions in a manner biased to favor their candidate of choice. Similarly, areas of the brain responsible for reasoning (presumably the prefrontal cortex) did not respond during these conclusions while areas of the brain controlling emotions (presumably the amygdala and/or cingulate gyrus) showed increased activity as compared to the subject's responses to politically neutral statements associated with politically neutral people (such as Tom Hanks).

Subjects were then presented with information that exonerated their candidate of choice. When this occurred, areas of the brain involved in reward processing (presumably the orbitofrontal cortex and/or striatum / nucleus accumbens) showed increased activity.

Dr. Westen said:

"None of the circuits involved in conscious reasoning were particularly engaged... Essentially, it appears as if partisans twirl the cognitive kaleidoscope until they get the conclusions they want... Everyone... may reason to emotionally biased judgments when they have a vested interest in how to interpret 'the facts.'
The study was published in the Journal of Cognitive Neuroscience, a peer-reviewed scientific journal."

Even before being peer-reviewed and published, Michael Shermer used the presentation by Dr. Westen as the basis for his July 2006 Skeptic column in the magazine Scientific American."