Australasian Science: Australia's authority on science since 1938

Scientists Don’t Turn a Blind Eye to Bias

The full text of this article can be purchased from Informit.

Scientific journals should insist on more robust experimental processes, according to biologists who reviewed nearly 900,000 experiments.

The team from The Australian National University found that non-blind experiments – where scientists knew which samples they were recording – averaged a 27% stronger result than blind trials. However, their review suggests that less than one in four experiments used blind data recording.

“We found that non-blind papers tended to exaggerate differences between the experimental group and the control group,” said lead researcher Dr Luke Holman.“For example, a non-blind trial of a new drug might conclude that it is way more effective than a placebo, when in fact the drug’s true effect is rather modest, simply because the researchers’ expectations biased the results.”

The research, published in PLOS Biology, comes at a time when experimental processes are under the microscope following increased levels of retractions and a review of peer review procedures by some journals.

The team used automated “data mining” to analyse nearly 900,000 papers from the PubMed life sciences database. They also compared 83 pairs of evolutionary biology papers on similar topics in which the data were collected blind in only one of each pair.

The team found that non-blind studies rejected the null hypothesis more strongly. “Non-...

The full text of this article can be purchased from Informit.