Wednesday, February 20, 2013

Doing Peer Review Wrong

In science the concept of peer review is essential to establishing the validity of any working hypothesis. For a scientific discovery to be accepted, the data that supports it must be subjected to rigorous and thorough criticism by others working in the same field. Once a consensus is reached regarding the discovery, it will either be added to the existing canon of scientific fact or discarded as an anomalous finding. At least, that's the ideal of how it's supposed to work.

Back in November, I reported that a group of geneticists claimed to have sequenced Bigfoot DNA and concluded that the fortean primate was an ape/human hybrid. The researchers claimed that they would soon subject their findings to peer review, but as it turns out rather than following accepted scientific procedures and publishing the findings in a standard journal, the researchers started their own. This is pretty close to the opposite of actual peer review, since it's not like there aren't other journals out there publishing articles on biology and genetics. See, when you review your own findings, that's pretty much the definition of doing peer review wrong. And clicking through the journal's website reveals other obvious problems.

Please note the helpfully labeled slideshow cycling through on the front page — pollen; ladybug; eagle; h2o — as if to say: "Don't worry, this place is legit. Look, stock photos. Also this is a ladybug because you probably didn't know that."

The site also claims to be "open access," but charges 30 bucks to access the Bigfoot genome paper. It bears mentioning that the Bigfoot genome paper, at the time of this posting, is also the only paper in Vol. 1, Issue 1 of the new journal. Seeing as "open access" clearly does not mean what these researchers think it means, you'll forgive us if we remain skeptical when they say their data "conclusively proves that the Sasquatch exist as an extant hominin"; if we had to guess, we'd say that "conclusively proves that the Sasquatch exist as an extant hominin" doesn't mean what the researchers think it means, either.

Ars Technica has scored a copy of the paper, and is working with someone with relevant genomics experience to analyze the DNA sequences presented therein. So far, says Ars science editor John Timmer, "much of the paper speaks for itself — and it says some very strange things."

It's also rather telling that these researchers are looking to make a quick buck off their Bigfoot paper - I don't even get $30 for a whole book, and it fact you could buy both of the titles I have in print for that amount of money. It's good to see that somebody has taken the plunge, though, and will be conducting an actual review of the research. My guess from Timmer's initial statement is that the "Bigfoot DNA analysis" is likely so poorly done that the researchers were unable to get it accepted by any real scientific journal and were left with no other choice but to found their own. Clearly, that does not bode well for the legitimacy of their research.

Technorati Digg This Stumble Stumble

No comments: