Saturday, February 25, 2017

Non-Replicable "Science": Post Truth and Post Empirical

The business of Grants, Prestige, and Falsifiability: Two out of three, anyway.
Most scientists 'can't replicate studies by their peers'
Science is facing a "reproducibility crisis" where more than two-thirds of researchers have tried and failed to reproduce another scientist's experiments, research suggests.

This is frustrating clinicians and drug developers who want solid foundations of pre-clinical research to build upon.

From his lab at the University of Virginia's Centre for Open Science, immunologist Dr Tim Errington runs The Reproducibility Project, which attempted to repeat the findings reported in five landmark cancer studies.

"The idea here is to take a bunch of experiments and to try and do the exact same thing to see if we can get the same results."

You could be forgiven for thinking that should be easy. Experiments are supposed to be replicable.

The authors should have done it themselves before publication, and all you have to do is read the methods section in the paper and follow the instructions.

Sadly nothing, it seems, could be further from the truth.

After meticulous research involving painstaking attention to detail over several years (the project was launched in 2011), the team was able to confirm only two of the original studies' findings.


[...]

According to a survey published in the journal Nature last summer, more than 70% of researchers have tried and failed to reproduce another scientist's experiments.

Marcus Munafo is one of them. Now professor of biological psychology at Bristol University, he almost gave up on a career in science when, as a PhD student, he failed to reproduce a textbook study on anxiety.

"I had a crisis of confidence. I thought maybe it's me, maybe I didn't run my study well, maybe I'm not cut out to be a scientist."

The problem, it turned out, was not with Marcus Munafo's science, but with the way the scientific literature had been "tidied up" to present a much clearer, more robust outcome.

"What we see in the published literature is a highly curated version of what's actually happened," he says.

"The trouble is that gives you a rose-tinted view of the evidence because the results that get published tend to be the most interesting, the most exciting, novel, eye-catching, unexpected results.

"What I think of as high-risk, high-return results."


The reproducibility difficulties are not about fraud, according to Dame Ottoline Leyser, director of the Sainsbury Laboratory at the University of Cambridge.

That would be relatively easy to stamp out. Instead, she says: "It's about a culture that promotes impact over substance, flashy findings over the dull, confirmatory work that most of science is about."

She says it's about the funding bodies that want to secure the biggest bang for their bucks, the peer review journals that vie to publish the most exciting breakthroughs, the institutes and universities that measure success in grants won and papers published and the ambition of the researchers themselves.

"Everyone has to take a share of the blame," she argues. "The way the system is set up encourages less than optimal outcomes."
Well, that actually IS fraud. It is fraud which is systemically induced and supported. And that means that the scientists are complicit and the data is fundamentally useless. A giant taxpayer rip-off.

Best Quotes:
"Replication is something scientists should be thinking about before they write the paper," says Ritu Dhand, the editorial director at Nature.
[...]
But we need to be bolder, according to the Edinburgh neuroscientist Prof Malcolm Macleod.

"The issue of replication goes to the heart of the scientific process."

"Without efforts to reproduce the findings of others, we don't know if the facts out there actually represent what's happening in biology or not."

Without knowing whether the published scientific literature is built on solid foundations or sand, he argues, we're wasting both time and money.

"It could be that we would be much further forward in terms of developing new cures and treatments. It's a regrettable situation, but I'm afraid that's the situation we find ourselves in."
Today's "science" is on the rocks. You can bet on two things: Whatever is published in science journals is likely to be neither complete nor valid; the last line in the study goes something like this: "more study is needed".

4 comments:

Zach said...

Isn't this good news though? It means that scientific research is being investigated for self-correction. If a result is not replicated, then we should remain skeptical of the results. But when it is replicated, say in 30% of the case per your article (According to a survey published in the journal Nature last summer, more than 70% of researchers have tried and failed to reproduce another scientist's experiments.)

Note that the 70% here could actually be much lower, or higher. The sentence means that 70% of scientists, the people, were not able to reproduce at least 1 research result. It does not tell us anything about how often they are successful at reproducing the research.

Regardless of the actual number, which would be interesting to find, it means that the results that do get reproduced are strengthen, while those that are not reproduced are weaken. The scientific method continues to be used to improve our understanding of the world.



What's the problem?



Obviously, the only problem is that the OP author, Stan, is an emotional right-wing nut job who thins that he follows the scientific method, but really just spit out what he feels is correct. That's why the right rail of this piece of shit of a blog has challenges to evolutionists on the side. The scientific method was used to study evolution and concluded several things about how we, as people, as homo sapiens, as primates, as mammals ... evolved. It's not even a debate, it's not even controversial, but Stan knows best. Stan knows that evolution is wrong.

And he preferred to post comic strips, instead of replying to Shadow, and instead of my post regarding relevant quotes from The 13th, documented in a press article as well. Serious stuff about how Black people were, and still are, targeted by law enforcement more than non-Blacks.

But Stan did replied to some stupid troll called Yck (did he choose a name close mine purposely? do you often get idiots like that here? are Steven and Robert also trolls like him?) Because, you know, that is some important shit right there. You must point out that this person is stupid. It's so important.

Almost as important as spitting out White Supremacist ideas and blaming Leftists for everything that goes wrong.

Stan said...

Since you feel so strongly that this is a "piece of shit blog", you will be protected from having to comment here any longer. Hope that helps with your Left Wing, Marxist, Self-Righteous worldview.

Adios

Stan said...

Without knowing for certain, I think this takes care of yck and shadow, too, since they're all the same person.

Robert Coble said...

I think Zach (Yck? Shadow? Anonymous?) has served up the best illustrations of the various Ad Hominem fallacies: Abusive, Tu Quoque, Circumstantial, and Guilt by Association. S/he/it should be given an award commensurate with s/he/its accomplishment:

"Secret King" Gamma RULES!

CRAP! (Sorry, but I got the impression from some idiot in the comment boxes that this was the "piece of shit" blog, so a reference to excrement in some form was obligatory in all comments.) I don't get my comment written out and you've already bid s/he/it Adios.