http://chronicle.com/blogs/percolator/is-psychology-about-to-come-undone/29045
The plan is to replicate or fail to replicate all 2008 articles from three major Psychology journals.
ETA: http://openscienceframework.org/ is the homepage of the group behind this. It’s still in Beta, but will eventually include some nifty looking science toolkits in addition to the reproducibility project.
Nice. People taking the time to do actual science.
Also, I’m trying to set up some PB predictions on this but am having trouble wording them clearly. Any advice?
Comment
First one is here, but I’m not sure if the wording makes sense to anyone else.
Reproducing papers is a very good idea. Maybe instead of students wasting time and money doing useless "experiments" in school that don’t teach them real science, students could replicate papers? It would be hard to find full-time scientists to do these kind of studies, after all. (Of course, students aren’t as good as full-time scientists but if hundreds of them do a single study, and nobody can replicate it, then it should raise a red flag.)
Comment
I would bet this is totally impractical for most studies. In the medical sciences the cost is prohibitive and for many other studies you need permission to experiment on organisms (especially hard when humans or human tissues are involved). Perhaps it would be easier for some of the soft sciences, but even psychology studies often work with human subjects and that would require non-trivial approval.
Comment
Finding participants is already one of the biggest bottlenecks in psychology research, and it would get worse in shend’s scenario, because the supply of participants is fairly inelastic.
Comment
The supply of participants, to a great extent, is the students, so there’s something kind of cannibalistic there...
I look forward to the results of this study. Quite frankly, most soft science fields could use this sort of scrutiny. I’d also love to see how reproducible the studies done by medical doctors (as opposed to research scientists) are. Quite frankly, even the hard sciences have a lot of publications with problems, however, these erroneous results, especially if they are important to current topics of interest, are relatively quickly discovered since other labs often need to reproduce the results before moving forward.
I would add one caution. Failure to replicate an article’s results does not necessarily mean the results are wrong. It could simply mean the group trying to reproduce the results had any number of other problems.