Author: Bruce Bower / Source: Science News
What started out a few years ago as a crisis of confidence in scientific results has evolved into an opportunity for improvement. Researchers and journal editors are exposing how studies get done and encouraging independent redos of published reports. And there’s nothing like the string of failed replications to spur improved scientific practice.
That’s the conclusion of a research team, led by Caltech economist Colin Camerer, that examined 21 social science papers published in two major scientific journals, Nature and Science, from 2010 to 2015. Five replication teams directed by coauthors of the new study successfully reproduced effects reported for 13 of those investigations, the researchers report online August 27 in Nature Human Behavior. Results reported in eight papers could not be replicated.
The new study is an improvement over a previous attempt to replicate psychology findings (SN: 4/2/16, p. 8). But the latest results underscore the need to view any single study with caution, a lesson that many researchers and journal gatekeepers have taken to heart over the past few years, Camerer’s team says. An opportunity now exists to create a scientific culture of replication that provides a check on what ends up getting published and publicized, the researchers contend.
Still, the new study reveals a troubling aspect of successful experimental redos. Camerer’s team found that for repeat studies that panned out, which included four to five times as many participants as originally studied, the statistical strength to detect actual effects was weaker than reported for the initial investigations. In other words, the best replications — which exceeded initial studies in their ability to detect actual effects — were only partially successful.
One reason for that trend is that scientific journals have tended not to publish studies that disconfirm previous findings, leaving initial findings unchallenged until now, says study coauthor and psychologist Brian Nosek of the University of Virginia in Charlottesville. Even the most prestigious journals have often published results that garner lots of scientific and media attention but that could easily have occurred randomly, he says.
On the plus side, the new report appears as such practices are changing. “The social and behavioral sciences are in the midst…
The post ‘Replication crisis’ spurs reforms in how science studies are done appeared first on FeedBox.