Next time you read a psychology study that makes you scratch your head, trust your instincts. A new yearlong effort to reproduce 100 studies from 3 leading journals has found that over half the findings did not hold up when repeated. (Oops!)
Arthur Caplan, medical ethicist of the NYU Langone Medical Center and host of the "Everyday Ethics" podcast, points to two driving factors: an academic and media climate that favors big, flashy conclusions, and a peer review system that is not as rigorous as it should be.
"Psychologists get more and more rewards for coming up with cool and hip things," Caplan explained. "We love to believe we're unlocking the deep secrets of our psyche...so I think the hype gets rewarded. You get poor studies published, people respond to that."
Another problem is how the studies are vetted before they go to publication. Most major journals use a peer review system, where other scientists review that work for free. But these reviewers are under more strain than ever before.
"People are busier and they don't read the papers carefully, and they really don't pay attention to the statistics, which is where these things rise and fall," Caplan said.
Caplan says the solution may be to move toward a system where people are paid to review studies, rather than doing it for free. And, of course, for the media to exercise more restraint when it comes to covering studies that seem to be a little too wild to be true.
"Too many things are slipping through the cracks," he said.
To hear more from Arthur Caplan, tune in to Boston Public Radio above.
