| Why aren’t these “failed” experiments published? What’s the definition of a failed experiment anyway.
I think that if the scientific question is well formed and well motivated AND the methods sound and appropriate for addressing the question, then whatever the result may be, this seems like a good experiment and one that should be published. Sent from my iPhone Ken W. Grant, PhD Chief, Scientific and Clinical Studies National Military Audiology and Speech-Pathology Center (NMASC) Walter Reed National Military Medical Center Bethesda, MD 20889 Office: 301-319-7043 Cell: 301-919-2957 The view that RRs will stifle progress is both true and false. While the increased load of advanced registration and rigidity in methods would, as Les points out, become burdensome for most of our basic work, there is another side to this. This is not a matter of morals (hiding a bad result, or fabricating a good result) or how to do our experiments. It’s a matter of the standards of *publication*, which you will notice was the scope of Tim’s original call to action. In general, we only ever read about experiments that came out well (and not the ones that didn’t). If there is a solution to that problem, then we should consider it, or at least acknowledge that some solution might be needed. This is partly the culture of scientific journals, and partly the culture of the institutions that employ us. There's no need to question anybody's integrity in order to appreciate some benefit of RRs.
Think for a moment about the amount of wasted hours spent by investigators who repeat the failed methods of their peers and predecessors, only because the outcomes of failed experiments were never published. Or those of us who cling to theories based on initial publications of work that later fails replication, but where those failed replications never get published. THIS stifles progress as well. If results were to be reported whether or not they come out as planned, we’d have a much more complete picture of the evidence for and against the ideas. Julia's story also resonates with me; we've all reviewed papers where we've thought "if only the authors had sought input before running this labor-intensive study, the data would be so much more valuable."
The arguments against RRs in this thread appear in my mind to be arguments against *compulsory* RRs for *all* papers in *all* journals, which takes the discussion off course. I have not heard such radical calls. If you don’t want to do a RR, then don’t do it. But perhaps we can appreciate the goals of RR and see how those goals might be realized with practices that suit our own fields of work.
Matt
|