Subject: Re: [AUDITORY] Registered reports From: Ken Grant <ken.w.grant@xxxxxxxx> Date: Sat, 9 Jun 2018 01:18:48 -0400 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>--Apple-Mail-CF5FE5CC-B3CE-4090-8E61-27EBEB996A52 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Why aren=E2=80=99t these =E2=80=9Cfailed=E2=80=9D experiments published? Wha= t=E2=80=99s the definition of a failed experiment anyway.=20 I think that if the scientific question is well formed and well motivated AN= D the methods sound and appropriate for addressing the question, then whatev= er the result may be, this seems like a good experiment and one that should b= e published.=20 Sent from my iPhone Ken W. Grant, PhD Chief, Scientific and Clinical Studies National Military Audiology and Speech-Pathology Center (NMASC) Walter Reed National Military Medical Center Bethesda, MD 20889 kenneth.w.grant.civ@xxxxxxxx ken.w.grant@xxxxxxxx Office: 301-319-7043 Cell: 301-919-2957 > On Jun 9, 2018, at 12:48 AM, Matthew Winn <mwinn2@xxxxxxxx> wrote: >=20 > The view that RRs will stifle progress is both true and false. While the i= ncreased load of advanced registration and rigidity in methods would, as Les= points out, become burdensome for most of our basic work, there is another s= ide to this. This is not a matter of morals (hiding a bad result, or fabrica= ting a good result) or how to do our experiments. It=E2=80=99s a matter of t= he standards of *publication*, which you will notice was the scope of Tim=E2= =80=99s original call to action. In general, we only ever read about experim= ents that came out well (and not the ones that didn=E2=80=99t). If there is a= solution to that problem, then we should consider it, or at least acknowled= ge that some solution might be needed. This is partly the culture of scienti= fic journals, and partly the culture of the institutions that employ us. The= re's no need to question anybody's integrity in order to appreciate some ben= efit of RRs.=20 >=20 > Think for a moment about the amount of wasted hours spent by investigators= who repeat the failed methods of their peers and predecessors, only because= the outcomes of failed experiments were never published. Or those of us who= cling to theories based on initial publications of work that later fails re= plication, but where those failed replications never get published. THIS sti= fles progress as well. If results were to be reported whether or not they co= me out as planned, we=E2=80=99d have a much more complete picture of the evi= dence for and against the ideas. Julia's story also resonates with me; we've= all reviewed papers where we've thought "if only the authors had sought inp= ut before running this labor-intensive study, the data would be so much more= valuable." >=20 > The arguments against RRs in this thread appear in my mind to be arguments= against *compulsory* RRs for *all* papers in *all* journals, which takes th= e discussion off course. I have not heard such radical calls. If you don=E2=80= =99t want to do a RR, then don=E2=80=99t do it. But perhaps we can appreciat= e the goals of RR and see how those goals might be realized with practices t= hat suit our own fields of work. >=20 > Matt=20 >=20 >=20 > -------------------------------------------------------------- > Matthew Winn, Au.D., Ph.D. > Assistant Professor > Dept. of Speech & Hearing Sciences > University of Washington >=20 --Apple-Mail-CF5FE5CC-B3CE-4090-8E61-27EBEB996A52 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D= utf-8"></head><body dir=3D"auto">Why aren=E2=80=99t these =E2=80=9Cfailed=E2= =80=9D experiments published? What=E2=80=99s the definition of a failed expe= riment anyway. <div><br></div><div>I think that if the scientific quest= ion is well formed and well motivated AND the methods sound and appropriate f= or addressing the question, then whatever the result may be, this seems like= a good experiment and one that should be published. <br><br><div id=3D= "AppleMailSignature">Sent from my iPhone<div><span style=3D"font-size: 13pt;= ">Ken W. Grant, PhD</span></div><div><span style=3D"font-size: 13pt;">Chief,= Scientific and Clinical Studies</span></div><div><span style=3D"font-size: 1= 3pt;">National Military Audiology and Speech-Pathology Center (NMASC)</span>= </div><div><span style=3D"font-size: 13pt;">Walter Reed National Military Me= dical Center</span></div><div><div><span style=3D"font-size: 13pt;">Bethesda= , MD 20889</span></div><div><a href=3D"mailto:kenneth.w.grant.civ@xxxxxxxx">= kenneth.w.grant.civ@xxxxxxxx</a></div><div><a href=3D"mailto:ken.w.grant@xxxxxxxx= il.com">ken.w.grant@xxxxxxxx</a></div><div><span style=3D"font-size: 13pt;"= >Office: 301-319-7043</span></div><div><span style=3D"font-size: 13pt;= ">Cell: 301-919-2957</span></div><div><div><br><div><br><div><br></div= ></div></div></div></div></div><div><br>On Jun 9, 2018, at 12:48 AM, Matthew= Winn <<a href=3D"mailto:mwinn2@xxxxxxxx">mwinn2@xxxxxxxx</a>> wrote:<br><= br></div><blockquote type=3D"cite"><div><div dir=3D"ltr">The view that RRs w= ill stifle progress is both true and false. While the increased load of adva= nced registration and rigidity in methods would, as Les points out, become b= urdensome for most of our basic work, there is another side to this. This is= not a matter of morals (hiding a bad result, or fabricating a good result) o= r how to do our experiments. It=E2=80=99s a matter of the standards of *publ= ication*, which you will notice was the scope of Tim=E2=80=99s original call= to action. In general, we only ever read about experiments that came out we= ll (and not the ones that didn=E2=80=99t). If there is a solution to that pr= oblem, then we should consider it, or at least acknowledge that some solutio= n might be needed. This is partly the culture of scientific journals, and pa= rtly the culture of the institutions that employ us. There's no need to ques= tion anybody's integrity in order to appreciate some benefit of RRs. <br><br= >Think for a moment about the amount of wasted hours spent by investigators w= ho repeat the failed methods of their peers and predecessors, only because t= he outcomes of failed experiments were never published. Or those of us who c= ling to theories based on initial publications of work that later fails repl= ication, but where those failed replications never get published. THIS stifl= es progress as well. If results were to be reported whether or not they come= out as planned, we=E2=80=99d have a much more complete picture of the evide= nce for and against the ideas. Julia's story also resonates with me; we've a= ll reviewed papers where we've thought "if only the authors had sought input= before running this labor-intensive study, the data would be so much more v= aluable."<br><br>The arguments against RRs in this thread appear in my mind t= o be arguments against *compulsory* RRs for *all* papers in *all* journals, w= hich takes the discussion off course. I have not heard such radical calls. I= f you don=E2=80=99t want to do a RR, then don=E2=80=99t do it. But perhaps w= e can appreciate the goals of RR and see how those goals might be realized w= ith practices that suit our own fields of work.<br><br>Matt <br><br><div cla= ss=3D"gmail_extra"><div><div class=3D"gmail_signature" data-smartmail=3D"gma= il_signature"><div dir=3D"ltr"><div><div dir=3D"ltr"><br></div><div dir=3D"l= tr">--------------------------------------------------------------</div><div= dir=3D"ltr">Matthew Winn, Au.D., Ph.D.<br>Assistant Professor<br>Dept. of S= peech & Hearing Sciences<br>University of Washington</div></div></div></= div></div> <br></div></div> </div></blockquote></div></body></html>= --Apple-Mail-CF5FE5CC-B3CE-4090-8E61-27EBEB996A52--