Subject: Re: [AUDITORY] Registered reports From: "Schoof, Tim" <t.schoof@xxxxxxxx> Date: Fri, 8 Jun 2018 07:41:01 +0000 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>--_004_DB6PR0101MB2501F0BBA5C2473D868A53ABD87B0DB6PR0101MB2501_ Content-Type: multipart/alternative; boundary="_000_DB6PR0101MB2501F0BBA5C2473D868A53ABD87B0DB6PR0101MB2501_" --_000_DB6PR0101MB2501F0BBA5C2473D868A53ABD87B0DB6PR0101MB2501_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Dear List, It has been really great seeing such a lively discussion on the topic! While it is clear that not everyone sees the benefit of registered reports = - and they're probably not perfect, but hopefully they're a step in the rig= ht direction - I also know that a lot of people do value them and would be = interested in submitting their work as a registered report. I think it woul= d be good to at least have the option. My impression from psychology journa= ls that already offer registered reports is that they are not compulsory. Just over 30 people have contacted me these past few days to say they would= like to sign the letter<https://osf.io/3wct2/wiki/Journal%20Requests/>. I = plan to send this out early next week. If anyone still wants to add their n= ame to the list, just let me know. As a result of this discussion on the auditory list, I know that a few edit= ors / journals have started thinking and talking about the possibility of o= ffering registered reports. I'd be more than happy to keep the list updated= on any decisions that are made by the various journals in the coming month= s. Best, Tim ________________________________ From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxx>= on behalf of Les Bernstein <lbernstein@xxxxxxxx> Sent: 07 June 2018 08:40 To: AUDITORY@xxxxxxxx Subject: {SPAM?} Re: Registered reports Peter-- I think you have benefited the discussion by focusing on the "confirmatory"= vs. "exploratory" CONTINUUM. Thank you! I agree with these two (among ot= hers) of your statements: 1) I think that the debate about registered reports deteriorates too easily= into an all-or-nothing argument. 2) What we need is honesty about where our research falls on this continuum= . Despite your recognition of a continuum, your reply references a dichotomou= s view, given your reference to Tukey. Consider this definition of "confir= matory research" from http://www.butlerscientifics.com/single-post/2014/10/= 08/Exploratory-vs-Confirmatory-Research: [http://static.wixstatic.com/media/625cd8_6c7038b4085c4433623af11549ec6aad.= png]<http://www.butlerscientifics.com/single-post/2014/10/08/Exploratory-vs= -Confirmatory-Research> AutoDiscovery Automatic Intelligent Exploratory Data ...<http://www.butlers= cientifics.com/single-post/2014/10/08/Exploratory-vs-Confirmatory-Research> www.butlerscientifics.com AutoDiscovery is an automatic intelligent exploratory data analysis softwar= e that unveils complex relationships in the data files of your experiments. Confirmatory research (a.k.a. hypothesis testing<https://explorable.com/sta= tistical-hypothesis-testing>) is where researchers have a pretty good idea = of what's going on. That is, researcher has a theory (or several theories),= and the objective is to find out if the theory is supported by the facts. While the notion of a continuum is helpful, I would argue that only a very = narrow set of studies are ever merely "confirmatory." The problem lies wit= h the identification of "hypothesis testing" with "confirmatory." As I see= it, it is a false equivalence and is naive. In many cases a study that se= eks to test a hypothesis or hypotheses falls on your proposed continuum suc= h that a registered reports (RR) would be undesirable for many of the reaso= ns I identified in previous responses. As I see it, the set of studies tha= t might be appropriate for RRs are those that offer a virtually unequivocal= , binary set of potential outcomes. Those, in my view, are few and far bet= ween and are, more often than not, relatively uninteresting. So, if people= wish to use RRs for such studies, then fine. For the remainder-- most of = scientific output-- RRs hold little value and could, in my view, serve to s= tifle progress. Les On 6/6/2018 8:04 AM, Peter Harrison wrote: Dear list, I=92ve found this debate very interesting, thank you. Here are some thought= s of my own: I think that the debate about registered reports deteriorates too easily in= to an all-or-nothing argument. Registered reports are ideal for confirmator= y research, where it is realistic to specify the analysis in advance, where= the hypothetico-deductive method makes sense, and where the researcher has= a good plan of what the final paper should look like before they conduct t= he study. However, they are often not well-suited to exploratory research w= here the goal is simply to find out more about a given phenomenon. As noted= already on this thread, in such cases the branching factor of potential an= alyses may be simply too high to be worth specifying in advance. This is pa= rticularly true when the researcher wishes to conduct follow-up experiments= based on the results of previous experiments. Confirmatory and exploratory research are both vital - neither one is suffi= cient by itself (see e.g. Turkey 1980 - http://www.jstor.org/stable/2682991= <https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A__www.jstor.org_stable= _2682991&d=3DDwMGaQ&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQZ_7Fw9yfotck&r=3D2Pw= 2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DQgj_3ZpUHv4jHVkW8H2ur5onMjbdO0= aRXlJg2eqSzb4&s=3DtIRQcUn_AXQVySU6FSm3c0ihAcsI-voUUoP9ji6-mWE&e=3D>). What = we need is honesty about where our research falls on this continuum. Unfort= unately the emphasis on the hypothetico-deductive method and null hypothesi= s significance testing in psychological teaching provides many with the imp= ression that confirmatory studies with error control are the only =91scient= ific=92 way to generate knowledge. Perhaps if Bayesian techniques were more= commonly taught then we=92d be more likely to see the gradual accumulation= of positive evidence as a valid alternative. On registered reports, then, I believe the following: - they should be an good tool for preventing exploratory research from bein= g published as confirmatory research -> we should encourage journals to offer registered reports as an option -> we should support Tim Schoof in the initiative to write to hearing journ= als -> if you conduct a confirmatory study, then submitting it as a registered = report should be a good way to enhance the credibility of your findings - registered reports are often not suitable for exploratory research -> we shouldn=92t let the impression persist that registered reports should= be compulsory for all research. Best wishes, Peter On 6 Jun 2018, at 09:57, Nilesh Madhu <000000405df1884c-dmarc-request@xxxxxxxx= .MCGILL.CA<mailto:request@xxxxxxxx>> wrote: Good morning Bas, I see your point and I do agree that, in cases such as those you mention, p= re-defining and getting feedback on the trial procedure you wish to conduct= makes sense. However, for such scenarios, shouldn't there be industry-wide= standards on testing/trials and reporting in place already? If someone intentionally conducts a poorly conceived trial, the paper would= /should be rejected in the peer-review anyway ("reject/do not encourage res= ubmission"). As I see it, registered reports can be helpful if researchers = are not sure what trials they should conduct and want early feedback. Even = here, perhaps, a simpler solution would be to make guidelines available, in= stead of enforcing an 'administrative' layer. Of course, all of this is assuming that the intent is to make registered re= ports compulsory for any/every article. If this is not the case, the point = is moot :) Greetings from lovely, sunny Belgium (yes, we do have such days!) Nilesh PS: Massimo, I like your point about the carpenter being adaptive on the jo= b. Previously I attributed this necessity to adapt to my poor skills ;) On 2018-06-06 09:35, Bas Van Dijk wrote: Hi Nilesh, I agree to certain extend but I do feel that registered reporting makes sense for 'close to product' trials and trials that lead to treatments (for example evaluation of a fitting algorithm). In fact, it should not really be ' double work' as you fear because if you execute a poor trial and then try to get it published (believe me.. it happens :-) )but it gets rejected and you basically have no option but to redo (part of) the work. (and -re-writing the text to get a poor trial accepted for publication is of course exactly what you don't want...). That's more double work that writing up a good trial proposal, have it reviewed and then know that if you execute according to plan it's likely to get published even if the results are negative or non-conclusive, that could be a pre as well. Best wishes, Bas Bas Van Dijk Program Manager, A&A - Clinician and Research Tools Cochlear Technology Centre Belgium Schali=EBnhoevedreef 20 I 2800 Mechelen BELGIUM Phone: +3215795528 Mobile: +32473976270 Email: BVanDijk@xxxxxxxx<mailto:BVanDijk@xxxxxxxx> www.cochlear.com<http://www.cochlear.com> -----Original Message----- From: AUDITORY - Research in Auditory Perception [mailto:AUDITORY@xxxxxxxx On Behalf Of Nilesh Madhu Sent: dinsdag 5 juni 2018 13:16 To: AUDITORY@xxxxxxxx<mailto:AUDITORY@xxxxxxxx> Subject: [AUDITORY] Registered reports Dear Tim, I appreciate your initiative towards reproducible research. However I fear that registered reports would just add another layer of overhead to academics and students already under the pressure to publish. If I understand correctly, this involves two rounds of review: a first review based on the methodology and evaluation and a second based on the results of the research. For each stage, probably at least two review rounds would be needed (going by the current publishing cycle). I fear, as Gaston does, this might stifle creativity and lead to overwork also for reviewers and editors. Of course, this is assuming you want to make registered reports compulsory... Furthermore, such an approach may not be equally applicable to all research. For research into algorithms, for example, the value of the research lies, usually, in the core idea. There are myriad accepted forms of evaluation and to force a strict evaluation pattern/methodology would be counterproductive. Reproducible research in this case is targeted by encouraging authors to make their code and test data public. What I would support are (voluntary) guidelines on reporting results of experiments. This is often to be found in in the engineering field, when one participates in an open challenge. Lastly, the main reason for this initiative is to avoid 'mis-reporting' the results in favour of a hypothesis. Surely, this calls for self policing? Aren't we, as researchers, possessed of sufficient integrity and ethics to present our research in the correct light? If this core value is missing, I fear no external policing is going to help. Best regards Nilesh Madhu =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D "The information contained in this e-mail message may be confidential information, and may also be privileged. If you are not the intended recipient, any use, interference with, disclosure or copying of this material is unauthorised and prohibited. If you have received this message in error, please notify us by return email and delete the original message." -- Leslie R. Bernstein, Ph.D. | Professor Depts. of Neuroscience and Surgery (Otolaryngology)| UConn School of Medici= ne 263 Farmington Avenue, Farmington, CT 06030-3401 Office: 860.679.4622 | Fax: 860.679.2495 [cid:part5.17ABD5DC.AA243958@xxxxxxxx --_000_DB6PR0101MB2501F0BBA5C2473D868A53ABD87B0DB6PR0101MB2501_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1= 252"> <style type=3D"text/css" style=3D"display:none;"><!-- P {margin-top:0;margi= n-bottom:0;} --></style> </head> <body dir=3D"ltr"> <div id=3D"divtagdefaultwrapper" style=3D"font-size:12pt;color:#000000;font= -family:Calibri,Helvetica,sans-serif;" dir=3D"ltr"> <p style=3D"margin-top:0;margin-bottom:0"></p> <div> <p style=3D"margin-top:0; margin-bottom:0">Dear List,</p> <p style=3D"margin-top:0; margin-bottom:0"><br> </p> <p style=3D"margin-top:0; margin-bottom:0">It has been really great seeing = such a lively discussion on the topic!</p> <p style=3D"margin-top:0; margin-bottom:0"><br> </p> <p style=3D"margin-top:0; margin-bottom:0">While it is clear that not every= one sees the benefit of registered reports - and they're probably not perfe= ct, but hopefully they're a step in the right direction - I also know that = a lot of people do value them and would be interested in submitting their work as a registered report. I thi= nk it would be good to at least have the option. My impression from psychol= ogy journals that already offer registered reports is that they are not com= pulsory.<br> </p> <p style=3D"margin-top:0; margin-bottom:0"><br> </p> <div>Just over 30 people have contacted me these past few days to say they = would like to sign the <a href=3D"https://osf.io/3wct2/wiki/Journal%20Requests/" target=3D"_blank"= rel=3D"noopener noreferrer" class=3D"x_OWAAutoLink" id=3D"LPlnk901916" pre= viewremoved=3D"true"> letter</a>. I plan to send this out early next week. If anyone still wants = to add their name to the list, just let me know.</div> <div><br> </div> <div>As a result of this discussion on the auditory list, I know that a few= editors / journals have started thinking and talking about the possibility= of offering registered reports. I'd be more than happy to keep the list up= dated on any decisions that are made by the various journals in the coming months.<br> </div> <p style=3D"margin-top:0; margin-bottom:0"><br> </p> <p style=3D"margin-top:0; margin-bottom:0">Best,</p> Tim</div> <br> <p></p> <br> <br> <div style=3D"color: rgb(0, 0, 0);"> <hr style=3D"display:inline-block;width:98%" tabindex=3D"-1"> <div id=3D"divRplyFwdMsg" dir=3D"ltr"><font style=3D"font-size:11pt" face= =3D"Calibri, sans-serif" color=3D"#000000"><b>From:</b> AUDITORY - Research= in Auditory Perception <AUDITORY@xxxxxxxx> on behalf of Les B= ernstein <lbernstein@xxxxxxxx><br> <b>Sent:</b> 07 June 2018 08:40<br> <b>To:</b> AUDITORY@xxxxxxxx<br> <b>Subject:</b> {SPAM?} Re: Registered reports</font> <div> </div> </div> <meta content=3D"text/html; charset=3Dutf-8"> <div style=3D"background-color:#FFFFFF"><font size=3D"-1"><font face=3D"Ver= dana">Peter--<br> <br> I think you have benefited the discussion by focusing on the "confirma= tory" vs. "exploratory" CONTINUUM. Thank you! I = agree with these two (among others) of your statements:<br> </font></font><br> <font size=3D"-1"><font face=3D"Verdana">1) I think that the debate about r= egistered reports deteriorates too easily into an all-or-nothing argument.<= br> </font></font><font size=3D"-1"><font face=3D"Verdana">2) What we need is h= onesty about where our research falls on this continuum.<br> <br> Despite your recognition of a continuum, your reply references a dichotomou= s view, given your reference to Tukey. Consider this definition of &q= uot;confirmatory research" from <a class=3D"x_moz-txt-link-freetext OWAAutoLink" href=3D"http://www.butlers= cientifics.com/single-post/2014/10/08/Exploratory-vs-Confirmatory-Research"= id=3D"LPlnk601084" previewremoved=3D"true"> http://www.butlerscientifics.com/single-post/2014/10/08/Exploratory-vs-Conf= irmatory-Research</a>: <div id=3D"LPBorder_GT_15284434574320.6518915388476153" style=3D"margin-bot= tom: 20px; overflow: auto; width: 100%; text-indent: 0px;"> <table id=3D"LPContainer_15284434574160.4089859836121047" style=3D"width: 9= 0%; background-color: rgb(255, 255, 255); position: relative; overflow: aut= o; padding-top: 20px; padding-bottom: 20px; margin-top: 20px; border-top: 1= px dotted rgb(200, 200, 200); border-bottom: 1px dotted rgb(200, 200, 200);= " role=3D"presentation" cellspacing=3D"0"> <tbody> <tr style=3D"border-spacing: 0px;" valign=3D"top"> <td id=3D"ImageCell_15284434574210.18652971682735642" style=3D"width: 250px= ; position: relative; display: table-cell; padding-right: 20px;" colspan=3D= "1"> <div id=3D"LPImageContainer_15284434574210.9425310098884251" style=3D"backg= round-color: rgb(255, 255, 255); height: 192px; position: relative; margin:= auto; display: table; width: 192px;"> <a id=3D"LPImageAnchor_15284434574240.46424718325185654" style=3D"display: = table-cell; text-align: center;" href=3D"http://www.butlerscientifics.com/s= ingle-post/2014/10/08/Exploratory-vs-Confirmatory-Research" target=3D"_blan= k"><img style=3D"display: inline-block; max-width: 250px; max-height: 250px= ; height: 192px; width: 192px; border-width: 0px; vertical-align: bottom;" = id=3D"LPThumbnailImageID_15284434574240.6935587594381046" width=3D"192" hei= ght=3D"192" src=3D"http://static.wixstatic.com/media/625cd8_6c7038b4085c443= 3623af11549ec6aad.png"></a></div> </td> <td id=3D"TextCell_15284434574280.8516227008042454" style=3D"vertical-align= : top; position: relative; padding: 0px; display: table-cell;" colspan=3D"2= "> <div id=3D"LPRemovePreviewContainer_15284434574280.8456838948382911"></div> <div id=3D"LPTitle_15284434574280.9960968851374994" style=3D"top: 0px; colo= r: rgb(0, 120, 215); font-weight: 400; font-size: 21px; font-family: "= wf_segoe-ui_light", "Segoe UI Light", "Segoe WP Light&q= uot;, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif= ; line-height: 21px;"> <a id=3D"LPUrlAnchor_15284434574290.4638314355321973" style=3D"text-decorat= ion: none;" href=3D"http://www.butlerscientifics.com/single-post/2014/10/08= /Exploratory-vs-Confirmatory-Research" target=3D"_blank">AutoDiscovery Auto= matic Intelligent Exploratory Data ...</a></div> <div id=3D"LPMetadata_15284434574300.2898083831144924" style=3D"margin: 10p= x 0px 16px; color: rgb(102, 102, 102); font-weight: 400; font-family: "= ;wf_segoe-ui_normal", "Segoe UI", "Segoe WP", Taho= ma, Arial, sans-serif; font-size: 14px; line-height: 14px;"> www.butlerscientifics.com</div> <div id=3D"LPDescription_15284434574310.7831664953430287" style=3D"display:= block; color: rgb(102, 102, 102); font-weight: 400; font-family: "wf_= segoe-ui_normal", "Segoe UI", "Segoe WP", Tahoma, = Arial, sans-serif; font-size: 14px; line-height: 20px; max-height: 100px; o= verflow: hidden;"> AutoDiscovery is an automatic intelligent exploratory data analysis softwar= e that unveils complex relationships in the data files of your experiments.= </div> </td> </tr> </tbody> </table> </div> <br> <br> </font></font><font size=3D"-1"><font face=3D"Verdana"><span style=3D"margi= n:0px; padding:0px; border:0px; outline:0px; vertical-align:baseline; backg= round:0px 0px; color:rgb(89,89,89); font-family:Basic,sans-serif; font-size= :18px; font-style:normal; letter-spacing:normal; orphans:2; text-align:left= ; text-indent:0px; text-transform:none; white-space:normal; widows:2; word-= spacing:0px; font-weight:bold">Confirmatory research</span><span style=3D"color:rgb(89,89,89); font-family:Basic,sans-= serif; font-size:18px; font-style:normal; font-weight:400; letter-spacing:n= ormal; orphans:2; text-align:left; text-indent:0px; text-transform:none; wh= ite-space:normal; widows:2; word-spacing:0px; display:inline!important; flo= at:none"><span> </span>(a.k.a.<span> </span></span><span style=3D= "margin:0px; padding:0px; border:0px; outline:0px; vertical-align:baseline;= background:0px 0px; color:rgb(89,89,89); font-family:Basic,sans-serif; fon= t-size:18px; font-style:normal; font-weight:400; letter-spacing:normal; orp= hans:2; text-align:left; text-indent:0px; text-transform:none; white-space:= normal; widows:2; word-spacing:0px; text-decoration:underline"><a href=3D"h= ttps://explorable.com/statistical-hypothesis-testing" target=3D"_blank" rel= =3D"undefined" style=3D"margin:0px; padding:0px; border:0px; outline:0px; v= ertical-align:baseline; background:0px 0px; text-decoration:none; color:inh= erit" id=3D"LPlnk492768" class=3D"OWAAutoLink" previewremoved=3D"true">hypo= thesis testing</a></span><span style=3D"color:rgb(89,89,89); font-family:Basic,sa= ns-serif; font-size:18px; font-style:normal; font-weight:400; letter-spacin= g:normal; orphans:2; text-align:left; text-indent:0px; text-transform:none;= white-space:normal; widows:2; word-spacing:0px; display:inline!important; = float:none">) is where researchers have a pretty good idea of what's going on. That is, = researcher has a theory (or several theories), and the objective is to find= out if the theory is supported by the facts.<br> <br> </span></font></font><font size=3D"-1"><font face=3D"Verdana"><span style= =3D"font-family:Basic,sans-serif; font-size:18px; font-style:normal; font-w= eight:400; letter-spacing:normal; text-align:left; text-indent:0px; text-tr= ansform:none; white-space:normal; word-spacing:0px; display:inline!importan= t; float:none"><font size=3D"-1"><font face=3D"Verdana">While the notion of a continuum is helpful, I would argue that only a very narro= w set of studies are ever merely "confirmatory." The proble= m lies with the identification of "hypothesis testing" with "= ;confirmatory." As I see it, it is a false equivalence and is naive. In many cases a study that seeks to test a hypothesis or hypo= theses falls on your proposed continuum such that a registered reports (RR)= would be undesirable for many of the reasons I identified in previous resp= onses. As I see it, the set of studies that might be appropriate for RRs are those that offer a virtually unequiv= ocal, binary set of potential outcomes. Those, in my view, are few an= d far between and are, more often than not, relatively uninteresting. = So, if people wish to use RRs for such studies, then fine. For the remainder-- most of scientific output-- RRs hold = little value and could, in my view, serve to stifle progress.<br> <br> Les<br> </font></font></span></font></font> <div class=3D"x_moz-cite-prefix"><br> <br> On 6/6/2018 8:04 AM, Peter Harrison wrote:<br> </div> <blockquote type=3D"cite"> <div class=3D"">Dear list,</div> <div class=3D""><br class=3D""> </div> <div class=3D"">I=92ve found this debate very interesting, thank you. Here = are some thoughts of my own:</div> <div class=3D""><br class=3D""> </div> <div class=3D"">I think that the debate about registered reports deteriorat= es too easily into an all-or-nothing argument. Registered reports are ideal= for confirmatory research, where it is realistic to specify the analysis i= n advance, where the hypothetico-deductive method makes sense, and where the researcher has a good plan of what the f= inal paper should look like before they conduct the study. However, they ar= e often not well-suited to exploratory research where the goal is simply to= find out more about a given phenomenon. As noted already on this thread, in such cases the branching factor of pot= ential analyses may be simply too high to be worth specifying in advance. T= his is particularly true when the researcher wishes to conduct follow-up ex= periments based on the results of previous experiments.</div> <div class=3D""><br class=3D""> </div> Confirmatory and exploratory research are both vital - neither one is suffi= cient by itself (see e.g. Turkey 1980 - <a href=3D"https://u= rldefense.proofpoint.com/v2/url?u=3Dhttp-3A__www.jstor.org_stable_2682991&a= mp;d=3DDwMGaQ&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQZ_7Fw9yfotck&r=3D2= Pw2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DQgj_3ZpUHv4jHVkW8H2ur5on= MjbdO0aRXlJg2eqSzb4&s=3DtIRQcUn_AXQVySU6FSm3c0ihAcsI-voUUoP9ji6-mWE&= ;e=3D" class=3D"OWAAutoLink" id=3D"LPlnk816638" previewremoved=3D"true">htt= p://www.jstor.org/stable/2682991</a>). What we need is honesty about where our research falls on this continuum. = Unfortunately the emphasis on the hypothetico-deductive method and null hyp= othesis significance testing in psychological teaching provides many with t= he impression that confirmatory studies with error control are the only =91scientific=92 way to generate k= nowledge. Perhaps if Bayesian techniques were more commonly taught then we= =92d be more likely to see the gradual accumulation of positive evidence as= a valid alternative. <div class=3D""><br class=3D""> </div> <div class=3D"">On registered reports, then, I believe the following:</div> <div class=3D""><br class=3D""> </div> <div class=3D"">- they should be an good tool for preventing exploratory re= search from being published as confirmatory research </div> <div class=3D"">-> we should encourage journals to offer registered repo= rts as an option</div> <div class=3D"">-> we should support Tim Schoof in the initiative to wri= te to hearing journals</div> <div class=3D"">-> if you conduct a confirmatory study, then submitting = it as a registered report should be a good way to enhance the credibility o= f your findings</div> <div class=3D""><br class=3D""> </div> <div class=3D"">- registered reports are often not suitable for exploratory= research </div> <div class=3D"">-> we shouldn=92t let the impression persist that regist= ered reports should be compulsory for all research.</div> <div class=3D""><br class=3D""> </div> <div class=3D"">Best wishes,</div> <div class=3D"">Peter<br class=3D""> <br class=3D""> <blockquote type=3D"cite" class=3D"">On 6 Jun 2018, at 09:57, Nilesh Madhu = <000000405df1884c-dmarc-<a href=3D"mailto:request@xxxxxxxx" class= =3D"OWAAutoLink" id=3D"LPlnk332991" previewremoved=3D"true">request@xxxxxxxx= CGILL.CA</a>> wrote:<br class=3D""> <br class=3D""> Good morning Bas,<br class=3D""> <br class=3D""> I see your point and I do agree that, in cases such as those you mention, p= re-defining and getting feedback on the trial procedure you wish to conduct= makes sense. However, for such scenarios, shouldn't there be industry= -wide standards on testing/trials and reporting in place already?<br class=3D""> <br class=3D""> If someone intentionally conducts a poorly conceived trial, the paper would= /should be rejected in the peer-review anyway ("reject/do not enc= ourage resubmission"). As I see it, registered reports can be hel= pful if researchers are not sure what trials they should conduct and want early feedback. Even here, perhaps, a simpler soluti= on would be to make guidelines available, instead of enforcing an 'adm= inistrative' layer.<br class=3D""> <br class=3D""> Of course, all of this is assuming that the intent is to make registered re= ports compulsory for any/every article. If this is not the case, the p= oint is moot :)<br class=3D""> <br class=3D""> Greetings from lovely, sunny Belgium (yes, we do have such days!)<br class= =3D""> <br class=3D""> Nilesh<br class=3D""> <br class=3D""> PS: Massimo, I like your point about the carpenter being adaptive on the jo= b. Previously I attributed this necessity to adapt to my poor skills ;= )<br class=3D""> <br class=3D""> <br class=3D""> <br class=3D""> On 2018-06-06 09:35, Bas Van Dijk wrote:<br class=3D""> <blockquote type=3D"cite" class=3D"">Hi Nilesh,<br class=3D""> I agree to certain extend but I do feel that registered reporting<br class= =3D""> makes sense for 'close to product' trials and trials that lead to<br class= =3D""> treatments (for example evaluation of a fitting algorithm). In fact,<br cla= ss=3D""> it should not really be ' double work' as you fear because if you<br = class=3D""> execute a poor trial and then try to get it published (believe me.. it<br c= lass=3D""> happens :-) )but it gets rejected and you basically have no option but<br c= lass=3D""> to redo (part of) the work. (and -re-writing the text to get a poor<br clas= s=3D""> trial accepted for publication is of course exactly what you don't<br class= =3D""> want...). That's more double work that writing up a good trial<br cla= ss=3D""> proposal, have it reviewed and then know that if you execute according<br c= lass=3D""> to plan it's likely to get published even if the results are negative<br cl= ass=3D""> or non-conclusive, that could be a pre as well.<br class=3D""> Best wishes,<br class=3D""> Bas<br class=3D""> Bas Van Dijk<br class=3D""> Program Manager, A&A - Clinician and Research Tools<br class=3D""> Cochlear Technology Centre Belgium<br class=3D""> Schali=EBnhoevedreef 20 I<br class=3D""> 2800 Mechelen<br class=3D""> BELGIUM<br class=3D""> Phone: +3215795528<br class=3D""> Mobile: +32473976270<br class=3D""> <a href=3D"mailto:BVanDijk@xxxxxxxx" class=3D"OWAAutoLink" id=3D"LPlnk6= 82197" previewremoved=3D"true">Email: BVanDijk@xxxxxxxx</a><br class=3D= ""> <a class=3D"x_moz-txt-link-abbreviated OWAAutoLink" href=3D"http://www.coch= lear.com" id=3D"LPlnk67861" previewremoved=3D"true">www.cochlear.com</a><br= class=3D""> -----Original Message-----<br class=3D""> From: AUDITORY - Research in Auditory Perception<br class=3D""> [<a class=3D"x_moz-txt-link-freetext OWAAutoLink" href=3D"mailto:AUDITORY@xxxxxxxx= ISTS.MCGILL.CA" id=3D"LPlnk925949" previewremoved=3D"true">mailto:AUDITORY@xxxxxxxx= LISTS.MCGILL.CA</a>] On Behalf Of Nilesh Madhu<br class=3D""> Sent: dinsdag 5 juni 2018 13:16<br class=3D""> To: <a class=3D"x_moz-txt-link-abbreviated OWAAutoLink" href=3D"mailto:AUDI= TORY@xxxxxxxx" id=3D"LPlnk710703" previewremoved=3D"true"> AUDITORY@xxxxxxxx</a><br class=3D""> Subject: [AUDITORY] Registered reports<br class=3D""> Dear Tim,<br class=3D""> I appreciate your initiative towards reproducible research. However I<br cl= ass=3D""> fear that registered reports would just add another layer of overhead<br cl= ass=3D""> to academics and students already under the pressure to publish. If I<br cl= ass=3D""> understand correctly, this involves two rounds of review: a first<br class= =3D""> review based on the methodology and evaluation and a second based on<br cla= ss=3D""> the results of the research. For each stage, probably at least two<br class= =3D""> review rounds would be needed (going by the current publishing cycle).<br c= lass=3D""> I fear, as Gaston does, this might stifle creativity and lead to<br class= =3D""> overwork also for reviewers and editors. Of course, this is assuming<br cla= ss=3D""> you want to make registered reports compulsory...<br class=3D""> Furthermore, such an approach may not be equally applicable to all<br class= =3D""> research. For research into algorithms, for example, the value of the<br cl= ass=3D""> research lies, usually, in the core idea. There are myriad accepted<br clas= s=3D""> forms of evaluation and to force a strict evaluation<br class=3D""> pattern/methodology would be counterproductive. Reproducible research<br cl= ass=3D""> in this case is targeted by encouraging authors to make their code and<br c= lass=3D""> test data public.<br class=3D""> What I would support are (voluntary) guidelines on reporting results<br cla= ss=3D""> of experiments. This is often to be found in in the engineering field,<br c= lass=3D""> when one participates in an open challenge.<br class=3D""> Lastly, the main reason for this initiative is to avoid 'mis-reporting'<br = class=3D""> the results in favour of a hypothesis. Surely, this calls for self<br class= =3D""> policing? Aren't we, as researchers, possessed of sufficient integrity<br c= lass=3D""> and ethics to present our research in the correct light? If this core<br cl= ass=3D""> value is missing, I fear no external policing is going to help.<br class=3D= ""> Best regards<br class=3D""> Nilesh Madhu<br class=3D""> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br class=3D= ""> "The information contained in this e-mail message may be confidential<= br class=3D""> information, and may also be privileged. If you are not the intended<br cla= ss=3D""> recipient, any use, interference with, disclosure or copying of this<br cla= ss=3D""> material is unauthorised and prohibited. If you have received this<br class= =3D""> message in error, please notify us by return email and delete the<br class= =3D""> original message."<br class=3D""> </blockquote> </blockquote> <br class=3D""> </div> </blockquote> <br> <br> <div class=3D"x_moz-signature">-- <br> <b><span style=3D"">Leslie R. Bernstein, Ph.D. </span></b><b><span style=3D= "">| </span> </b><span style=3D"">Professor</span><span style=3D""></span><span style=3D= ""><br> Depts. of Neuroscience and Surgery (Otolaryngology)| UConn School of Medici= ne </span> <br> <span style=3D""></span><span style=3D"">263 Farmington Avenue, Farmington,= CT 06030-3401</span><br> <span style=3D""></span><span style=3D"">Office: 860.679.4622 | Fax: 860.67= 9.2495<br> <br> <img alt=3D"" tabindex=3D"0" width=3D"125" height=3D"48" src=3D"cid:part5.1= 7ABD5DC.AA243958@xxxxxxxx"><br> </span></div> </div> </div> </div> </body> </html> --_000_DB6PR0101MB2501F0BBA5C2473D868A53ABD87B0DB6PR0101MB2501_-- --_004_DB6PR0101MB2501F0BBA5C2473D868A53ABD87B0DB6PR0101MB2501_ Content-Type: image/png; name="uconnhealth_stacked_blue_email.png" Content-Description: uconnhealth_stacked_blue_email.png Content-Disposition: inline; filename="uconnhealth_stacked_blue_email.png"; size=796; creation-date="Fri, 08 Jun 2018 04:11:13 GMT"; modification-date="Fri, 08 Jun 2018 04:11:13 GMT" Content-ID: <part5.17ABD5DC.AA243958@xxxxxxxx> Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAH0AAAAwCAMAAAALmIWlAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJ bWFnZVJlYWR5ccllPAAAADNQTFRFKzVYHCZM4eLn8PHzpKi30dTbaG+IlZqrs7fDWWB8O0Nkd32U wsXPSlJwhoufDRhA////A68jmAAAABF0Uk5T/////////////////////wAlrZliAAACYklEQVR4 2uzY22KDIAwAUC7e6gjw/187gUC4OetaupflaTXKWSuBKNMuJptC+QMCP20auPHBQCubBWU46K24 uB7NfZ58ZqaMP8D8AEBHtT/goXnB8WPwBf8r1WYUXWxUOZobHcJXIMcfONclmE4s0lrRzYBIOjvT jX5WV6YbP2ZQJ6TWjfiAbuYzHW7p+xpuolQrP83sjQ5nupnu6LqaJ92MbnSznulcfkBHpNXNY6y+ c0JKnWX1eFeflQt5qcNESKnrQMrf6LSAXeiWkEoXdOU4nZBKxzPnoTohtW5ZZyK8W09Io6tYjyP1 iDS6XbAeh815S0irS6zHUfXuB0Ok1S3W41AdEWh1/Nb7UD1bWytd0LY4ThdnerYbBJ1taT5XBmy4 J8mNnWagp5fbba6HeiTdneK6vNgoFj0EhyO4eSZT6AlpdNXoNzqYi94GbIU0eqhHp3d5drRf4heZ Uo9Iq8vYptup6VB37H6m/W6m0hFpdaxH3/yK9cHpVq5Zy59n2GMV/QxPGdV9NElPE9njyOozsfWW Kq6j7bLuw97KPBnM/mX86//6n+tH/yLKP7AUXcyd80Lx53WnspB47ty5qKND1gLockfIFzHR7srp Q7X0+6GgHPNFXedPyaN1CPFli71z+ZBeTxbcI2RXxwOQj/lW/Ui610Z6gA4a32WRrssXYbO76V/Z y6FLnZVj/qCnqGcdZE1DeFKcntWrMV/RZXCh7B/fpF/+8tQQx6J74y9/Oeuyrm75+Jyf8oKWH9CL 1SYNJtJ91HTOAD2fdjPN9UcsumwmDNYXqvPtOLa9pH8LMACnoV0siZAyOAAAAABJRU5ErkJggg== --_004_DB6PR0101MB2501F0BBA5C2473D868A53ABD87B0DB6PR0101MB2501_--