Subject: Re: [AUDITORY] Registered reports From: julienbesle <julienbesle@xxxxxxxx> Date: Wed, 6 Jun 2018 19:46:25 +0300 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>This is a multi-part message in MIME format. --------------E3ADA6FD22290D9ACF6395AE Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by edgeum1.it.mcgill.ca id w56GkgBx014348 I concur with Massimo's remarks. I would like to add the following, in=20 response to some of the emails that have been sent earlier: - I share the concern that not all (good) science is=20 hypothetico-deductive. And certainly pre-registration should not be made=20 mandatory: purely exploratory studies should certainly be allowed,=20 encouraged and publishable. However, they should be published for what=20 they are: exploratory. A result from an exploratory study (or an=20 unexpected result in a pre-registered study) needs to be replicated,=20 even more so than one from a hypothetico-deductive study, so it should=20 carry less weight. Widespread use of pre-registration would help=20 distinguish what results are serendipitous, exploratory or predicted. I=20 really cannot see what's there not to like in this. Like everyone else=20 here I have published results that were unexpected or unpredicted, or I=20 have ran studies that were formulated as questions rather than=20 hypotheses, but I suspect that the way I presented them in the published=20 report sometimes could suggest otherwise, partly because I was=20 encouraged by my supervisors to write a "story", or a narrative around=20 the results. I think that pre-registration would be prevent this to a=20 large extent. - I can also see that pre-registration might not apply to all types of=20 science (see the computer science example from Nilesh). However, I=20 suspect that it could apply to a lot of experimental studies (except=20 purely exploratory ones). - I also understand the concern that pre-registration would increase=20 overhead. In my opinion, this is unlikely and I would think that on the=20 contrary, it would decrease that burden. That a study is pre-approved by=20 peers means that the introduction and part of the methods do not need to=20 be re-reviewed at the second review stage. Also, as reviewers, we would=20 not have to keep wondering whether what is presented as a predicted=20 result was indeed predicted or if history was re-written at the same=20 time as the manuscript (as I suspect is too often the case). - I think we shouldn't underestimate how much we can deceive ourselves=20 (even scientists can) and to what extent adequate incentives can change=20 behaviours. We're deceiving ourselves if we think that self-policy,=20 integrity and ethics can solve problems such as replicability, biased=20 reporting and p-hacking. - I am not sure I see what is wrong in accepting a manuscript=20 "provisionally", "in principle", or even in publication being=20 "guaranteed" once the pre-registration is accepted. If the study has=20 been deemed worthy by reviewers in the first stage, I cannot see why it=20 shouldn't be accepted in the second unless the agreed protocol has not=20 been followed or factual errors are made. Since there is a second round=20 of review, presumably, this means that reviewers can ask authors for=20 revisions and that errors can be corrected. Of course, if the agreed=20 experimental protocol has not been followed, the study should not be=20 published. I think what is meant by "provisionally" is that the=20 manuscript is pre-approved /whatever results come out of the experiment.=20 /This is meant to counter biased reviews from reviewers who might want=20 to reject the study because they disagree with the outcome/./ Provided=20 that=A0 statistical power has been properly evaluated in the first phase,= =20 there is nothing wrong with flooding the literature with null results=20 (it needs it). - If one has an a priori hypothesis, tests it and finds that the data do=20 not support it, they should be able to report both the (wrong) a priori=20 hypothesis and the results. This would better reflect the non-linearity=20 of scientific discovery and this is precisely what pre-registration=20 allows and what the current system discourages. This is not HARKing.=20 What constitutes HARKing is when one changes their hypothesis post-hoc=20 but writes the manuscript as if the new post-hoc hypothesis had been=20 their a priori hypothesis all along. - I agree that choosing a specific statistical test in advance would be=20 unnecessarily restrictive. Choosing a statistical test after data have=20 been measured or even looked at is not necessarily p-hacking. If this=20 was the case, then testing the normality of a distribution before=20 deciding whether to use a parametric or a non-parametric test in a small=20 sample (for instance) would be called p-hacking. It's not. In fact, it's=20 recommended. It is p-hacking only if both tests are run and one chooses=20 the best of both outcomes. Dr Julien (My students actually call me Dr Julien, believe it or not). On 06/06/2018 09:09, Massimo Grassi wrote: > First of all I would like to thank Tim for the initiative. > > A few replies and comments: > - registered reports have the results section divided in the parts: the > "planned analysis" (those you discussed with editor and reviewers) and > the "new exploratory analysis". Therefore, I do not see the problem > risen by Les. > > - in my opinion registered reports rise the standard level of current > science. Registered reports (like a preregistration but even better) > reveal how limited is our ability to predict. It is difficult to predic= t > how the data will look like, what data point will be an outlier, whethe= r > data should be analysed in this or that way. We teach to students that > the path of science is hypothetical deductive. In reality we move more > like a carpenter trying to adapt and adjust things in real time. > > - about the possible "uncontrolled dissemination of null results", I > think that (for science) the current uncontrolled dissemination of type > I errors is worse. > > A nice day to everybody from a summer-sunny Italy, > > m > >> Dear List, >> >> For this topic, I'll violate my rule of not posting replies here.=A0 I >> agree with Ms. Rankovic.=A0 I sure did not miss the substance and deta= il >> of Mr. Schoof's email.=A0 I also read over the information in the link= s. >> Indeed, the proposed plan provides for a second review.=A0 It seems to= me, >> however, that the provisional acceptance is a key aspect of the >> process.=A0 If it were the case that manuscripts were rejected upon se= cond >> review with substantial frequency, then the philosophy of the register= ed >> report would be violated and the system would collapse.=A0 So, unless >> there are egregious errors or flaws in the full manuscript, it seems >> that it would be published.=A0 Note that, in this linked reference >> <https://orca.cf.ac.uk/59475/1/AN2.pdf>, publication is assumed to be >> "guaranteed." >> >> In my opinion, the criticism found within the FAQ here >> <https://cos.io/rr/>, that "The Registered Reports model is based on a >> na=EFve conceptualisation of the scientific method." is well-founded! = The >> reply offered to counter that criticism is quite weak and unconvincing. >> I would replace "scientific method" in that criticism with "the way go= od >> science is done." >> >> Question 17 in Chambers et al. (2014-- linked above) provides an apt >> example.=A0 In the process of conducting complex experiments, it is ve= ry >> often the case that unexpected results lead to important follow-up or >> control experiments.=A0 Chambers et al. handle this issue by proposing >> that in Stage 1 of a registered report, contingencies be stated such >> that "If A is observed, then we will..."=A0 That, of course, assumes t= hat >> one knows the decision tree in advance!=A0 In my experience, science >> simply does not work that way. >> >> While I find the intent of registered reports to be laudable, in my >> opinion, it substitutes one potential set of problems with another bas= ed >> on a narrow view of how science proceeds.=A0 Indeed, one may have a >> hypothesis to be tested and gather a set of data to address it only to >> find that the results support a substantially altered view.=A0 Is that= , >> NECESSARILY, the dreaded "HARKing?"=A0 I think not.=A0 Scientific thou= ght >> and inquiry do not always proceed in a linear fashion.=A0 One cannot a= nd >> should not always know the precise questions or list of contingencies = a >> priori and be restricted to answering only those.=A0 Then there are >> experiments in which there are no specific hypotheses.=A0 They may be = of >> the form, "What is the effect of variable A on measurements of X?" >> Assuming the question is non-trivial, those are often the most reveali= ng >> experiments because any outcome is of interest.=A0 There is no "positi= ve" >> or "negative."=A0 Sure, one can cast such experiments in terms of >> hypotheses but doing so often involves a contrivance. >> >> Then there is the matter of "p-hacking" and what I would call >> "statistics shopping."=A0 Indeed, it is a problem.=A0 Unexpected outco= mes >> and patterns of data in a complex experiment often require one to choo= se >> the appropriate statistic after the fact. It is sometimes the correct >> thing to do!=A0 Whether it is proper can and should be judged by revie= wers >> with the requisite expertise.=A0 Good peer-review should distinguish >> between p-hacking and a rational choice that conveys information and >> "truth."=A0 The notion that one can and should use only the statistic >> decided upon in advance is unnecessary restrictive. >> >> Finally, there is the matter of archival value.=A0 According to Chambe= rs >> et al., "...if the rationale and methods are sound then the journal >> should agree to publish the final paper regardless of the specific >> outcome."=A0 It is often the case that rationale and methods are sound= but >> the data provide no substantial advance or archival value.=A0 I'm not = sure >> that "approving" a method and rationale and virtually guaranteeing >> publication will afford the same level of judgment in terms of archiva= l >> value that is afforded by the current system. >> >> Les Bernstein >> >> --=20 >> *Leslie R. Bernstein, Ph.D. **| *Professor >> Depts. of Neuroscience and Surgery (Otolaryngology)| UConn School of >> Medicine >> 263 Farmington Avenue, Farmington, CT 06030-3401 >> Office: 860.679.4622 | Fax: 860.679.2495 >> >> >> >> >> >> >> >> <https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__cos.io_rr_&d=3D= DwMFAg&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQZ_7Fw9yfotck&r=3D2Pw2GwelGcMR49= 53G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DSr0Ep-Gx1c9KJlrgGBL4rmcUvd9qeDUDnFKUymD= oKpI&s=3DvXqZBKaP1dUovPzwBwC5DalLCB6UxwKuM9x_SQCbw5I&e=3D>=20 >> >> >> On 6/4/2018 7:51 AM, Christine Rankovic wrote: >>> >>> Mr. Schoof: >>> >>> It is beyond ridiculous to accept partial manuscripts for publication. >>> >>> Christine Rankovic, PhD >>> >>> Scientist, Speech and Hearing >>> >>> Newton, MA=A0 USA >>> >>> rankovic@xxxxxxxx >>> >>> *From:*AUDITORY - Research in Auditory Perception >>> [mailto:AUDITORY@xxxxxxxx *On Behalf Of *Schoof, Tim >>> *Sent:* Monday, June 04, 2018 4:06 AM >>> *To:* AUDITORY@xxxxxxxx >>> *Subject:* Registered reports >>> >>> Dear list, >>> >>> I'm going to try and get hearing science journals to start offering >>> registered reports. These reports are basically peer-reviewed >>> pre-registration documents where you outline your methods and propose= d >>> analyses. If this document makes it through peer-review, the >>> manuscript is provisionally accepted for publication. This process >>> should reduce certain questionable research practices, such >>> as=A0selective reporting of results and publication bias. If you're >>> sceptical about registered reports,=A0the Center for Open Science has >>> compiled a nice FAQ list that might address some of your concerns: >>> https://cos.io/rr/ >>> <https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__cos.io_rr_&d=3D= DwMFAg&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQZ_7Fw9yfotck&r=3D2Pw2GwelGcMR49= 53G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DSr0Ep-Gx1c9KJlrgGBL4rmcUvd9qeDUDnFKUymD= oKpI&s=3DvXqZBKaP1dUovPzwBwC5DalLCB6UxwKuM9x_SQCbw5I&e=3D>=20 >>> >>> >>> I think this is the direction science is going in now and it would be >>> great if hearing science joined in. I plan to contact as many hearing >>> science journals as possible. I'm compiling a list of journals to >>> contact. Please add to this list if I'm missing anything: >>> https://tinyurl.com/yaf9r7bk >>> <https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__tinyurl.com_y= af9r7bk&d=3DDwMFAg&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQZ_7Fw9yfotck&r=3D2P= w2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DSr0Ep-Gx1c9KJlrgGBL4rmcUvd9= qeDUDnFKUymDoKpI&s=3Dsk2rFf3fImx-wI9S05uLc7WYgADb5BupEMAQvL3hz-0&e=3D>.=20 >>> >>> I don't think any of these journals offer (or are in the process of >>> offering) registered reports yet, but correct me if I'm wrong. >>> >>> If you agree that registered reports are a good idea and want to sign >>> the letter I intend to send (see here for a template: >>> https://osf.io/3wct2/wiki/Journal%20Requests/ >>> <https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__osf.io_3wct2_= wiki_Journal-2520Requests_&d=3DDwMFAg&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQ= Z_7Fw9yfotck&r=3D2Pw2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DSr0Ep-Gx= 1c9KJlrgGBL4rmcUvd9qeDUDnFKUymDoKpI&s=3DG-jhAt3_0f5cPPX7aRpPgVfihZYm_ZTuP= ohnhVfxWFw&e=3D>),=20 >>> >>> let me know and I'll add you to the list. And please spread the word >>> of course. The more people agree, the more likely it is we can get >>> some of these journals on board! >>> >>> Best, >>> >>> >>> Tim Schoof >>> >>> --=20 >>> >>> Research Associate >>> >>> UCL Speech, Hearing and Phonetic Sciences >>> >>> Chandler House >>> >>> 2 Wakefield Street >>> >>> London WC1N 1PF >>> >>> United Kingdom >>> >> >> >> --=20 ------------------------------------------------------ Julien Besle Assistant Professor Department of Psychology Faculty of Arts and Sciences American University of Beirut Riad El-Solh / Beirut 1107 2020 Lebanon Jesup Hall, Room 103E Tel: +961 1 350 000 ext. 4927 ----------------------------------------------------- --------------E3ADA6FD22290D9ACF6395AE Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by edgeum1.it.mcgill.ca id w56GkgBx014348 <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dwindows-1252"> </head> <body text=3D"#000000" bgcolor=3D"#FFFFFF"> <div class=3D"moz-cite-prefix">I concur with Massimo's remarks. I would like to add the following, in response to some of the emails that have been sent earlier:<br> <br> - I share the concern that not all (good) science is hypothetico-deductive. And certainly pre-registration should not be made mandatory: purely exploratory studies should certainly be allowed, encouraged and publishable. However, they should be published for what they are: exploratory. A result from an exploratory study (or an unexpected result in a pre-registered study) needs to be replicated, even more so than one from a hypothetico-deductive study, so it should carry less weight. Widespread use of pre-registration would help distinguish what results are serendipitous, exploratory or predicted. I really cannot see what's there not to like in this. Like everyone else here I have published results that were unexpected or unpredicted, or I have ran studies that were formulated as questions rather than hypotheses, but I suspect that the way I presented them in the published report sometimes could suggest otherwise, partly because I was encouraged by my supervisors to write a "story", or a narrative around the results. I think that pre-registration would be prevent this to a large extent.<br> <br> - I can also see that pre-registration might not apply to all types of science (see the computer science example from Nilesh). However, I suspect that it could apply to a lot of experimental studies (except purely exploratory ones).<br> <br> - I also understand the concern that pre-registration would increase overhead. In my opinion, this is unlikely and I would think that on the contrary, it would decrease that burden. That a study is pre-approved by peers means that the introduction and part of the methods do not need to be re-reviewed at the second review stage. Also, as reviewers, we would not have to keep wondering whether what is presented as a predicted result was indeed predicted or if history was re-written at the same time as the manuscript (as I suspect is too often the case).<br> <br> - I think we shouldn't underestimate how much we can deceive ourselves (even scientists can) and to what extent adequate incentives can change behaviours. We're deceiving ourselves if we think that self-policy, integrity and ethics can solve problems such as replicability, biased reporting and p-hacking. <br> <br> - I am not sure I see what is wrong in accepting a manuscript "provisionally", "in principle", or even in publication being "guaranteed" once the pre-registration is accepted. If the study has been deemed worthy by reviewers in the first stage, I cannot see why it shouldn't be accepted in the second unless the agreed protocol has not been followed or factual errors are made. Since there is a second round of review, presumably, this means that reviewers can ask authors for revisions and that errors can be corrected. Of course, if the agreed experimental protocol has not been followed, the study should not be published. I think what is meant by "provisionally" is that the manuscript is pre-approved <i>= whatever results come out of the experiment. </i>This is meant to counter biased reviews from reviewers who might want to reject the study because they disagree with the outcome<i>.</i> Provided that=A0 statistical power has been properly evaluated in the first phase, there is nothing wrong with flooding the literature with null results (it needs it).<br> <br> - If one has an a priori hypothesis, tests it and finds that the data do not support it, they should be able to report both the (wrong) a priori hypothesis and the results. This would better reflect the non-linearity of scientific discovery and this is precisely what pre-registration allows and what the current system discourages. This is not HARKing. What constitutes HARKing is when one changes their hypothesis post-hoc but writes the manuscript as if the new post-hoc hypothesis had been their a priori hypothesis all along.<br> <br> - I agree that choosing a specific statistical test in advance would be unnecessarily restrictive. Choosing a statistical test after data have been measured or even looked at is not necessarily p-hacking. If this was the case, then testing the normality of a distribution before deciding whether to use a parametric or a non-parametric test in a small sample (for instance) would be called p-hacking. It's not. In fact, it's recommended. It is p-hacking only if both tests are run and one chooses the best of both outcomes. <br> <br> Dr Julien<br> <br> (My students actually call me Dr Julien, believe it or not).<br> <br> On 06/06/2018 09:09, Massimo Grassi wrote:<br> </div> <blockquote type=3D"cite" cite=3D"mid:28255_1528266196_5B177DD4_28255_141_1_533f202b-ce6c-8e58-226e= -d515f154b519@xxxxxxxx">First of all I would like to thank Tim for the initiative. <br> <br> A few replies and comments: <br> - registered reports have the results section divided in the parts: the <br> "planned analysis" (those you discussed with editor and reviewers) and <br> the "new exploratory analysis". Therefore, I do not see the problem <br> risen by Les. <br> <br> - in my opinion registered reports rise the standard level of current <br> science. Registered reports (like a preregistration but even better) <br> reveal how limited is our ability to predict. It is difficult to predict <br> how the data will look like, what data point will be an outlier, whether <br> data should be analysed in this or that way. We teach to students that <br> the path of science is hypothetical deductive. In reality we move more <br> like a carpenter trying to adapt and adjust things in real time. <b= r> <br> - about the possible "uncontrolled dissemination of null results", I <br> think that (for science) the current uncontrolled dissemination of type <br> I errors is worse. <br> <br> A nice day to everybody from a summer-sunny Italy, <br> <br> m <br> <br> <blockquote type=3D"cite">Dear List, <br> <br> For this topic, I'll violate my rule of not posting replies here.=A0 I <br> agree with Ms. Rankovic.=A0 I sure did not miss the substance and detail <br> of Mr. Schoof's email.=A0 I also read over the information in the links. <br> Indeed, the proposed plan provides for a second review.=A0 It seems to me, <br> however, that the provisional acceptance is a key aspect of the <br> process.=A0 If it were the case that manuscripts were rejected upon second <br> review with substantial frequency, then the philosophy of the registered <br> report would be violated and the system would collapse.=A0 So, unless <br> there are egregious errors or flaws in the full manuscript, it seems <br> that it would be published.=A0 Note that, in this linked referenc= e <br> <a class=3D"moz-txt-link-rfc2396E" href=3D"https://orca.cf.ac.uk/59475/1/AN2.pdf"><https://orca= .cf.ac.uk/59475/1/AN2.pdf></a>, publication is assumed to be <br> "guaranteed." <br> <br> In my opinion, the criticism found within the FAQ here <br> <a class=3D"moz-txt-link-rfc2396E" href=3D"https://cos.io/rr/">&l= t;https://cos.io/rr/></a>, that "The Registered Reports model is based on a <br> na=EFve conceptualisation of the scientific method." is well-founded! The <br> reply offered to counter that criticism is quite weak and unconvincing. <br> I would replace "scientific method" in that criticism with "the way good <br> science is done." <br> <br> Question 17 in Chambers et al. (2014-- linked above) provides an apt <br> example.=A0 In the process of conducting complex experiments, it is very <br> often the case that unexpected results lead to important follow-up or <br> control experiments.=A0 Chambers et al. handle this issue by proposing <br> that in Stage 1 of a registered report, contingencies be stated such <br> that "If A is observed, then we will..."=A0 That, of course, assumes that <br> one knows the decision tree in advance!=A0 In my experience, science <br> simply does not work that way. <br> <br> While I find the intent of registered reports to be laudable, in my <br> opinion, it substitutes one potential set of problems with another based <br> on a narrow view of how science proceeds.=A0 Indeed, one may have a <br> hypothesis to be tested and gather a set of data to address it only to <br> find that the results support a substantially altered view.=A0 Is that, <br> NECESSARILY, the dreaded "HARKing?"=A0 I think not.=A0 Scientific thought <br> and inquiry do not always proceed in a linear fashion.=A0 One cannot and <br> should not always know the precise questions or list of contingencies a <br> priori and be restricted to answering only those.=A0 Then there are <br> experiments in which there are no specific hypotheses.=A0 They ma= y be of <br> the form, "What is the effect of variable A on measurements of X?" <br> Assuming the question is non-trivial, those are often the most revealing <br> experiments because any outcome is of interest.=A0 There is no "positive" <br> or "negative."=A0 Sure, one can cast such experiments in terms of <br> hypotheses but doing so often involves a contrivance. <br> <br> Then there is the matter of "p-hacking" and what I would call <br= > "statistics shopping."=A0 Indeed, it is a problem.=A0 Unexpected outcomes <br> and patterns of data in a complex experiment often require one to choose <br> the appropriate statistic after the fact. It is sometimes the correct <br> thing to do!=A0 Whether it is proper can and should be judged by reviewers <br> with the requisite expertise.=A0 Good peer-review should distinguish <br> between p-hacking and a rational choice that conveys information and <br> "truth."=A0 The notion that one can and should use only the statistic <br> decided upon in advance is unnecessary restrictive. <br> <br> Finally, there is the matter of archival value.=A0 According to Chambers <br> et al., "...if the rationale and methods are sound then the journal <br> should agree to publish the final paper regardless of the specific <br> outcome."=A0 It is often the case that rationale and methods are sound but <br> the data provide no substantial advance or archival value.=A0 I'm not sure <br> that "approving" a method and rationale and virtually guaranteeing <br> publication will afford the same level of judgment in terms of archival <br> value that is afforded by the current system. <br> <br> Les Bernstein <br> <br> -- <br> *Leslie R. Bernstein, Ph.D. **| *Professor <br> Depts. of Neuroscience and Surgery (Otolaryngology)| UConn School of <br> Medicine <br> 263 Farmington Avenue, Farmington, CT 06030-3401 <br> Office: 860.679.4622 | Fax: 860.679.2495 <br> <br> <br> <br> <br> <br> <br> <br> <a class=3D"moz-txt-link-rfc2396E" href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__cos.io_rr_= &d=3DDwMFAg&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQZ_7Fw9yfotck&r= =3D2Pw2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DSr0Ep-Gx1c9KJlrgGB= L4rmcUvd9qeDUDnFKUymDoKpI&s=3DvXqZBKaP1dUovPzwBwC5DalLCB6UxwKuM9x_SQC= bw5I&e=3D"><https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A_= _cos.io_rr_&d=3DDwMFAg&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQZ_7Fw9y= fotck&r=3D2Pw2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DSr0Ep-G= x1c9KJlrgGBL4rmcUvd9qeDUDnFKUymDoKpI&s=3DvXqZBKaP1dUovPzwBwC5DalLCB6U= xwKuM9x_SQCbw5I&e=3D></a> <br> <br> On 6/4/2018 7:51 AM, Christine Rankovic wrote: <br> <blockquote type=3D"cite"> <br> Mr. Schoof: <br> <br> It is beyond ridiculous to accept partial manuscripts for publication. <br> <br> Christine Rankovic, PhD <br> <br> Scientist, Speech and Hearing <br> <br> Newton, MA=A0 USA <br> <br> <a class=3D"moz-txt-link-abbreviated" href=3D"mailto:rankovic@xxxxxxxx">rankovic@xxxxxxxx= on.com</a> <br> <br> *From:*AUDITORY - Research in Auditory Perception <br> [<a class=3D"moz-txt-link-freetext" href=3D"mailto:AUDITORY@xxxxxxxx">mailto:AUDITORY@xxxxxxxx= S.MCGILL.CA</a>] *On Behalf Of *Schoof, Tim <br> *Sent:* Monday, June 04, 2018 4:06 AM <br> *To:* <a class=3D"moz-txt-link-abbreviated" href=3D"mailto:AUDITORY@xxxxxxxx">AUDITORY@xxxxxxxx= L.CA</a> <br> *Subject:* Registered reports <br> <br> Dear list, <br> <br> I'm going to try and get hearing science journals to start offering <br> registered reports. These reports are basically peer-reviewed <br> pre-registration documents where you outline your methods and proposed <br> analyses. If this document makes it through peer-review, the <b= r> manuscript is provisionally accepted for publication. This process <br> should reduce certain questionable research practices, such <br= > as=A0selective reporting of results and publication bias. If you're <br> sceptical about registered reports,=A0the Center for Open Science has <br> compiled a nice FAQ list that might address some of your concerns: <br> <a class=3D"moz-txt-link-freetext" href=3D"https://cos.io/rr/">= https://cos.io/rr/</a> <br> <a class=3D"moz-txt-link-rfc2396E" href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__cos.io_rr_= &d=3DDwMFAg&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQZ_7Fw9yfotck&r= =3D2Pw2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DSr0Ep-Gx1c9KJlrgGB= L4rmcUvd9qeDUDnFKUymDoKpI&s=3DvXqZBKaP1dUovPzwBwC5DalLCB6UxwKuM9x_SQC= bw5I&e=3D"><https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A_= _cos.io_rr_&d=3DDwMFAg&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQZ_7Fw9y= fotck&r=3D2Pw2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DSr0Ep-G= x1c9KJlrgGBL4rmcUvd9qeDUDnFKUymDoKpI&s=3DvXqZBKaP1dUovPzwBwC5DalLCB6U= xwKuM9x_SQCbw5I&e=3D></a> <br> <br> I think this is the direction science is going in now and it would be <br> great if hearing science joined in. I plan to contact as many hearing <br> science journals as possible. I'm compiling a list of journals to <br> contact. Please add to this list if I'm missing anything: <br> <a class=3D"moz-txt-link-freetext" href=3D"https://tinyurl.com/yaf9r7bk">https://tinyurl.com/yaf= 9r7bk</a> <br> <a class=3D"moz-txt-link-rfc2396E" href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__tinyurl.co= m_yaf9r7bk&d=3DDwMFAg&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQZ_7Fw9yf= otck&r=3D2Pw2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DSr0Ep-Gx= 1c9KJlrgGBL4rmcUvd9qeDUDnFKUymDoKpI&s=3Dsk2rFf3fImx-wI9S05uLc7WYgADb5= BupEMAQvL3hz-0&e=3D"><https://urldefense.proofpoint.com/v2/url?u=3D= https-3A__tinyurl.com_yaf9r7bk&d=3DDwMFAg&c=3DEZxp_D7cDnouwj5YEFH= gXuSKoUq2zVQZ_7Fw9yfotck&r=3D2Pw2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3= sWM&m=3DSr0Ep-Gx1c9KJlrgGBL4rmcUvd9qeDUDnFKUymDoKpI&s=3Dsk2rFf3fI= mx-wI9S05uLc7WYgADb5BupEMAQvL3hz-0&e=3D></a>. <br> I don't think any of these journals offer (or are in the process of <br> offering) registered reports yet, but correct me if I'm wrong. <br> <br> If you agree that registered reports are a good idea and want to sign <br> the letter I intend to send (see here for a template: <br> <a class=3D"moz-txt-link-freetext" href=3D"https://osf.io/3wct2/wiki/Journal%20Requests/">https:= //osf.io/3wct2/wiki/Journal%20Requests/</a> <br> <a class=3D"moz-txt-link-rfc2396E" href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__osf.io_3wc= t2_wiki_Journal-2520Requests_&d=3DDwMFAg&c=3DEZxp_D7cDnouwj5YEFHg= XuSKoUq2zVQZ_7Fw9yfotck&r=3D2Pw2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3s= WM&m=3DSr0Ep-Gx1c9KJlrgGBL4rmcUvd9qeDUDnFKUymDoKpI&s=3DG-jhAt3_0f= 5cPPX7aRpPgVfihZYm_ZTuPohnhVfxWFw&e=3D"><https://urldefense.proofp= oint.com/v2/url?u=3Dhttps-3A__osf.io_3wct2_wiki_Journal-2520Requests_&= ;d=3DDwMFAg&c=3DEZxp_D7cDnouwj5YEFHgXuSKoUq2zVQZ_7Fw9yfotck&r=3D2= Pw2GwelGcMR4953G-STHGpPJm2-pYYYSPmTwJk3sWM&m=3DSr0Ep-Gx1c9KJlrgGBL4rm= cUvd9qeDUDnFKUymDoKpI&s=3DG-jhAt3_0f5cPPX7aRpPgVfihZYm_ZTuPohnhVfxWFw= &e=3D></a>), <br> let me know and I'll add you to the list. And please spread the word <br> of course. The more people agree, the more likely it is we can get <br> some of these journals on board! <br> <br> Best, <br> <br> <br> Tim Schoof <br> <br> -- <br> <br> Research Associate <br> <br> UCL Speech, Hearing and Phonetic Sciences <br> <br> Chandler House <br> <br> 2 Wakefield Street <br> <br> London WC1N 1PF <br> <br> United Kingdom <br> <br> </blockquote> <br> <br> <br> </blockquote> </blockquote> <pre class=3D"moz-signature" cols=3D"72">--=20 ------------------------------------------------------ Julien Besle Assistant Professor Department of Psychology Faculty of Arts and Sciences American University of Beirut Riad El-Solh / Beirut 1107 2020 Lebanon Jesup Hall, Room 103E Tel: +961 1 350 000 ext. 4927 -----------------------------------------------------</pre> </body> </html> --------------E3ADA6FD22290D9ACF6395AE--