Subject: Re: [AUDITORY] Registered reports From: Nilesh Madhu <000000405df1884c-dmarc-request@xxxxxxxx> Date: Wed, 6 Jun 2018 10:57:33 +0200 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>Good morning Bas, I see your point and I do agree that, in cases such as those you=20 mention, pre-defining and getting feedback on the trial procedure you=20 wish to conduct makes sense. However, for such scenarios, shouldn't=20 there be industry-wide standards on testing/trials and reporting in=20 place already? If someone intentionally conducts a poorly conceived trial, the paper=20 would/should be rejected in the peer-review anyway ("reject/do not=20 encourage resubmission"). As I see it, registered reports can be helpful=20 if researchers are not sure what trials they should conduct and want=20 early feedback. Even here, perhaps, a simpler solution would be to make=20 guidelines available, instead of enforcing an 'administrative' layer. Of course, all of this is assuming that the intent is to make registered=20 reports compulsory for any/every article. If this is not the case, the=20 point is moot :) Greetings from lovely, sunny Belgium (yes, we do have such days!) Nilesh PS: Massimo, I like your point about the carpenter being adaptive on the=20 job. Previously I attributed this necessity to adapt to my poor skills=20 ;) On 2018-06-06 09:35, Bas Van Dijk wrote: > Hi Nilesh, >=20 > I agree to certain extend but I do feel that registered reporting > makes sense for 'close to product' trials and trials that lead to > treatments (for example evaluation of a fitting algorithm). In fact, > it should not really be ' double work' as you fear because if you > execute a poor trial and then try to get it published (believe me.. it > happens :-) )but it gets rejected and you basically have no option but > to redo (part of) the work. (and -re-writing the text to get a poor > trial accepted for publication is of course exactly what you don't > want...). That's more double work that writing up a good trial > proposal, have it reviewed and then know that if you execute according > to plan it's likely to get published even if the results are negative > or non-conclusive, that could be a pre as well. >=20 > Best wishes, > Bas >=20 >=20 > Bas Van Dijk > Program Manager, A&A - Clinician and Research Tools >=20 > Cochlear Technology Centre Belgium > Schali=C3=ABnhoevedreef 20 I > 2800 Mechelen > BELGIUM >=20 > Phone: +3215795528 > Mobile: +32473976270 > Email: BVanDijk@xxxxxxxx > www.cochlear.com >=20 > -----Original Message----- > From: AUDITORY - Research in Auditory Perception > [mailto:AUDITORY@xxxxxxxx On Behalf Of Nilesh Madhu > Sent: dinsdag 5 juni 2018 13:16 > To: AUDITORY@xxxxxxxx > Subject: [AUDITORY] Registered reports >=20 > Dear Tim, >=20 > I appreciate your initiative towards reproducible research. However I > fear that registered reports would just add another layer of overhead > to academics and students already under the pressure to publish. If I > understand correctly, this involves two rounds of review: a first > review based on the methodology and evaluation and a second based on > the results of the research. For each stage, probably at least two > review rounds would be needed (going by the current publishing cycle). > I fear, as Gaston does, this might stifle creativity and lead to > overwork also for reviewers and editors. Of course, this is assuming > you want to make registered reports compulsory... >=20 > Furthermore, such an approach may not be equally applicable to all > research. For research into algorithms, for example, the value of the > research lies, usually, in the core idea. There are myriad accepted > forms of evaluation and to force a strict evaluation > pattern/methodology would be counterproductive. Reproducible research > in this case is targeted by encouraging authors to make their code and > test data public. >=20 > What I would support are (voluntary) guidelines on reporting results > of experiments. This is often to be found in in the engineering field, > when one participates in an open challenge. >=20 > Lastly, the main reason for this initiative is to avoid 'mis-reporting' > the results in favour of a hypothesis. Surely, this calls for self > policing? Aren't we, as researchers, possessed of sufficient integrity > and ethics to present our research in the correct light? If this core > value is missing, I fear no external policing is going to help. >=20 > Best regards > Nilesh Madhu >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > "The information contained in this e-mail message may be confidential > information, and may also be privileged. If you are not the intended > recipient, any use, interference with, disclosure or copying of this > material is unauthorised and prohibited. If you have received this > message in error, please notify us by return email and delete the > original message."