Subject: Re: [AUDITORY] arXiv web of trust From: Etienne Gaudrain <egaudrain.cam@xxxxxxxx> Date: Wed, 24 May 2023 11:20:59 +0200--0000000000004f194205fc6d06f5 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks for opening this nice debate, Max! I side with Brian for the need of serious peer-review, but I am less sure how this can be achieved nowadays. Publishers are increasingly pressuring reviewers to work fast because their business model relies on volume, and there seems to be little cost to publishing poor quality papers. With the ever precarisation of research, it takes a very strong faith in the ethos of scientific integrity to remain a thorough reviewer. If we accept that, as a consequence of this pressure, there are more flawed papers that pass the review process, it would mean that we, as consumers of the literature, should be more cautious when citing articles. We should more critically examine what we cite, and sort of perform our own review. But of course, that's also very time consuming... and it is also very inefficient at the scale of the community: me *not* citing an article because I found that it is potentially flawed will not prevent others from citing it, and the effort I will have put in reviewing it will be largely wasted. So I do believe that there is a strong benefit in having more open discussions about papers, and in some cases, the fact that they are published or not in the traditional sense, may be partially irrelevant. We definitely don't want to turn the scientific community into social media, where a few arbitrary influencers get to decide what's worthy and what isn't. But there are now places where scientific arguments can be shared, and reflections can be had, constructively. That's what we tried to do for the last edition of the International Symposium on Hearing, but hosting the papers as "pre-print" (for lack of a better term) freely available on Zenodo ( https://zenodo.org/communities/ish2022/), and reviews are made publically available on PubPeer (and more can be added; here's an example: https://pubpeer.com/publications/B12EF572A02E04659AF006FF9C5C91). Contributors are still able to publish their articles in the traditional sense, and hopefully the published version will be connected to the ISH version in some form so that users can view the history and comments. In others words, there is much benefit for the two systems to co-exist (we can get rid of private publishers, though, and switch to decentralized institutional ones). Remains the problem raised by Alain: as readers, how do we deal with the volume? While publishers have been selling us "reputation" in the form of journals in very much overrated ways (such as impact factors, and what not), it is true that journals do have a curating role that should not be underestimated. This being said, editors do not actively seek authors to steer publications towards a specific topic (besides Frontiers' take it all harassment approach). It is still the authors that decide to submit to a specific journal or another. As a result, following the JASA TOC gives us access to a semi-random sample of what's going on in the field. It does offer, stochastically, some degree of protection against confirmation bias in literature search (whereby you only look for papers that confirm your idea). I wonder if automatic suggestions of "related papers" could achieve something similar in other venues? Cheers, -Etienne -- Etienne Gaudrain, PhD Lyon Neuroscience Research Centre / Auditory Cognition and Psychoacoustics (CAP) CNRS UMR5292, Inserm U1028, Universit=C3=A9 Lyon 1 Centre Hospitalier Le Vinatier - B=C3=A2timent 462 - Neurocampus 95 boulevard Pinel, 69675 Bron Cedex, France On Wed, 24 May 2023 at 10:56, Alain de Cheveigne < alain.de.cheveigne@xxxxxxxx> wrote: > Hi Jonathan, all, > > Here's a different perspective. > > First of all, the issue of peer review should be distinguished from that > of publishers shaving the wool off our backs (more below). > > Peer review offers functions that we miss out on in the preprint model. > Weeding out junk is one, improving papers (and the ideas in them) is > another. A third is reducing the bulk of things to read. > > The last might seem counterintuitive: surely, more is better? The thing > is, we have limited time and cognitive bandwidth. Lack of time is the maj= or > obstacle to keeping abreast, and lack of time of the potential audience i= s > what prevents our ideas having an impact. You painstakingly work to solve= a > major problem in the field, write it up carefully, and no one notices > because attention is carried away by the tweet cycle. > > The review/journal model helps in several ways. First, by prioritizing > things to read (as an alternative to the random - or otherwise biased - > selection induced by lack of time). Second, by improving the readability > of the papers: more readable means less time per paper means more attenti= on > for other papers - including possibly yours. Third, by organizing - howev= er > imperfectly - the field. > > For example, you can (or could) keep abreast of a topic in acoustics by > scanning JASA and a few other journals. With the preprint/twitter model t= he > 'field' risks being shattered into micro-fields, bubbles, or cliques. > > My experience of the review process is - as everyone's - mixed. I > remember intense frustration at the reviewer's dumbness, and despair at > ever getting published. I also remember what I learned in the process. > Almost invariably, my papers were improved by orders of magnitude (not ju= st > incrementally). > > I also spend a lot of time reviewing. I find it a painful process, as it > involves reading (I'm a bit dyslexic), and trying to understand what is > written and - to be helpful to the author - what the author had in mind a= nd > how he/she could better formulate it to get the message across, and avoid > wasting the time of - hopefully - countless readers. It does involve > weeding out some junk too. > > Science is not just about making new discoveries or coming up with > radically new ideas. These are few and far between. Rather, it's a slow > process of building on other people's ideas, digesting, tearing down, > clearing the rubble, and building some more. The review process makes the > edifice more likely to stand. Journals play an important role in this > accumulation, even if most content is antiquated and boring. It's a mirac= le > that some journals have done this over decades, even centuries. > > Which brings back to the issue of money, impact factors, and careers. > Lots to say about that, mostly depressing, but mainly orthogonal from the > peer-review issue. > > Alain > > > > > > > On 23 May 2023, at 13:54, Jonathan Z Simon <jzsimon@xxxxxxxx> wrote: > > > > Matt, > > > > In this context I would avoid the term =E2=80=9Cpublishing=E2=80=9D, si= nce that has such > a different meaning for so many people, but I personally do take advantag= e > of posting preprints on a public server (like arXiv) almost every chance = I > get. > > > > Preprints (preprint =3D a fully written paper that is not (yet) publish= ed) > have been useful for many decades, originally in physics, as a way of > getting one's research results out in a timely manner. Other key benefits > are that it establishes primacy of the research findings, that it is > citable in other researchers' papers, and that it can be promoted by soci= al > media such as this listserve (more below on this). But the biggest benefi= t > is typically getting the paper out into the world for others to learn fro= m, > without having to wait based on the whims of publishers and individual > reviewers. If most of your published papers get accepted eventually, and > the most important findings don=E2=80=99t get cut in the review process, = then > preprints are something you should definitely consider. Reviewers often > make published papers better, but maybe not so much better that it=E2=80= =99s worth > waiting many months for others to see your results. > > > > arXiv is the oldest website for posting preprints, and if its Audio and > Speech section is active, that might be a good place to post your > preprints. But there may be other options for you. As an auditory > neuroscientist I typically use bioRxiv (e.g., "Changes in Cortical > Directional Connectivity during Difficult Listening in Younger and Older > Adults=E2=80=9D <https://www.biorxiv.org/content/10.1101/2023.05.19.54150= 0>), but > I also use PsyArXiv if the topic is more perceptual than neural (e.g., > =E2=80=9CAttention Mobilization as a Modulator of Listening Effort: Evide= nce from > Pupillometry=E2=80=9D <https://psyarxiv.com/u5xw2>). [See what I mean abo= ut > promoting your research on social media?] > > > > I=E2=80=99m sure others have opinions too. > > > > Jonathan > > > > > >> On May 22, 2023, at 6:45 PM, Matt Flax <flatmax@xxxxxxxx> wrote: > >> > >> Is anyone publishing on arXiv at the moment ? It seems that to publish > there they rely on a web of trust. > >> > >> There is an Audio and Speech section of arXiv which would suit our > community. > >> > >> thanks > >> > >> Matt > > > > -- > > Jonathan Z. Simon (he/him) > > University of Maryland > > Dept. of Electrical & Computer Engineering / Dept. of Biology / > Institute for Systems Research > > 8223 Paint Branch Dr. > > College Park, MD 20742 USA > > Office: 1-301-405-3645, Lab: 1-301-405-9604, Fax: 1-301-314-9281 > > http://www.isr.umd.edu/Labs/CSSL/simonlab/ > > > > > --0000000000004f194205fc6d06f5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr">Thanks for opening this nice debate, Max!= <div><br></div><div>I side with Brian for the need of serious peer-review, = but I am less sure how this can be achieved nowadays. Publishers are increa= singly pressuring reviewers to work fast because their business model relie= s on volume, and there seems to be little cost to publishing poor quality p= apers. With the ever precarisation of research, it takes a very strong fait= h in the ethos of scientific integrity to remain a thorough reviewer.</div>= <div><br></div><div>If we accept that, as a consequence of this pressure, t= here are more flawed papers that pass the review process, it would mean tha= t we, as consumers of the literature, should be more cautious when citing a= rticles. We should more critically examine what we cite, and sort of perfor= m our own review. But of course, that's also very time consuming... and= it is also very inefficient at the scale of the community: me *not* citing= an article because I found that it is potentially flawed will not prevent = others from citing it, and the effort I will have put in reviewing it will = be largely wasted.</div><div><br></div><div>So I do believe that there is a= strong benefit in having more open discussions about papers, and in some c= ases, the fact that they are published or not in the traditional sense, may= be partially irrelevant. We definitely don't want to turn the scientif= ic community into social media, where a few arbitrary influencers get to de= cide what's worthy and what isn't. But there are now places where s= cientific arguments can be shared, and reflections can be had, constructive= ly.</div><div><br></div><div>That's what we tried to do for the last ed= ition of the International Symposium on Hearing, but hosting the papers as = "pre-print" (for lack of a better term) freely available on Zenod= o (<a href=3D"https://zenodo.org/communities/ish2022/">https://zenodo.org/c= ommunities/ish2022/</a>), and reviews are made publically available on PubP= eer (and more can be added; here's an example: <a href=3D"https://pubpe= er.com/publications/B12EF572A02E04659AF006FF9C5C91">https://pubpeer.com/pub= lications/B12EF572A02E04659AF006FF9C5C91</a>). Contributors are still able = to publish their articles in the traditional sense, and hopefully the publi= shed version will be connected to the ISH version in some form so that user= s can view the history and comments. In others words, there is much benefit= for the two systems to co-exist (we can get rid of private publishers, tho= ugh, and switch to decentralized institutional ones).</div><div><br></div><= div>Remains the problem raised by Alain: as readers, how do we deal with th= e volume? While publishers have been selling us "reputation" in t= he form of journals in very much overrated ways (such as impact factors, an= d what not), it is true that journals do have a curating role that should n= ot be underestimated. This being said, editors do not actively seek authors= to steer publications towards a specific topic (besides Frontiers' tak= e it all harassment approach). It is still the authors that decide to submi= t to a specific journal or another. As a result, following the JASA TOC giv= es us access to a semi-random sample of what's going on in the field. I= t does offer, stochastically, some degree of protection against confirmatio= n bias in literature search (whereby you only look for papers that confirm = your idea). I wonder if automatic suggestions of "related papers"= could achieve something similar in other venues?</div><div><br></div><div>= Cheers,<br>-Etienne</div><div><br></div><div><div><br></div><div><span styl= e=3D"color:rgb(117,123,128);font-family:Corbel,Skia,sans-serif;font-size:13= .3333px">--</span><br style=3D"color:rgb(33,33,33);font-family:wf_segoe-ui_= normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,se= rif,EmojiFont;font-size:13.3333px"><span style=3D"color:rgb(117,123,128);fo= nt-family:Corbel,Skia,sans-serif;font-size:13.3333px">Etienne Gaudrain, PhD= </span><br style=3D"color:rgb(33,33,33);font-family:wf_segoe-ui_normal,&quo= t;Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFo= nt;font-size:13.3333px"><br style=3D"color:rgb(33,33,33);font-family:wf_seg= oe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-se= rif,serif,EmojiFont;font-size:13.3333px"><span style=3D"color:rgb(117,123,1= 28);font-family:Corbel,Skia,sans-serif;font-size:13.3333px">Lyon Neuroscien= ce Research Centre / Auditory Cognition and Psychoacoustics (CAP)</span><br= style=3D"color:rgb(33,33,33);font-family:wf_segoe-ui_normal,"Segoe UI= ","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont;font-si= ze:13.3333px"><span style=3D"color:rgb(117,123,128);font-family:Corbel,Skia= ,sans-serif;font-size:13.3333px">CNRS UMR5292, Inserm U1028, Universit=C3= =A9 Lyon 1</span><br style=3D"color:rgb(33,33,33);font-family:wf_segoe-ui_n= ormal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,ser= if,EmojiFont;font-size:13.3333px"><span style=3D"color:rgb(117,123,128);fon= t-family:Corbel,Skia,sans-serif;font-size:13.3333px">Centre Hospitalier Le = Vinatier - B=C3=A2timent 462 - Neurocampus</span><br style=3D"color:rgb(33,= 33,33);font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP&q= uot;,Tahoma,Arial,sans-serif,serif,EmojiFont;font-size:13.3333px"><span sty= le=3D"color:rgb(117,123,128);font-family:Corbel,Skia,sans-serif;font-size:1= 3.3333px">95 boulevard Pinel, 69675 Bron Cedex, France</span><br></div></di= v></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr= "><br></div></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" cla= ss=3D"gmail_attr">On Wed, 24 May 2023 at 10:56, Alain de Cheveigne <<a h= ref=3D"mailto:alain.de.cheveigne@xxxxxxxx">alain.de.cheveigne@xxxxxxxx<= /a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0= px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">H= i Jonathan, all,<br> <br> Here's a different perspective. <br> <br> First of all, the issue of peer review should be distinguished from that of= publishers shaving the wool off our backs (more below).<br> <br> Peer review offers functions that we miss out on in the preprint model. Wee= ding out junk is one, improving papers (and the ideas in them) is another. = A third is reducing the bulk of things to read. <br> <br> The last might seem counterintuitive: surely, more is better?=C2=A0 The thi= ng is, we have limited time and cognitive bandwidth. Lack of time is the ma= jor obstacle to keeping abreast, and lack of time of the potential audience= is what prevents our ideas having an impact. You painstakingly work to sol= ve a major problem in the field, write it up carefully, and no one notices = because attention is carried away by the tweet cycle.<br> <br> The review/journal model helps in several ways. First, by prioritizing thin= gs to read (as an alternative to the random - or otherwise biased - selecti= on induced by lack of time).=C2=A0 Second, by improving the readability of = the papers: more readable means less time per paper means more attention fo= r other papers - including possibly yours. Third, by organizing - however i= mperfectly - the field. <br> <br> For example, you can (or could) keep abreast of a topic in acoustics by sca= nning JASA and a few other journals. With the preprint/twitter model the &#= 39;field' risks being shattered into micro-fields, bubbles, or cliques.= <br> <br> My experience of the review process is - as everyone's - mixed.=C2=A0 I= remember intense frustration at the reviewer's dumbness, and despair a= t ever getting published. I also remember what I learned in the process.=C2= =A0 Almost invariably, my papers were improved by orders of magnitude (not = just incrementally). <br> <br> I also spend a lot of time reviewing. I find it a painful process, as it in= volves reading (I'm a bit dyslexic), and trying to understand what is w= ritten and - to be helpful to the author - what the author had in mind and = how he/she could better formulate it to get the message across, and avoid w= asting the time of - hopefully - countless readers. It does involve weeding= out some junk too. <br> <br> Science is not just about making new discoveries or coming up with radicall= y new ideas. These are few and far between. Rather, it's a slow process= of building on other people's ideas, digesting, tearing down, clearing= the rubble, and building some more. The review process makes the edifice m= ore likely to stand. Journals play an important role in this accumulation, = even if most content is antiquated and boring. It's a miracle that some= journals have done this over decades, even centuries.<br> <br> Which brings back to the issue of money, impact factors, and careers.=C2=A0= Lots to say about that, mostly depressing, but mainly orthogonal from the = peer-review issue.<br> <br> Alain<br> <br> <br> <br> <br> <br> > On 23 May 2023, at 13:54, Jonathan Z Simon <<a href=3D"mailto:jzsim= on@xxxxxxxx" target=3D"_blank">jzsimon@xxxxxxxx</a>> wrote:<br> > <br> > Matt,<br> > <br> > In this context I would avoid the term =E2=80=9Cpublishing=E2=80=9D, s= ince that has such a different meaning for so many people, but I personally= do take advantage of posting preprints on a public server (like arXiv) alm= ost every chance I get.<br> > <br> > Preprints (preprint =3D a fully written paper that is not (yet) publis= hed) have been useful for many decades, originally in physics, as a way of = getting one's research results out in a timely manner. Other key benefi= ts are that it establishes primacy of the research findings, that it is cit= able in other researchers' papers, and that it can be promoted by socia= l media such as this listserve (more below on this). But the biggest benefi= t is typically getting the paper out into the world for others to learn fro= m, without having to wait based on the whims of publishers and individual r= eviewers. If most of your published papers get accepted eventually, and the= most important findings don=E2=80=99t get cut in the review process, then = preprints are something you should definitely consider. Reviewers often mak= e published papers better, but maybe not so much better that it=E2=80=99s w= orth waiting many months for others to see your results.<br> > <br> > arXiv is the oldest website for posting preprints, and if its Audio an= d Speech section is active, that might be a good place to post your preprin= ts. But there may be other options for you. As an auditory neuroscientist I= typically use bioRxiv (e.g., "Changes in Cortical Directional Connect= ivity during Difficult Listening in Younger and Older Adults=E2=80=9D <<= a href=3D"https://www.biorxiv.org/content/10.1101/2023.05.19.541500" rel=3D= "noreferrer" target=3D"_blank">https://www.biorxiv.org/content/10.1101/2023= .05.19.541500</a>>), but I also use PsyArXiv if the topic is more percep= tual than neural (e.g., =E2=80=9CAttention Mobilization as a Modulator of L= istening Effort: Evidence from Pupillometry=E2=80=9D <<a href=3D"https:/= /psyarxiv.com/u5xw2" rel=3D"noreferrer" target=3D"_blank">https://psyarxiv.= com/u5xw2</a>>). [See what I mean about promoting your research on socia= l media?] <br> > <br> > I=E2=80=99m sure others have opinions too.<br> > <br> > Jonathan<br> > <br> > <br> >> On May 22, 2023, at 6:45 PM, Matt Flax <<a href=3D"mailto:flatm= ax@xxxxxxxx" target=3D"_blank">flatmax@xxxxxxxx</a>> wrote:<br> >> <br> >> Is anyone publishing on arXiv at the moment ? It seems that to pub= lish there they rely on a web of trust.<br> >> <br> >> There is an Audio and Speech section of arXiv which would suit our= community.<br> >> <br> >> thanks<br> >> <br> >> Matt<br> > <br> > --<br> > Jonathan Z. Simon (he/him)<br> > University of Maryland<br> > Dept. of Electrical & Computer Engineering / Dept. of Biology / In= stitute for Systems Research<br> > 8223 Paint Branch Dr.<br> > College Park, MD 20742 USA<br> > Office: 1-301-405-3645, Lab: 1-301-405-9604, Fax: 1-301-314-9281<br> > <a href=3D"http://www.isr.umd.edu/Labs/CSSL/simonlab/" rel=3D"noreferr= er" target=3D"_blank">http://www.isr.umd.edu/Labs/CSSL/simonlab/</a><br> > <br> > <br> </blockquote></div> --0000000000004f194205fc6d06f5--