Re: [AUDITORY] [External] Re: [AUDITORY] Biases in career evolution

Dear Deniz,

Your example is exactly the problem with these unprinted/unreviewed manuscripts. They are not reviewed and do not merit the same consideration. They are, as a consequence, more often flawed than published peer-reviewed works.

Just as with a conference proceedings, the flaw remains. I always look for the peer-reviewed version. If there isn't one, it is a likely sign the paper was too flawed to simply correct. There is true meaning is this. Hence, the misleading role of socalled preprints we have evoked here.

Brian FG Katz

Equipe LAM : Lutheries Acoustique Musique

Sorbonne Université, CNRS, Institut ∂'Alembert

-------- Original message --------

From: "Baskent, D (kno)" <00000187abab8d23-dmarc-request@xxxxxxxxxxxxxxx>

Date: 6/11/23 11:17 (GMT+01:00)

To: AUDITORY@xxxxxxxxxxxxxxx

Subject: Re: [AUDITORY] [External] Re: [AUDITORY] Biases in career evolution

Dear all, theBob,

These are good thoughts. I also wanted to add to it, that, we are all equally responsible for the turn-around times and fairness of peer review as we are each others' reviewers. So if we collectively decide we should speed up the process and we should be fairer, we should all chime in.

I have one question, coming from my own unfamiliarity with preprints: It happens at times that a paper has a flaw, for example, in study design or something equally crucial, and this only becomes clear during peer-review. What happens to the pre-print then, and is there a way to make it public also of such flaw? I assume newer versions or peer-reviewed and published versions can be added, but I am talking about a situation where the preprint may have to be withdrawn. Is there a concern that by that time some potentially faulty information was already disseminated?

Cheers,

Deniz

--------------------------
Prof. dr. ir. D. Başkent

Speech Perception Lab (dB SPL)

Department of Otorhinolaryngology
School of Behavioral and Cognitive Neuroscience (BCN)

W.J. Kolff Institute for Biomedical Engineering and Materials Science

University Medical Center Groningen (UMCG)
University of Groningen (RUG)

Tel: +31 (0) 50 3612540 (KNO Office)

Mobile: ‭+31 6 25651377‬

Visiting address: UMCG, Hanzeplein 1, Room P4.220

Website (also for dB SPL): dbaskent.org

Van: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxxxxxxxxx> namens McMurray, Bob <bob-mcmurray@xxxxxxxxx>
Verzonden: zaterdag 10 juni 2023 16:21
Aan: AUDITORY@xxxxxxxxxxxxxxx <AUDITORY@xxxxxxxxxxxxxxx>
Onderwerp: Re: [External] Re: [AUDITORY] Biases in career evolution

Hi Colleagues-

This has been a really interesting discussion so far, and I'm glad these issues are coming out, even if they raise some uncomfortable issues.

I'd like to redirect back to Arxiv services. Why are these related?

Start from the fact that scientific publishing is slow. We can all agree on that. Arxiv services can be fast. In fact, I recently submitted a paper to JASA and simultaneously posted it to PsyArxiv. A colleague saw it that same day and tweeted it out. Probably a few hundred people saw their tweet and some of them even wrote me! Four days later, it made it through JASA preprocessing and arrived on that same colleague's desk as the action editor! My paper literally made it out to 200 colleagues before it even got to the same person to send out for review. Sigh.

But now consider how these delays and barriers might relate to the diversity of voices in auditory sciences: people of different genders, races and nationalities, people at different career stages (students , pre-trenure faculty, and even senior colleagues who are shifting into auditory science), people with significant responsibilities outside of work, emeritus faculty.

This leads to two ideas. First, different people face different career and personal demands. When you consider it from this perspective, Arxiv services can play an extremely valuable role. Second, the lags in peer review may manifest very differently for different people. Even if a paper is objectively good and accepted on those grounds, a new participant in the field, may be subject to additional scrutiny (an extra round of review, additional issues that need to be clarified) than a more established one. Some people may face longer delays because they have additional teaching (e.g., they are at a smaller college), family responsibilities, or a large lab with lots of demands. All of this means that even if peer-review is objective, it is not equal for different people.

This is not to bypass the need for peer review -- that is the final arbiter of scientific acceptability (at least until something better comes along). But in light of all of this, pre-prints contribute to addressing a number of problems.

Publication rates are slow and variable. The speed at which a paper gets through peer-review can also be subject to all sorts of personal factors. Tt is thus easy to be "scooped" on a new finding, not because you actually discovered it later, but because the journal that you chose has a longer review process or a longer time to publication (that happened to my own dissertation), or because your personal factors lead to a longer delay. A preprint can get your finding out there with a DOI and a date on it.
Peer-review is designed to advise a Journal editor, not the scientist. It is often not constructive and quite discouraging. The kind of comments one gets on a pre-print are very different . We all like to talk about the people who succeeded despite their challenges. But as David pointed out -- what about the people who are discouraged and leave the field? Preprints can help people get supportive advice and connect with the people that can help mentor them.
Some of us may be the only auditory scientist at our institution. For an early career scientist. or someone from a traditionally excluded group, preprints, academic social media, and the kind of informal conversations that result from these things can be a helpful source of networking.
Some of us are working on a timeline. A doctoral student only has 5 years before they have to go on the market, a new faculty member may only have 6 years before tenure (and three for a contract renewal), and many of us face 5 year deadlines for grant renewals. A preprint is not a true scientific product and should never be treated that way. But it is a way to demonstrate to the field (grant reviewers, search committees, and faculty evaluation committees) that the work is "done" (even if it will be 2-3 years before it is actually accepted).
Much of our evaluation relies on reputation (right or wrong). We ask for letters when people go up for promotion. Grants are often reviewed (right or wrong) in part on the reputation of the scholars. This should not be subject to the 1-3 year delay imposed by traditional publishing. For a new scientist (or an old scientist switching fields) pre-prints can help build reputation while the peer-review process unfolds. seen my own students get great feedback from senior colleagues in response to a preprint and it's made a difference in their careers.
Finally, it stinks to finish a well-crafted study only to discover that another lab is working on something similar or has just finished it. More established people avoid this easily -- we have an established network, we communicate directly to each other, and we generally know the kinds of things that we're all working on. However, newer people—or people who have struggled to build that network—may struggle to access this. Conferences obviously help, but with three ASA, ARO, AAAS, and ASHA (not to mention all the conferences that aren't quite auditory but are relevant ), people can't just travel enough. Preprints help.

I apologize in advance for the discoursive email. But I was sucked into Arxiving kicking and screaming by my own junior colleagues and students. I'm glad I did. And I haven't really seen the benefits of preprints articulated yet. This is important not just for younger people but for older people too. I'm strongly committed to peer review (and do a lot of it), although we can all make it work better. But the benefits of preprints -- particularly to emerging scholars -- far outweigh the few minutes it takes to post.

But that said, we should all be thinking of how these issues pervade our scientific culture... including peer review. Reviewer 2 could take a few extra minutes to be kind and offer helpful advice.. Reviewer 2 might find it useful to remember that that one minor experiment they asked for might be a lot more difficult if the student who wrote the paper has left the field, the faculty member is at a teaching college with less access research materials, the grant that funded the work is expired, or someone had to take a leave of absence to care for a family member (all situations that result in papers that never get published but live on preprint servers). Science should be objective, but it is a personal exercise for all of us and it is silly to ignore that.

In my view, many of these issues are being led by a new generation of younger scientists who are more diverse, both intellectually and demographically. We would be wise to listen.

theBob

From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxxxxxxxxx> on behalf of David McAlpine <david.mcalpine@xxxxxxxxx>
Sent: Saturday, June 10, 2023 4:48 AM
To: AUDITORY@xxxxxxxxxxxxxxx <AUDITORY@xxxxxxxxxxxxxxx>
Subject: [External] Re: [AUDITORY] Biases in career evolution

I strongly disagree Brian. The explicit connection of review with general bias operates out of those leading scientific nations that host the important journals and from which the vast majority of reviewers are drawn. These are inseparable.

Sent from my iPad

> On 10 Jun 2023, at 7:44 pm, Brian FG Katz (SU) <brian.katz@xxxxxxxxxxxxxxxxxxxxxx> wrote:
>
>
> With the aim of providing at least a clearer forum for this discussion, let us at least provide a relevant message header.
>
> I would then only like to add/point out that which scientific questions and peer-review journal publications are international by nature and affect us all equally, questions of gender/racial/religious/economic/nationality/genetic/age/etc. biases *and how they are being addressed* is highly cultural and regional around the world, even specific to different institutions. As such, generalizations and observations of the presence of issues, or lack thereof, are going to be equally regional in nature.
>
> I would therefore only recommend that if there are any further discussions on this topic here, in this international forum, that such caveats are considered when both presenting and defending arguements.
>
> --
> Brian FG Katz
> Equipe LAM : Lutheries Acoustique Musique
> Sorbonne Université, CNRS, Institut ∂'Alembert
>
>

De inhoud van dit bericht is vertrouwelijk en alleen bestemd voor de geadresseerde(n). Anderen dan de geadresseerde(n) mogen geen gebruik maken van dit bericht, het niet openbaar maken of op enige wijze verspreiden of vermenigvuldigen. Het UMCG kan niet aansprakelijk gesteld worden voor een incomplete aankomst of vertraging van dit verzonden bericht.

The contents of this message are confidential and only intended for the eyes of the addressee(s). Others than the addressee(s) are not allowed to use this message, to make it public or to distribute or multiply this message in any way. The UMCG cannot be held responsible for incomplete reception or delay of this transferred message.