[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [AUDITORY] arXiv versus the peer machines we trust



The history of the peer review practice is fascinating. I definitely agree with Matt that the Ivory tower is still unobtainable for large segments of society and it is definitely growing taller and  harder to enter in a lot of societies, I am grateful for the opportunity to have access in my own way.

Often we can tease out people's theoretical opinions (and sometimes come to surprising results) by taking a complex system and performing peripheral mind experiments at the edge of what is currently known and common opinion.

If Kurzweil is right, then it is likely that by 2029 computers will be able to reason on par with humans. The question of what the implications are around machines of utility being available on the large scale for creating more accessible information, support, companionship and probably learning for people is still open. One thing is for certain, if a company or institution can speed its system with machines, it probably will.

If future peer reviews in journals are conducted by machines, in an effort to speed up the system, improve bandwidth, profit and compete with preprint servers, is that still legitimate peer reviewed publishing in your opinion? Does this machine led scenario change anything for anyone who uses preprint servers ? Does this machine led scenario change anything for people who use peer reviewed journals ? Are we going to have/need a third class of publishing which is biologically based ?

We have a spread of classical to modern preference for information dissemination. In a little more detail: Dick pointed out earlier (in the quality vs speed thread) that he was happy to  publish his Gamma Distribution research on a preprint server and wait for peer reviewed publication (then was faced with maintaining the arXiv version). In that way some people are happy to use both modes of release. Then some people are pretty keen to maintain classical peer review mechanisms, whilst others prefer a more modern approach for various reasons.

Do people with classical preference get split into classical and neoclassical ?
How does the parameter space change from our current one dimensional (possibly periodic) spread ?

Matt

On Thu, Jun 1, 2023 at 2:12 PM Matthew Winn <0000011b522b2e6a-dmarc-request@xxxxxxxxxxxxxxx> wrote:
There are statements in this thread that cannot go unchallenged, because they condone and perpetuate harmful ideas that need to end. Specifically:
1) “If one is not a sufficiently confident and independent thinker such that one can express ideas, arguments, disagreements, etc. with anyone in the field, regardless of stature, then that is a weakness”
This statement ignores the multiple power structures that affect the lives and employment of those below the ‘upper echelon’ in the field. Expressing an idea involves risk when your position is precarious. Adapting to and weighing that risk is a key survival strategy, not a weakness. I have a blind spot for this risk – not because I’m so great at science, but because my culture gives me unearned respect because of my demographics. For people like me (and, I will note, virtually everyone on this thread), we live in a culture that insulates us from any sense that our voice doesn’t belong.

2) “think about how such researchers earned such status.  It was not because they had friends, it was not because people liked them.  It was because they established a track-record of contributions that the field, in general, held in very high regard.”
This is a self-serving narrative that reflects survivorship bias and which ignores everything we know about how people act in real life. Science is done by humans, who have personal interests, biases, and who live within a culture where status is built on many layers of privilege. Every decision we make is filtered by these factors, which allow some people (like me) to accumulate a variety of advantages at every career stage, simply because of how they look, who their friends are, and where they grew up. They are more likely to have papers accepted, to be selected for podium presentations, to have a job application reviewed, to be interviewed, to be hired, to be selected as editors and reviewers, to be elected to positions of leadership, and to be given favorable treatment in the workplace. To be taken seriously. If we pretend that these advantages are ALL due to the scientific merit of one’s work, we are characterizing scientists as some species entirely separate from the rest of humanity.

3) “Stature does not count. Everyone should be held to the very same standard”
We all agree that work should not be judged on the basis of who wrote it. But importantly, the influence of stature doesn’t need to be explicitly suggested in order to actually take place. Similar to the last point, the idea of equal standards and equal treatment is a convenient fiction that allows people like me to feel superior because I can attribute my success to my own hard work and merit, even though many factors that led to that success were unearned.

=========== ==============
What does this have to do with preprints? The point is to consider that others have a different set of constraints, and that our definitions of merit are tailored to suit those who are already enjoying a wide variety of privileges. Consider the forces that lead authors to think that preprints are useful, and also whether you are facing the same expectations and constraints that they are. Numerous people have pointed at the apparent generational divide on this issue - let's figure out why. Graduate admissions and fellowship review increasingly expect a publication record that far exceeds anything that would have been expected of the reviewers when they were at that same career stage. For various reasons, the timeline of publication is increasingly long. Exacerbating this, it is no longer enough to simply conduct a good study; one must also curate a data management and sharing plan that includes open-access data and documented code. One must learn and conduct the latest statistical techniques that their advisors never needed to learn, and sift through a much broader set of literature that includes a lot of garbage. To compete for stable employment, younger scholars need an internet presence and must learn to incorporate inclusive language in their writing, even if that were not part of their training. They need to express how their work contributes to the reduction of harm in society, despite being advised by some of the people who are doing the harm.

Preprints are not a magical solution that can eliminate the multiple barriers that I described above. But they have tangible value, and reflect adaptation to a changing academic landscape, rather than reflecting some loss of “standards” that are designed to protect those already at the top, and which were established under an entirely different system of constraints.
 
Preprints help address the needs for 1) visibility and 2) quicker feedback on your work from a wider variety of scholars who might not have been invited to review, simply because they were not in the network of the associate editor. These factors are often yoked together; the channels that spread awareness of a preprint (like Twitter) might also be the same channels that generate discussion that becomes useful feedback. The tendency (or need) to use these dissemination channels probably reinforces the generational divide on this thread. I assure you that the comments I've received from people enthusiastic enough to read a preprint have had meaningful influence and value. And those comments can come from a wider variety of people whose opinions have been historically discounted. Experienced reviewers will always have a place in our scientific discourse, but to discount the benefit and potential of preprints is to be willfully detached from our current reality.

Matt