Obviously, the first step to fix the Interpreter would be for the Interpreters to want to fix the Interpreter. But they're happy with it the way it is because their feedback loop confirms their biases.
Which makes this more of an aspirational post.
The article below from the WSJ sets out the problem and proposes a solution.
Key excerpts that apply to the Interpreter.
Scientists and scholars are susceptible to career incentives, moral fads, groupthink and fear. When those pressures capture journals or entire fields, peer review can become less a filter for error than a credentialing system for fashionable nonsense....
If reviewers have the same blind spots as the editors and authors, then a process meant to remove flaws and bias can instead facilitate them.
Article:
Excerpts:
A Way to Challenge the Groupthink of Scholarly Journals
Peer review has become a closed system that protects shoddy and politically motivated research.
Updated ET
We’re often told that science is “self-correcting.” But science isn’t like a thermostat regulating your home’s temperature. It’s a human institution run by fallible human beings. Scientists and scholars are susceptible to career incentives, moral fads, groupthink and fear. When those pressures capture journals or entire fields, peer review can become less a filter for error than a credentialing system for fashionable nonsense.
Modern prepublication peer review became common in the mid-20th century. At its best, peer review improves papers before publication and screens out weak work, but its usefulness depends on the quality and independence of a field’s “expert” reviewers. If reviewers have the same blind spots as the editors and authors, then a process meant to remove flaws and bias can instead facilitate them.
Decades of studies on publication bias, replication failures and political bias in the social sciences have shown that peer-reviewed papers are often less reliable than the public assumes. ...
In fields that rely heavily on narrative or qualitative methods, or that touch on politicized topics (as much social science does), ideology influences which questions are asked and which conclusions are professionally acceptable.
Sometimes the failure is so obvious it’s comical. ...
Two decades later, James Lindsay, Helen Pluckrose and Peter Boghossian conducted a more elaborate “grievance studies” hoax. They managed to place absurd papers in peer-reviewed journals by dressing their claims in progressive academic language. One paper argued that dog parks could illuminate “rape culture” and “queer performativity.” Another rewrote portions of Hitler’s “Mein Kampf” in the style of feminist theory. The scandal wasn’t merely that hoax papers were published, but that they were indistinguishable from actual scholarship in those fields.
This problem is growing more serious. Across swaths of the humanities, social sciences, medicine and biology, some narratives have become taboo. Papers presenting contrary evidence or dissenting viewpoints are rejected without comment. Letters to the editor, which are supposed to provide a quick way to respond to flawed work, are ignored or unavailable. The result is an ideologically biased literature that’s presented as an expert consensus and cited by journalists, courts, school boards, medical associations, government agencies and lawmakers to justify policies that affect millions of people.
The most obvious answer is better peer review. But ideologically captured fields have little incentive to correct themselves.
...
This is where the myth of “self-correcting” science becomes a problem. People assume the system will fix itself, but first someone has to notice the problem and create a mechanism for correction.
That is what we have done. As an editor-in-chief and a member of the editorial advisory board of Theory and Society, an interdisciplinary journal published by Springer Nature, we are proud to announce a first-of-its-kind article type called “Peer Review.” The purpose is to avoid procedural traps that can prevent legitimate criticism from being published and to recover what peer review was supposed to be: serious, good-faith analysis by experts seeking clarity and truth.
As in postpublication peer review, a Peer Review article may address a paper from any scholarly journal so long as it raises concerns about methods, evidence, logic, definitions or theory. The focus must be on claims, arguments and scholarly standards, not the author’s character or motives.
Submissions, limited to 2,500 words, will undergo a simple merit review rather than endless rounds of gatekeeping. An editor or subject-matter expert will ask a straightforward question: Is this critique coherent, serious and reasonable enough to deserve scholarly attention? If so, it will be accepted.
The format includes a built-in right of reply. Once a Peer Review article is provisionally accepted, the authors of the critiqued paper will be notified and invited to respond. Readers will be able to see the critique and the reply in a legitimate academic venue.
Science becomes self-correcting only when scholars create the institutional mechanisms that make it possible. Our new Peer Review format is one such mechanism. Now it’s time for academics to use it.
Publication should mark the beginning of academic scrutiny, not the end of it.
Mr. McCaffree is an associate professor of sociology at the University of North Texas and editor-in-chief of Theory and Society. Mr. Wright is a fellow at the Manhattan Institute and an editorial adviser for Theory and Society.
No comments:
Post a Comment