Preprint research platform arXiv recently announced that review articles and position papers submitted to its computer science category must be “accepted at a journal or a conference and complete successful peer review” in order to qualify for publication.
This has, technically, always been the rule. According the fine print on arXiv’s website, publication of reviews and position papers has always been at the discretion of moderators.
But longtime fans of the site’s computer science section know that “moderators discretion” has basically meant “open the floodgates.” Now, thanks to the magic of AI, a preprint position paper has to be published before it can be pre-published — like a carriage dragging a horse behind it.
The reason behind the sudden shift in policy? The site’s computer science category has experienced an increasing flood of worthless, AI-generated papers making it impossible for the site’s volunteer moderators to keep up.
“Generative AI/large language models have added to this flood by making papers – especially papers not introducing new research results – fast and easy to write,” the organization said.
The blog post noted that other categories had also received an increase in submissions over the past few years, but the influx was “particularly pronounced in the CS category.”
ArXiv clearly did the right thing here and the AI-generated slop won’t be missed. But there’s a small part of me that laments what we’re losing.
Some of the most entertaining science writing I’ve ever laughed and grinned my way through came from arXiv’s CS section. And, while I won’t name names or post links, the most ridiculous and wacky ones were most certainly AGI literature reviews and position papers that were written by humans.
It’s sad to think we might be losing that (and perhaps even some good science, too!). But there’s a more important consideration to ponder: what happens when AI-generated research goes from sci-slop to breakthrough science?
Dr. AGI, PhD
A significant portion of the AI community believes the emergence of AGI is just around the corner. Luminaries such as Sam Altman and Elon Musk contend that AGI could arrive by next year.
But what, exactly, would that mean? What can an AGI do that a well-trained AI can’t? According to IBM:
“Artificial general intelligence (AGI) is a hypothetical stage in the development of machine learning (ML) in which an artificial intelligence (AI) system can match or exceed the cognitive abilities of human beings across any task.”
By this definition, an AGI could do anything a human scientist could. It could write a novel research paper that meets the minimum standards for publication in any journal.
And, if this machine could be sufficiently scaled, it might be capable of outputting a steady stream of human-level research. Functionally, it’d be like an artificial PhD who can devote 100% of its time to literature reviews and the search for novel insights. One that can read and write entire books in mere seconds.
Soon thereafter, it seems, a backlog would ensue. With human reviewers already overwhelmed, there’s no chance the QA system in place could withstand the output from a competent AGI researcher if humans have to personally authorize every publication.
That leaves us with a difficult ethical decision to consider: if AGI is better at science than we are, should humans remain in the loop?
Imagine breakthrough after breakthrough piling up into a mountain of data waiting to be reviewed by human eyes as the years go on. The secret to life, the universe, and everything could wind up buried in some weary-eyed old physicist’s review backlog.
Of course, it’s just as easy to imagine how giving machines defacto control over science and technology by taking humans out of the loop could backfire on us.
Perhaps the solution is as simple and elegant as merely reframing the problem. Instead of treating the glut of AI-generated sci-slop papers flooding journals as a nuisance, some clever development team could use these rejected papers to train narrow-AI models to act as the initial gatekeeper in the acceptance process.
While there are no guarantees that such a system would be any better at parsing research than today’s models are at generating it, the reality is that the scientific community — and humanity itself — would be best served if this problem is addressed well before AGI emerges.
A veteran AI journalist, Tristan was the creator and managing editor of The Next Web’s “Neural” imprint. He is one of the first and most prolific reporters to cover artificial intelligence as a full-time beat.