Imagine a world where artificial intelligence could revolutionize the way research grants are awarded, potentially speeding up the process and freeing up valuable time for experts. But here's where it gets controversial: the UK’s primary funding body, UK Research and Innovation (UKRI), is now testing whether AI can handle the delicate task of peer reviewing grant proposals. With over £8 billion in research funding allocated annually, UKRI faces a daunting challenge: while the number of funded grants has halved in the past seven years, the volume of applications has skyrocketed by more than 80%. This mismatch has prompted UKRI to explore innovative solutions, and generative AI is at the forefront of their experiment.
Starting in October, a research team led by Mike Thelwall, a data scientist at the University of Sheffield, embarked on a mission to determine if AI can predict the scores and recommendations of human peer reviewers. Funded by the UK Metascience Unit—the first governmental body dedicated to improving research processes—Thelwall’s team gained access to 1,000–2,000 full-text grant proposals, both funded and rejected. These documents, typically kept confidential, will be analyzed by large language models (LLMs) to assess their predictive accuracy.
And this is the part most people miss: while the AI won’t know the actual scores or outcomes of the proposals, its ability to predict them could significantly streamline the review process. ‘If AI can reasonably predict the scores, it might help expedite reviews or support human reviewers,’ explains Thelwall. This isn’t his first foray into AI-assisted evaluation; he previously explored its use in refereeing articles for the UK’s Research Excellence Framework. In 2022, his team found that AI matched human scores 72% of the time—impressive, but not yet the 95% accuracy Thelwall deems necessary for widespread adoption.
However, not everyone is convinced. Mohammad Hosseini, an AI ethics researcher at Northwestern University, raises ‘serious doubts’ about LLMs’ ability to generate or recognize truly novel ideas. ‘AI is trained on existing data,’ he notes, ‘so it may struggle to identify groundbreaking proposals that haven’t yet been realized.’ Another concern? Transparency. If funders don’t disclose the criteria fed into the AI, researchers may revolt. Conversely, if the process is too transparent, applicants might game the system by tailoring their proposals to appease the AI.
So, where does this leave UKRI? Thelwall suggests AI could serve as a tiebreaker or an additional reviewer, or even help desk-reject low-potential proposals to reduce the burden on human experts. He points to the la Caixa Foundation in Barcelona, which uses AI to prescreen grants, saving experts from evaluating proposals with minimal funding chances. While 90% of applications still undergo full peer review, the time saved is significant.
But here’s the burning question: Can AI truly handle the nuanced, creative aspects of grant evaluation, or will it remain a tool for efficiency rather than innovation? What do you think? Is AI the future of grant review, or does it risk overlooking the next big idea? Let us know in the comments—this debate is just getting started.
