Grant proposals rejected by ‘AI triage’ to be reassessed

A research funding network that used artificial intelligence to reject half of grant proposals to its latest funding call has committed to redoing the review process.

UK Research and Innovation (UKRI) has faced criticism from academics after it was revealed that a network it supports, called CRANE, used an AI “triage” process to determine which of the 179 proposals it received would progress to human review. Almost half of the applications were rejected by the AI, and were not looked at by humans.

The network, which aims to strengthen the UK’s cybersecurity research, has now written to researchers to apologise, promising to review all the proposals again.

“Although we undertook the AI triage process with care, with human sampling and statistical cross-checking, and in a way that ensured confidentiality, we recognise that we were not transparent enough about this approach, and for that we are sorry,” says the update, seen by Times Higher Education.

“We have listened carefully to the feedback and understand the questions this has raised.”

The funding call for cybersecurity-related projects involved was the CRANE phase one pilot, hosted at the University of Oxford and funded by UKRI subsidiary, the Engineering and Physical Sciences Research Council.

CRANE says: “We have discussed the best approach with EPSRC (and the CRANE Advisory Board) and agreed that given the complexities and sensitivities around the use of AI tools in peer review, we will re-review all applications that were initially screened out using AI and reconvene the funding panel to make decisions. Offers previously issued are suspended.”

The network added that “applicants have been contacted directly to explain the re-review process and next steps”. Both CRANE and UKRI were approached for comment.

After the initial outcry from academics, UKRI said that the release of confidential material into generative AI tools constitutes a breach of confidentiality and integrity as set out in its policy on the governance of good research practice and principles of assessment and decision-making policy.

UKRI has recently released its own proposals for contending with the rapid growth in grant applications that is being driven partially by AI. These include using in-person interviews for “higher-risk” awards “where additional assurances are required” and asking UKRI staff to conduct more triage exercises.

In addition, it proposes “exploring the safe use of AI-assisted assessment” but the funder insisted that this would have “appropriate controls and safeguards”, including a trial designed to compare what AI would have concluded from past applications against decisions that were ultimately made.

In its update CRANE says: “We want to be clear that the confidentiality of proposals was safeguarded throughout, in keeping with the terms outlined in the call for proposals. The AI triage used closed, ring-fenced read-only LLMs to access the relevant proposal data. The models had no internet access and could access only the specific elements of proposals required for the evaluation. Proposal data has not been, and will not be, used for model training.”

The network stressed that: “no public AI services, such as ChatGPT, were used, and proposal text remained under the confidential control of the review team throughout”.

It promised that the “trust and confidence of CRANE members in our work matters greatly to us and is fundamental to our objective of strengthening the UK’s cyber security research landscape”.

“We value the feedback we have received and will take what we have learned from this experience into future funding activity.”

frances.jones@timeshighereducation.com