Paper Sets Out 44 Meta-Requirements For AI Robustness Audits

A team of researchers, led by Joshua Owotogbe, posted a paper on arXiv on October 9, 2026, addressing a gap that robustness auditing of AI systems has lacked: no consolidated list of what such an audit should actually check for. The authors frame the problem in design science terms. An auditing framework needs to decide whether available evidence supports a robustness conclusion, but nobody had agreed on what that framework itself must cover.

To build that list the team ran a systematic literature review, screening thousands of records down to a working set they judged relevant, then drew an initial batch of meta-requirements from that smaller body of work. Rather than stopping at a desk review, they took the draft list to 128 practitioners and asked for both quantitative ratings and free-text comments on each item. That step matters because a requirements list built only from papers can miss what auditors run into on the ground, and the practitioner round surfaced exactly one such gap: a requirement about external components and supply chain dependencies that the literature pass had not captured.

The numbers

The review screened 2,890 records and retained 74 of them as the literature base, from which 43 meta-requirements emerged across 16 categories. The practitioner survey produced 5,504 individual ratings and 586 substantive written comments, with 81.20% of those ratings showing agreement or strong agreement with the proposed requirements. Of the original 43, 41 came back strongly supported. Adding the one practitioner-identified gap brought the final set to 44 meta-requirements, of which 43 carry quantitative validation from the survey round.

For anyone building or buying an audit of an AI system’s robustness, a consolidated checklist like this changes the comparison available. Instead of taking a vendor’s internal audit process on trust, a buyer or a regulator can hold that process against a named set of categories and ask which ones it actually addresses and which it skips. It also gives framework designers a concrete yardstick rather than a general sense of thoroughness: a framework missing coverage of supply chain dependencies, for instance, is now missing something named and counted rather than something vaguely implied.

The paper stops short of naming any specific commercial auditing product or vendor, and its authors present the meta-requirements as a foundation for organizations assessing or building their own frameworks rather than as a finished standard. Whether any regulator or industry body adopts this particular list as a reference point is not yet established, and the paper itself does not claim that outcome. What it does supply is a tested, categorized starting point that previously did not exist in consolidated form.