Global Approaches to Auditing Artificial Intelligence: A Literature Review

This Synthesis Report is a literature review outlining the regulatory, industry, and academic approaches to AI audits. We review 78 articles published in peer-reviewed journals and as preprints, 21 documents from industry associations and standard-setting organizations, and national policy documents and regulations from 20 countries.

Based on this review, we identify three key takeaways about the landscape of AI auditing:

1. To accurately assess the potential risks, and impacts of AI systems, we need a trustworthy audit ecosystem with complementary approaches from internal, external, and community auditors.

2. Auditors need better access to data and audit artifacts from the developers and deployers of AI systems. Comprehensive auditability requires documentation and disclosure of an AI system’s model and data components, associated risks and impacts, and easily understandable explanations of its outcomes.

3. Given that the development and use of AI systems impacts communities across the world, audit regimes must account for their global effects. Most existing audits have been conducted in North America, Europe, and other regions of the ‘global north’, with their results typically published in English and focused on effects within these regions. The impacts of AI systems, however, include shifts in social and environmental conditions beyond the immediate development or application contexts of a system.

‍