Gold Standard Articulation

Frontier model companies face a collective action problem where no individual actor can afford to slow their rate of progress to engineer safer systems for fear that their competitors will out innovate them on system capabilities and render their safer systems irrelevant. 

Preventing a race to the bottom requires identifying aspirational goals society (and companies) may seek to achieve. Therefore, we outline a vision for frontier AI auditing: rigorous assessment of AI developers’ safety and security practices by independent third parties. The “gold standard” for frontier AI auditing that we describe – if applied across organizations building and deploying frontier AI systems – would improve safety and security outcomes, increase accountability for risk creation, enable more informed investment and deployment, and support international stability. We believe that striving for this gold standard is critical in order to ensure the AI industry’s growing risk profile is matched by a rising degree of justified trust in the sector’s risk management. 

Our proposed gold standard draws on lessons from current AI evaluation practices as well as several other domains in which there are more mature practices for third-party risk assessment. In each case we take inspiration from both what works and what doesn’t.