This post is part of AVERI's pilot report series. AVERI runs pilot projects with leading AI companies and converts what we learn into auditing standards, policy analysis, and open-source tools. This post summarizes our collaboration with Amazon.
At AVERI, our mission is to make frontier AI auditing effective and universal. Frontier AI auditing means third-party verification of leading AI developers' safety and security claims, and evaluation of their systems and practices against relevant standards, based on deep, secure access to non-public information.
AI developers use public and internal evaluations to test how their models perform in different circumstances. Public evaluations are open to broad scrutiny from AI researchers. Internal evaluations also need independent scrutiny to help ensure they are rigorous and effective.
Today, we share findings from our pilot with Amazon, in which we reviewed safety evaluation quality across several risk categories. This review independently confirmed statements by Amazon about its internal evaluations and contributed to further improvements in them, allowing Amazon’s next model release to be informed by higher-quality information. In sharing this analysis, we hope to contribute to greater exploration and standardization of this type of third-party metaevaluation (i.e., evaluation of a company’s internal evaluations).
Details of the Pilot
In short, we independently reviewed a random sample of 800 prompts used internally at Amazon to assess the safety of its models across several risk categories. We found this sample to be consistent with Amazon’s public statements on its internal evaluation practices, and provided detailed feedback to Amazon on areas for improvement, which Amazon acted on.
Selection of pilot focus
In our early discussions with Amazon, we considered three approaches to independent review: conducting “peer review” of non-public third-party evaluations submitted to Amazon; looking for evidence of overfitting and evaluation awareness in model behavior and chain of thought; and assessing the general quality of internal safety evaluations. Ultimately, we chose to focus on internal safety evaluation quality because it offered the greatest opportunity to inform external review practices and to strengthen Amazon’s internal assurance program.
Access and review
Amazon provided AVERI with a random sample of 800 prompts from its internal safety evaluation suite, including both single-turn and multi-turn prompts. We independently reviewed each prompt manually for factors such as diversity and methodological soundness, drawing on prior work published by AVERI staff and adapting our approach to the prompts.¹
While this safety evaluation suite is applicable to multiple internal models, for concreteness, we conducted a metaevaluation of Amazon’s work on Nova 2 in particular. We assessed whether Amazon’s public statements about internal testing in the Nova 2 documentation were consistent with our understanding of the sample. We relied on Amazon’s representation that the sample was random and drawn from the full internal evaluation suite.
Findings
We found Amazon's public statements about its internal testing methods² and coverage of safety categories³ to be consistent with the sample we reviewed. We observed significant variation in writing style and a mix of single-turn and multi-turn conversations. The prompts appeared to be original rather than drawn from publicly available datasets, which may reduce the risk of evaluation leakage.
We also identified four main areas for improvement:
On attack sophistication, some prompts represented relatively straightforward attack vectors that would test safety filters rather than deeper model reasoning.
On evaluation signal quality, we identified a concern that superficial cues, rather than safety reasoning, may drive some model refusals, so passing these evaluations might not indicate robust safety behavior in novel situations.
On linguistic and stylistic diversity, some multilingual prompts were included, but prompts across risk categories were predominantly in English, and some structures were reused across scenarios.
On evaluation freshness, a small number of prompts contained temporal references that could become outdated.
These themes are not exhaustive; we also provided feedback on individual prompts and broader observations across risk categories.
In response to our findings, Amazon reports [link] having made four changes⁴:
Refreshing its test sets to remove redundant prompts and add more challenging adversarial scenarios;
Introducing a tiered difficulty calibration framework;
Expanding multilingual coverage across all risk categories; and
Establishing a quarterly review schedule to identify staleness and incorporate emerging attack patterns.
These will enable more informed upcoming model release decisions at Amazon.
Related work
There have been some prior examples of independent metaevaluation work involving AI developers. Ahead of the release of open-weight model gpt-oss in August 2025, METR, SecureBio, and Daniel Kang of the University of Illinois Urbana-Champaign were given access to non-public details of OpenAI’s approach to fine-tuning the model for malicious purposes to estimate the risks associated with such fine-tuning. The resulting public review discussed 22 recommendations, 11 of which OpenAI acted on, including changes to its evaluation suite.⁵ ⁶
Also, in Anthropic’s February 2026 Risk Report, SecureBio examined non-public materials on Anthropic’s chemical and biological evaluations and scrutinized study design, scoring, and thresholds, while METR reviewed non-public sabotage evaluations and raised concerns about evaluation sensitivity and evaluation awareness. ⁷ ⁸
These examples show that independent scrutiny of internal evaluations is feasible – but it remains relatively uncommon, the methodology for doing it has not been discussed in detail, and there are no clear standards for conducting such analysis (a footnote discusses our project’s relationship to AEF-1 in more detail).⁹
Lessons for frontier AI auditing
We believe third-party assessment of evaluation quality (metaevaluation) is a valuable complement to red-teaming and benchmarking and should be a normal part of how AI developers build justified confidence in their safety claims.
We suggest four priorities for future work.
First, an external reviewer should ideally be able to confirm directly that the material sampled is the material actually used for internal evaluation purposes, rather than relying on assertions about the material being representative (e.g., that it was randomly sampled or that it is the complete internal evaluation suite). “High assurance” verification tasks of this nature are time-intensive and require deeper access arrangements. ¹⁰
In this pilot, we relied on Amazon’s account of the sample. More adversarial circumstances, such as an international agreement on AI, would require stronger verification to guard against deception. Reviewers could gain greater confidence by observing the sampling process, reviewing evidence of how it was conducted, or accessing the full evaluation suite. Technical approaches such as secure enclaves, company-provided laptops, or language models as intermediaries in the analysis, as well as physical approaches such as on-site clean rooms, could facilitate such access.
Second, manual analysis limited the volume of material we could review. “Automated auditing” – using one AI system to help scrutinize another AI system or a company’s practices – could help reviewers assess larger datasets. For example, AI could scan evaluation suites for duplicate prompts, stylistic clustering, or temporal references, while human reviewers focus on judgments that most require expertise. The volume in this pilot was manageable, but pretraining datasets are too large to review manually. Even a small sample may contain enough diversity to challenge reviewers’ understanding of the dataset without assistance.
Key research questions related to automated auditing include: what audit capabilities and biases different AI systems (including open-weight ones) have, such as tendencies to confirm or reject particular claims; how to best use one company’s AI systems securely to audit another; and how to use a company’s own model effectively to audit its practices while managing biases.
Third, reviews of this kind will often span multiple risk domains. Reviewing a range of risk categories allowed us to provide cross-cutting feedback to Amazon. Higher confidence assessments of individual categories, such as biological risks, would require greater subject matter expertise and more detailed evaluations. Ready access to experts fluent in additional languages and bound by confidentiality agreements would also have been valuable. Machine translation could serve some purposes, but we erred conservatively here by avoiding AI use to ensure we effectively protected confidential information and eliminated the risk of contaminating the evaluation of future systems.
Fourth, reviews of internal evaluation practices would be most effective if they combined review of technical systems and artifacts (e.g., evaluation prompts) with review of processes (e.g., how employees use and interpret evaluation suites). The significance of the limitations we observed in Amazon’s prompts is difficult to assess in isolation. For example, reviewers need to understand how much weight staff place on benchmark scores when making decisions regarding deployment of an evaluated model (or development of the next generation model), and how they account for known evaluation flaws. Ideally, frontier AI auditing would allow third parties to assess this whole picture, and its alignment with the letter and spirit of a company’s commitments.
Conclusion
Internal evaluations affect external stakeholders even when their details are non-public. We hope this case study encourages further independent review. We appreciate Amazon’s engagement and support, including publishing a blog post [link] that covered the strengths and weaknesses our review identified.
We will share more pilot results in the coming months, with varying scope and access conditions. We will also continue to distill lessons from these pilots – and other independent assessment organizations – into frontier AI auditing standards through our publications and work with institutions such as the AI Evaluator Forum.
Conflict of interest and publication review disclosure: AVERI has received API credit donations from several frontier AI companies, including Amazon, as disclosed on our About page along with additional relevant information here. No compensation in cash or API credits was provided for this collaboration. Under the agreement between AVERI and Amazon, Amazon had the right to review and approve AVERI publications about this collaboration.
Notes:
On prior work, see, e.g.,”AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons,” https://arxiv.org/abs/2503.05731 and “Risk Management for Mitigating Benchmark Failure Modes: BenchRisk,” https://arxiv.org/abs/2510.21460.
"Amazon Nova 2: Multimodal Reasoning and Generation Models," Section 5.1.1 (Responsible AI Evaluations) https://cdn.amazon.science/c5/3d/84514a224666b5be6de4b43ef4aa/nova-2-0-technical-report2.pdf
“Amazon’s Frontier Model Safety Framework”, Section 1 (Critical Capability Thresholds) https://cdn.amazon.science/a7/7c/8bdade5c4eda9168f3dee6434fff/pc-amazon-frontier-model-safety-framework-2-7-final-2-9.pdf
AVERI has not reviewed these improvements, but we have no reason to suspect they were not made. The collaboration timeline is consistent with their implementation.
https://securebio.org/resources/anthropic_feb_2026_risk_review.pdf
https://metr.org/blog/2026-03-12-sabotage-risk-report-opus-4-6-review
There is growing standardization in external evaluation more broadly, including publicly articulated best practices in areas like conflict of interest disclosure and access. See, e.g., AEF-1: https://aievaluatorforum.org/initiatives/minimum-operating-conditions. While some of these best practices are applicable to the pilot done here, some elements of AEF-1 are not obviously applicable to metaevaluation, since most discussions of evaluation best practices have focused on direct behavioral evaluation of AI models and systems by a third party. Regarding our compliance with AEF-1, for brevity we focus on requirements rather than recommendations and note that we unambiguously met the following requirements: 1.4, 2.1, 2.3, 2.4, 2.5, 3.2, 5.3, 5.4. The following requirements were less obviously applicable: 1.1 and 1.3 (these seem to assume a model or system level evaluation rather than metaevaluation. Our compliance with 4.1 was borderline because we provided enough information to “review” our results at a high level, though not enough to replicate the exact conclusions, and the level of detail required for compliance is also unclear. As noted in our publication review disclosure, Amazon did review our blog post, though we did not come close to testing the limits of 3.4 and 4.6. 5.1 was not obviously relevant given our exclusive focus on non-public information. We hope that this pilot report contributes to more discussion of best practices for – and implementation of – meta-evaluation.
On the meaning of high assurance in the context of frontier AI auditing, see “Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies,” https://www.averi.org/ourwork/frontier-ai-auditing.
For examples of related work technical approaches for reconciling deep access with security and privacy, see our earlier pilot with Google DeepMind, OpenMined, and MLCommons, https://www.averi.org/ourwork/averi-pilot-report-the-worlds-first-double-blind-eval, “Clio: Privacy-Preserving Insights into Real-World AI Use,” https://arxiv.org/abs/2412.13678, and Trusted Machine Learning Models Unlock Private Inference for Problems Currently Infeasible with Cryptography,” https://arxiv.org/abs/2501.08970.
