An update on independent evaluators

Anthropic announced that it has started giving employee‑like access to a team from METR, a nonprofit that evaluates frontier AI models, as part of its commitment to stronger independent safety assessment. METR will publish the results of its risk assessments, and Anthropic has worked with METR on several reviews of its Risk Reports over the past year. The company is also working with SecureBio and Accenture as evaluators and is in early discussions with other organizations, including Hugging Face. Anthropic provides evaluators with deep access to internal tools and systems, similar to its own risk assessment teams, while restricting access to certain sensitive categories such as customer data, attorney‑client privileged information, or model architecture details. METR will publish its analysis of Anthropic’s risk levels, incidents, or practices, and Anthropic can require redactions for predefined categories. Anthropic’s engagement with METR is limited to a small team under enforceable agreements, and other METR employees are siloed from that team’s work. Anthropic is developing three tiers of access for evaluators: a first tier with narrowly scoped investigations and approval‑required publication; a second tier with broader access and publication without approval, subject to redaction; and a third tier, identical to the METR arrangement, with employee‑like access and publication rights. SecureBio is reviewing the chemical and biological weapons chapter of Anthropic’s August Risk Report, and after completing a pilot review of a previous report, Anthropic will consider moving to projects with increased access. With Accenture and Faculty, Anthropic is working toward a partnership that includes evaluating and red‑teaming models, conducting alignment assessments, testing model safeguards, and building capabilities across organizations. Anthropic believes that embedding evaluators voluntarily is essential for frontier AI companies to ensure robust safety oversight.