How UK AISI and EvalEval Are Making Benchmark Results Reproducible
The UK AI Security Institute is adopting EvalEval infrastructure to openly share evaluation results and support reproducible AI evaluation science. The collaboration builds on previous joint research and utilizes EvalEval's Every Eval Ever schema and Evaluation Cards platform. AISI is making verified findings and configuration details for five benchmarks and related cyber evaluations publicly available.
The Core Development
The EvalEval Coalition is thrilled to share that the UK AI Security Institute (AISI) is using EvalEval's infrastructure to openly share evaluation results, supporting more reproducible and verifiable evaluation science.
AISI and EvalEval have previously collaborated on research that began at a joint workshop alongside NeurIPS 2025 , and feedback from the Institute has helped shape the Every Eval Ever (EEE) schema . This next phase of the collaboration puts that shared infrastructure into practice.
As AI deployment accelerates, evaluations are becoming increasingly important sources of evidence about model and system performance. Yet results are reported across many formats, platforms, and outlets, often without enough information to reproduce them. Running the evaluations again may itself be prohibitively expensive.
Deployment & Availability
When results are openly released with setup information, researchers and practitioners can examine individual studies more closely and compare findings across the wider ecosystem. Where other reports lack these details, releases like AISI's provide verified reference points for interpreting evaluations in context—for example, by helping researchers understand how setup choices may influence reported performance. As more evaluators adopt EEE, open comparisons like these can support broader and more reliable meta-research.
Strategic Perspective & Ecosystem Impact
EvalEval's mission is to improve this ecosystem through a shared reporting schema, Every Eval Ever , and an open platform, Evaluation Cards , that brings evaluation results and the information needed to interpret them into a common structure.
This builds naturally on AISI's work to make evaluation more efficient through OptStop , more statistically rigorous through HiBayES , and more standardised in areas including transcript analysis and capability elicitation. Together, AISI and EvalEval are working to diagnose gaps in evaluation reporting and build shared infrastructure to close them.
Transcript-level transparency matters not only for reproducibility, but also for analysis and diagnosis. In this new phase of the collaboration, AISI is making publicly reported evaluation methods and findings available through Evaluation Cards where appropriate. The release includes verified results, context, and configuration information for the five benchmarks in the paper's main experiment:
Why it matters
Standardized reporting schemas and open platforms help address reproducibility challenges across the AI evaluation ecosystem, allowing researchers to compare findings and understand setup impacts.