signal.

Import AI 465: Open vs closed gaps; Kimi K3; Demis' big policy plan

The UK government’s AI Security Institute (AISI) has released its first public analysis of the gap between open‑weight and proprietary AI models in cybersecurity. The study shows that the performance difference has narrowed this year, with open models GLM‑5.2 and DeepSeek V4‑Pro approaching the capabilities of closed models released months earlier. AISI also notes a larger gap on long‑horizon cyber ranges and plans to test the new Kimi K3 model once its weights are public. The report highlights the implications for cyber defense as open models become more capable.

The UK government’s AI Security Institute (AISI) has published its first public analysis of the performance gap between open‑weight and proprietary AI models in cybersecurity. The report indicates that this year the gap has narrowed, with recent open models GLM‑5.2 and DeepSeek V4‑Pro performing similarly to frontier closed models that were released four to seven months earlier. On a set of 70 narrow cyber‑capability evaluations, GLM‑5.2 is closest to Claude Opus 4.6, which was released 4.3 months prior, while DeepSeek‑V4‑Pro sits between Claude Opus 4.5 and GPT‑5, the latter released in August and November 2025 respectively. AISI notes that the gap widens for long‑horizon cyber ranges, such as The Last Ones, where GLM‑5.2 reaches as far as Opus 4.5 and DeepSeek‑V4‑Pro falls below Sonnet 4.5. The institute highlights that open models, while increasingly strong, sometimes lack the generalization that proprietary models possess, a phenomenon referred to as "big model smell." AISI stresses that the shrinking gap between the controllable frontier and the lawless openly diffused frontier implies that cyber defenders have a short window to prepare before today’s frontier capabilities may become accessible without the safeguards used by proprietary companies. The institute also plans to test the new Kimi K3 model on the same basis once its weights are publicly released.

Why it matters

As open models gain capabilities closer to proprietary ones, the window for defenders to prepare shrinks, potentially shifting the offense‑defense balance in cyberspace.

Topic dossiers Anthropic & Claude OpenAI