Opening article
The AISI Blog has published an outline regarding its approach to technical solutions for AI misuse and loss of control. The post focuses on addressing the growing gap between advanced AI capabilities and current safety mitigations. The source details specific technical strategies aimed at these safety concerns.
As AI capabilities advance, they present security risks which the AI Security Institute (AISI) is working to mitigate and control. This extends beyond our risk assessments , to conducting and funding technical research to actively improve safety and security measures.
However, the gap between capabilities and mitigations is growing fast. This gap exists across different risks, including adversarial actors misusing AI systems to facilitate large-scale harms, the possibility of developers losing control of highly intelligent AI systems, and a range of other potentially destabilising effects AI could have on society. This blog outlines our approach to technical solutions for misuse and loss of control .
We're hiring research scientists and engineers across our Safeguards Analysis team (to counter misuse) as well as our Control and Alignment teams (to counter loss of control).
To rapidly advance the science of AI safety and security, AISI aims to maximise the quality research tackling this problem. This is why we recently established the Challenge Fund , which will provide grants of up to £200,000 to academics and non-profits.
The risk of attackers misusing AI to cause large-scale harm or disrupting their operation to damage critical systems increases as AI systems advance. Research that seeks to understand, evaluate, and improve the technical measures designed to address these risks (safeguards) is urgently needed.
We cannot currently guarantee that highly capable AI systems will act in alignment with human intentions, and misuse safeguards are not designed to resist intentional subversion by misaligned AIs. The field of AI control attempts to design and evaluate control protocols : safety measures that are robust to intentional subversion by AI systems. Promising protocols vary widely, from applications of cyber security to AI monitors and human auditing. Protocols are evaluated empirically by a red team using proxy AIs and environments to simulate what would happen if a misaligned model were deployed.
This matters because bridging the gap between AI capabilities and safety mitigations is crucial for preventing misuse and maintaining control as technology advances. It highlights ongoing efforts by institutions to establish technical safeguards in the industry.