Mapping the limitations of current AI systems

Several leading AI companies have the stated goal of building AI systems that match or surpass humans across most cognitive domains. Such systems could have transformative effects, such as increasing productivity and accelerating scientific research in critical areas including energy and medicine. However, experts warn that they could also pose national security threats if not reliably aligned with human intent, as well as having the potential to cause disruptive labour market effects.

Among AISI’s priorities is to work closely with a range of experts and developers to understand AI trajectories. In a new report, we track progress towards hypothetical, future AI systems that would be capable of automating most cognitive labour. Today’s Large Language Models (LLMs) are impressive but fall short of this standard in several important respects. What barriers remain?

Our new report poses this question to experts both internal and external to AISI. Based on these interviews, a literature review, and a workshop with the AISI Research Unit, the report identifies eight limitations of current AI systems, alongside evidence that would indicate progress towards overcoming them.

Real-world work demands a high-level of adaptability, or ‘learning on the job’. This requires digesting a lot of context about the nature of the role, its overarching objectives, and so on. Improvements here could come from expanding, or making better use of, models’ context windows (how much information they can process together at any one time), or from methods for rapid, efficient adaptation of model weights to specific use cases.

There are existing benchmarks that test the contextual awareness of LLMs, such as LoCoMo and LongMemEval . Advances in techniques that allow models to adapt their own weights in response to new knowledge or examples (such as Self-Adapting Language Models ) could serve as a leading indicator for progress on continual learning, especially insofar as these techniques move beyond lab demos and into real-world applications.

AI systems already perform at the level of human experts in certain verifiable domains in which performance can be easily assessed, such as mathematics and coding, but still struggle with other tasks that would be required for automating most cognitive labour. When it’s straightforward to check whether a model submitted the correct solution to a problem, researchers can more easily generate a robust reward signal that can be used to improve performance further. But widespread automation of labour requires strong performance in a wider range of domains than the easily verifiable. For example, the consequences of many real-world strategic decisions can take a long time to manifest and are difficult to attribute to particular actions and interventions. Decisions can also require aesthetic or intuitive judgements that are harder to rate objectively.

The widespread automation of cognitive labour would require AI systems that can act reliably over long time-horizons. Many real-world tasks involve staying on track for hours, days, or even weeks at a time, self-correcting along the way. Humans currently maintain an advantage over AI systems in completing tasks that take more than a few hours – but this could change soon. Research by Model Evaluation and Threat Research (METR) shows that the length of software engineering tasks AI systems can complete is doubling approximately every seven months. Extrapolating this trend predicts models that will be able to complete month-long tasks (with 50% reliability) by 2030.

People performing real-world labour must do so in messy and complex environments, where they must communicate with others, confront unexpected obstacles and prioritise tasks. In-the-wild evaluations suggest that AI systems currently struggle in these more realistic environments. Examples include Anthropic’s Project Vend , which placed their flagship Claude model in charge of running a vending machine, and AI Village , in which teams of agents are given challenges such as raising money for a charity. These experiments have tended to find surprising performance limitations in these more realistic environments. For example, the Claude-operated vending machine did not turn a profit.

While the timing and likelihood of AI systems capable of automating most cognitive labour are all subjects of live debate, their consequences could be extremely significant, meaning that tracking progress towards them is critical. We use the common industry term “Artificial General Intelligence” (AGI) as a shorthand to describe such systems in our interviews and in the analysis below.

Despite the report’s focus on this particular milestone, we recognise that many impacts of highly capable AI systems may emerge gradually rather than arriving all at once. It should also be noted that AI systems with the technical capabilities required to automate most cognitive labour may be developed some considerable time before widespread automation actually occurs – there is often a lag between the development and the adoption of new technologies.

Some experts believe that continued scaling of Large Language Models (LLMs) will be sufficient to produce AGI, while others believe that fundamental paradigm shifts will be required. For our report, we largely interviewed experts who take the former view, to analyse barriers to AGI development in the relatively near future. Our report gives precedence to this scenario and so should not be read as representing the full spectrum of expert opinion.