Long-Form Tasks

Note to readers: we changed our name to the AI Security Institute on 14 February 2025. Read more here.

Large Language Models (LLMs) are showing promise in becoming helpful assistants to scientists. Whether up-skilling in some domain or planning and executing an experiment, an LLM’s ability to package and tailor complex scientific knowledge to be maximally helpful to a user is likely to be extremely valuable. The AI Safety Institute (AISI) aims to measure the extent to which current LLMs are helpful scientific assistants.

Most current evaluations comprise sets of short questions with a single correct answer. This is an effective way to measure a system’s existing knowledge, but domain knowledge alone is insufficient to measure scientific helpfulness. The gold standard for measuring helpfulness is a Human Uplift Study (HUS), which assess how much AI improves a user’s task performance, but these in-depth studies are time-consuming and expensive to run. To address the need for helpfulness evaluations that can be used at scale, AISI has designed an evaluation methodology called a Long-Form Task (LFT) which aims to assess the model’s ability to provide helpful instructions to a user in a scientific setting. These evaluations measure domain knowledge as well as the LLM’s ability to plan and reason. Using expert written grading guidance, solutions can be scored automatically allowing these evaluations to be deployed quickly and at a minimal cost.

This report outlines our evolving LFT methodology, by sharing this we hope to solicit feedback and new ideas from the research community.

Over the last decade, AI has been applied in many ways to accelerate scientific progress. For example, specialised tools have helped scientists improve weather forecasting i , design alien-looking parts for spacecraft ii , and predict the structures of millions of unknown proteins iii .

LLMs have enabled a new modality of collaboration between scientists and their tools: natural language. By simply “chatting” with LLMs via text or voice prompts, scientists can unlock the wealth of information absorbed by these models during pretraining. In theory, LLMs will have “read” a mountain of relevant literature and should be able to package this information to fit the knowledge gaps of their human collaborators. We see this clearly demonstrated in knowledge-based benchmarks such as GPQA iv . This promise has led to strong incentives for developers to make LLM-powered science assistants which are useful to researchers across industry and academia. Examples of this include OpenAI’s recent partnership with Los Alamos National Laboratory v and the development of a personalised science tutor by Khan Academy vi .