Better answers, broader thinking: What students gain from ChatGPT and critical-thinking training

What happens when students use ChatGPT on a real-world assignment? Do quality improvements come at the expense of originality?

A new experiment from researchers at Bocconi University, in collaboration with OpenAI Economic Research, found distinct and complementary effects from ChatGPT access and critical-thinking training. Access to ChatGPT improved the quality and coherence of students’ work, while an exercise in causal reasoning—a form of critical thinking—led students to generate more unique ideas. Students who received both ChatGPT access and the training showed both effects.

The results also highlight the importance of a holistic approach in assessing student progress. As AI makes it easier for students to produce polished answers, assignments may need to adapt to measure other desired qualities such as originality.

During the experiment, more than 1,000 first-year undergraduate students at Bocconi University worked on a real-world business case developing marketing recommendations for the university’s merchandise store.

The experiment’s randomized design allowed researchers to dig into these different factors, separating the effects of ChatGPT access and the critical-thinking exercise from the effect of combining them. That makes the experiment a particularly useful contribution to a rapidly growing body of research on the impact of AI on students and how to best structure and support their learning.

Students were randomly assigned by class period into one of four groups that received: access to ChatGPT (GPT‑4o), training in causal reasoning, both, or neither. Causal reasoning is a specific form of critical thinking related to linking cause and effect and explaining why a given solution may or may not work. The training students received was unrelated to AI, instead teaching students causal reasoning concepts through an exercise that involved a game, examples, questions, and feedback.

Student submissions were evaluated by trained human graders using a five-point rubric. Separately, researchers used automated text analysis to measure each submission’s number and variety of ideas, signs of causal reasoning, and similarity to submissions from three experts.

The students who had access to ChatGPT scored almost a full point higher on the five-point scale. Their answers included more ideas, followed clearer logic, and were more similar to recommendations written by experts. In other words, AI helped novices produce work that looked more professional. Importantly, students weren’t simply handing over their assignments to ChatGPT. They still had to decide what to ask, evaluate the responses, and choose what went into their final submission.