Import AI 476: Creativity and RSI; worse-than-China open weight models; AI goes to space

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.

New benchmark shows how AI models lack intuitive visual reasoning: …Humanity’s Sixth Sense highlights some shortcomings of current AI systems… AI startups Elorian and Scale AI have built and released Humanity’s Sixth Sense, a benchmark that “measures how a model interprets and reasons through information a visual scene implies but doesn’t explicitly show.” Some examples of the types of task in HSS are a picture of a bookcase and being asked whether it can fit more books, or a photo of horses pulling a plough and a question of whether the chains bear the plough’s drag weight.

HSS contains 522 questions that cover 288 images and 234 short video clips (totalling 17.6 hours of video). The questions fall into four categories:

Reflection makes a token-efficient worse-than-Chinese model: …Beam promises efficiency, and Reflection plans an open weight release… AI startup Reflection has revealed Beam, a 501 billion parameter MoE (23b active) LLM. Beam is being promoted on the basis of its token efficiency - being able to think intelligently while using way fewer tokens than other models. However, benchmark scores show it is behind the frontier capabilities defined by LLMs available from Chinese companies such as Zhipu (GLM) and Moonshot (Kimi) and DeepSeek. “We will release the weights, technical report, model card, and developer artifacts later this month,” the company wrote at the beginning of October.

Mistral makes a cyber-optimized, worse-than-Chinese model, plans open weight release: …Cyber refusals become a selling point… Europe’s leading AI developer, Mistral, has announced Mistral Large 4, an open-weight model which the company plans to release by the end of the month. One of the main selling points for Mistral is that it has been carefully tuned to be good at cyber tasks while having fewer annoying refusals than proprietary models from Anthropic and OpenAI. “ML4 pairs top-tier cyber performance with open weights and self-deployment, giving organizations both the capability and the autonomy to run advanced security work under their own policies,” the company writes. ML4 was trained on 3,800 NVIDIA Grace Blackwell GPUs in Mistral’s European datacenters.

Space might support an industrial flywheel where initial infrastructure begets more energy and manufacturing capacity at lower costs which begets more infrastructure, etc.

Temporal & causal dynamics: “What happened before the scene, and what happens next.”

Physical & spatial logic: “What fits, what’s in reach, and what someone else can see from where they are standing.”

Results: The human baseline for this is not perfect, but is very high, with people scoring aggregate pass@1 of 93.1, broken down across four categories: Image (94.1), video (91.9), time (97), and space (94.9). By comparison, the best scoring AI system, GPT 6 Astra, gets pass@1 of 53.6, across image (55.6), video (51.1), time (58.2), and space (53.6). Interestingly, Chinese open weight models do poorly here, with GLM-5.3-flash getting a pass@1 of 26.8, and Kimi K3 getting pass@1 of 25.5. “The two most common single mistakes were (1) missing a detail that decides the answer (21% of failures) and (2) misidentifying an object, person, or role (20%),” they write.