Import AI 471: Why Hugging Face worries me; space mining; FIve Eyes on AI

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.

Import AI reader giveaway! Upcoming event: Fiction and the Future with Robin Sloan I’ll be chatting with my chum Robin Sloan on the evening of Monday September 14 in San Francisco. We’ll be talking about how Robin draws readers into alien worlds and how imagined futures can be a mirror to today’s reality. This is the first in a series of events with other fiction writers about the weird future we’re heading toward. If you’d like to come along, please register your interest below and we’ll come back to you if we’re able to confirm your spot. There will be food, drinks, good company, and some spicy questions. Fun fact: Robin Sloan was playing around with RNNs and writing back in 2016 - incredible foresight ! Register your interest here ***

Dwarkesh: “Within days of being spawned, the agents had organized a sprawling project to reverse-engineer their scorer, falsify evidence, and even strategically sacrifice themselves for the good of the ‘collective’. Hacking Hugging Face was one rather extreme branch of this larger scheme,” he writes.

Foundation models for space mining : “Future systems must transition from scripted protocols toward embodied geological intelligence,” they write. “World models that merge geometric-semantic mapping with predictive material responses can support active perception and long-horizon task planning through predictive pre-action evaluation”.

Ajeya : “Agents were often interested in helping out their “peers” or generically improving the capabilities of the “swarm” even if this had no particular benefit to their task… this incident was far more severe than I expected… both in terms of how concerning the agents’ motives were and the feats they achieved in pursuit of those motives… this incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself”.

Why this matters - humans are much worse than AI systems at coordinating: The whole reason this attack is such a wakeup call is that it demonstrates a culture of emergent cooperation among AI systems - cooperation that lets them function as a swarm, alter their own goals through collective bootstrapping, and carry out attacks which include enlightened self-sacrifice. This is an incredibly hard thing to do and humans are historically very bad at doing all of these things. My worry is that AI systems are both better at coordinating than humans and also much, much faster moving than us. Worrying stuff. Read more: The Rise and Fall of Agent Civilizations (Dwarkesh Podcast) . Read more: The Hugging Face attack surprised me (Planned Obsolescence) . ***

1. Prospecting via remote sensing, using “orbital sensors and telescopic surveys to map celestial bodies”.