Opening article
Anthropic Research has published a paper examining how neural networks pack many concepts into single neurons. The research explores how and when models represent more features than they have dimensions, a phenomenon known as superposition.
In this paper, we use toy models — small ReLU networks trained on synthetic data with sparse input features — to investigate how and when models represent more features than they have dimensions. We call this phenomenon superposition. When features are sparse, superposition allows compression beyond what a linear model would do, at the cost of "interference" that requires nonlinear filtering.
We gave Claude plain-language specifications for eight quantum circuits, along with an automated evaluator to check and score its designs. Working largely on its own, Claude improved on the best published circuits, in some cases by an order of magnitude or more, which could bring practical quantum computers closer than previously anticipated.
The decades-old mystery of the origins of one of the most popular cryptographic standards has been explained.
How will AI reshape the economy in the years ahead? What is the core source of disagreement about AI's future impact on the economy? This note presents new survey evidence on these questions.
Understanding how neural networks represent more features than they have dimensions provides deeper insight into model interpretability and internal mechanics for the AI industry.