The Strange Physics That Gave Birth to AI.pdf

Physics and AI have a mutually beneficial, synergistic relationship: physics provides fundamental knowledge and data to improve AI, while AI offers powerful tools to analyze complex physics data, accelerate simulations, and enable new discoveries.

1. The Strange Physics That Gave Birth to AI
COMPLEX SYSTEMS
ByELISE CUTTSApril 30, 2025 Modern thinking machines owe their existence
to insights from the physics of complex
materials.
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 1/26

2. Irene Pérez for Quanta Magazine
S
pin glasses might turn out to be the most
useful useless things ever discovered.
These materials — which are typically made of
metal, not glass — exhibit puzzling behaviors
that captivated a small community of physicists
in the mid-20th century. Spin glasses
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 2/26

3. themselves turned out to have no imaginable
material application, but the theories devised to
explain their strangeness would ultimately
spark today’s revolution in artificial
intelligence.
In 1982, a condensed matter physicist named
John Hopfield borrowed the physics of spin
glasses to construct simple networks that could
learn and recall memories. In doing so, he
reinvigorated the study of neural networks —
tangled nets of digital neurons that had been
largely abandoned by artificial intelligence
researchers — and brought physics into a new
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 3/26

4. domain: the study of minds, both biological and
mechanical.
Hopfield reimagined memory as a classic
problem from statistical mechanics, the physics
of collectives: Given some ensemble of parts,
how will the whole evolve? For any simple
physical system, including a spin glass, the
answer comes from thermodynamics: “toward
lower energy.” Hopfield found a way to exploit
that simple property of collectives to store and
recall data using networks of digital neurons. In
essence, he found a way to place memories at
the bottoms of energetic slopes. To recall a
memory, a Hopfield network, as such neural
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 4/26

5. nets came to be known, doesn’t have to look
anything up. It simply has to roll downhill.
The Hopfield network was a “conceptual
breakthrough,” said Marc Mézard, a theoretical
physicist at Bocconi University in Milan. By
borrowing from the physics of spin glasses,
later researchers working on AI could “use all
these tools that have been developed for the
physics of these old systems.”
In 2024, Hopfield and his fellow AI pioneer
Geoffrey Hinton received the Nobel Prize in
Physics for their work on the statistical physics
of neural networks. The prize came as a
surprise to many; there was grumbling that it
appeared to be a win for research in AI, not
physics. But the physics of spin glasses didn’t
stop being physics when it helped model
memory and build thinking machines. And
today, some researchers believe that the same
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 5/26

6. physics Hopfield used to make machines that
could remember could be used to help them
imagine, and to design neural networks that we
can actually understand.
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 6/26

7. Emergent Memory
The American physicist John Hopfield, pictured
in 1988, developed a model of a neural network
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 7/26

8. Hopfield started his career in the 1960s working
out the physics of semiconductors. But by the
end of the decade, “I had run out of problems in
condensed matter physics to which my
particular talents seemed useful,” he wrote in a
2018 essay. So he went looking for something
new. After a foray into biochemistry that
produced a theory of how organisms “proofread
” biochemical reactions, Hopfield settled on
neuroscience.
“I was looking for a PROBLEM, not a problem,”
he recalled in his essay, emphasizing the need
to identify something truly important. “How
mind emerges from brain is to me the deepest
that laid the foundation for modern AI.
Caltech Archives and Special Collections
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 8/26

9. question posed by our humanity. Definitely a
PROBLEM.”
Associative memory, Hopfield realized, was a
part of that problem that his tool kit from
condensed matter physics could solve.
In a normal computer, data is stored statically
and accessed with an address. The address
doesn’t have anything to do with the
information that’s stored. It’s just an access
code. So if you get the address even a little bit
wrong, you’ll access the wrong data.
That’s not how humans seem to remember
things. We often remember by association.
Some cue or scrap of memory brings the full
thing flooding back. It’s what happens when
you smell lilacs and recall a childhood episode
in your grandpa’s garden, or when you hear the
first few lines of a song and find yourself
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 9/26

10. belting out every word to a ballad you didn’t
know you knew.
Hopfield spent years on understanding
associative memory and translating it to a
neural network. He tinkered with randomly
wired neural networks and other potential
models of memory. It wasn’t looking good until,
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 10/26

11. eventually, Hopfield identified an unlikely key
to the “PROBLEM.’’
Geoffrey Hinton (left) and John Hopfield
accepted the 2024 Nobel Prize in Physics at a
ceremony in Stockholm in December. The prize
honored their pioneering work on the earliest
neural network models, which were based on
the physics of spin glasses.
Wikimedia Commons
Spin Glasses
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 11/26

12. In the 1950s, scientists studying certain dilute
alloys such as iron in gold realized that their
samples were doing some strange things. Above
a certain temperature, these alloys behave
similarly to a normal material such as
aluminum. They aren’t magnetic on their own,
but they do interact weakly with external
magnetic fields. For instance, you can use a very
strong magnet to move an aluminum can, but
aluminum itself can’t work as a magnet.
Usually, materials such as aluminum lose their
magnetization as soon as the external magnet
disappears. But below a certain temperature,
spin glasses do something different. Their
transient magnetization sticks around, albeit at
a lower value. (This isn’t the only weird thing
that spin glasses do; their thermal properties
are also puzzling.)
Around 1970, condensed matter physicists
started to get a theoretical handle on these
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 12/26

13. materials by tweaking physicists’ go-to model
of collective magnetic behavior: the Ising
model.
An Ising model looks like a simple grid of
arrows, each of which can point up or down.
Every arrow represents the intrinsic magnetic
moment, or “spin,” of an atom. This is a
simplification of a real atomic system, but by
tweaking the rules by which nearby spins affect
one another, the model can generate
surprisingly complex behaviors.
In general, nearby arrows that point in the same
direction have low energy, while arrows that
point in opposite directions have high energy. If
the spins are free to flip, the Ising model’s state
will thus evolve towards a lower-energy state of
alignment, like a ball rolling downhill. Magnetic
materials such as iron end up settling into
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 13/26

14. simple states with their spins aligned in either
the all-up or all-down state.
In 1975, the physicists David
Sherrington and Scott Kirkpatrick devised a
model that could capture the more complicated
behavior of spin glasses by modifying the rules
of how spins interact. They randomly varied the
interaction strengths between spin pairs and
allowed each spin to interact with every other
spin — not just its nearest neighbors. That
change led to a rugged “landscape” of possible
energy states. There were peaks and valleys
corresponding to higher and lower energy
configurations; depending on where the spin
glass started off in this landscape, it would end
up in a unique valley, or low-energy equilibrium
state. That’s quite different from ferromagnets
such as iron, which “freeze” into one of two
orderly states with all spins aligned, and
nonmagnets, whose spins fluctuate randomly
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 14/26

15. and don’t settle down at all. In a spin glass,
randomness gets frozen.
The Ising model is very much a toy model.
Using it to try to predict anything about real
materials is a bit like using a stick figure to plan
a surgery. But remarkably, it often works. The
Ising model is now a workhorse of statistical
mechanics. Variations on its theme can be heard
in just about every corner of the study of
complex, collective phenomena — including,
because of Hopfield, memory.
Spin Memory
A simple view of interacting neurons has a lot in
common with an Ising model of magnetic
spins. For one thing, neurons are often modeled
as basically binary on-off switches; they either
fire or they don’t. Spins, likewise, can point
either up or down. In addition, a firing neuron
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 15/26

16. can either encourage or discourage the firing of
its neighbor. These variable interaction
strengths between neurons recall the
changeable interaction strengths between spins
in a spin glass. “Mathematically, one can replace
what were the spins or atoms,”  said Lenka
Zdeborová, a physicist and computer scientist at
the Swiss Federal Institute of Technology
Lausanne. “Other systems can be described
using the same toolbox.”
To make his network, Hopfield started with a
web of artificial neurons that can be either “on”
(firing) or “off” (resting). Each neuron
influences every other neuron’s state, and these
interactions can be adjusted. The network’s
state at any given time is defined by which
neurons are firing and which are at rest. You can
code these two states in binary: A firing neuron
is labeled with a 1 and a resting neuron with a 0.
Write out the state of the entire network at any
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 16/26

17. given moment, and you’ve got a string of bits.
The network doesn’t “store” information,
exactly. It is information.
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 17/26

18. Lenka Zdeborová, a physicist and
computer scientist at the Swiss
Federal Institute of Technology
Lausanne, studies how the
physics of matter can help model
the behavior of machine learning
algorithms.
Samuel Rubio for Quanta
Magazine
To “teach” the network a pattern, Hopfield
sculpted its energy landscape by modifying the
strengths of interactions between neurons so
that the desired pattern fell at a low-energy
steady state. In such a state, the network stops
evolving and stabilizes in just one pattern. He
found a rule for doing this inspired by
neuroscience’s classic “neurons that fire
together wire together” rule. He would tune up
interactions between neurons that both fire (or
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 18/26

19. both rest) in the desired final state and dial
down interactions between mismatched pairs.
Once a network is taught a pattern this way, it
can reach the pattern again simply by
navigating downhill through the network’s
energy landscape; it will naturally reach the
pattern when it settles into an equilibrium state.
“Hopfield made the connection and said, ‘Look,
if we can adapt, tune the exchange couplings in
a spin glass, maybe we can shape the
equilibrium points so that they can become
memories,’” Mézard said.
Hopfield networks can remember multiple
memories, each in its own little energy valley.
Which valley the network falls into depends on
where it begins in its energy landscape. In a
network that stores a picture of a cat and a
picture of a spaceship, for instance, a starting
state that’s vaguely cat-shaped will roll down
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 19/26

20. into the cat valley more often than not.
Likewise, starting the network in a state that
recalls the geometric forms of a spaceship will
usually prompt it to evolve toward the
spaceship. That’s what makes Hopfield
networks a model of associative memory: Given
a corrupted or incomplete version of a memory,
a Hopfield network dynamically reconstructs
the whole thing.
Old Model, New Ideas
From 1983 to 1985, Hinton and his colleagues
built on Hopfield’s work. They found ways to
inject randomness into Hopfield networks to
create a new type of neural network called a
Boltzmann machine. Rather than remember,
these networks learn the statistical patterns in
training data and spin up new data to match
those patterns — an early kind of generative AI.
In the 2000s, Hinton was able to use a pared-
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 20/26

21. down version of the Boltzmann machine to
finally crack the stubborn problem of training
“deep’’ neural networks consisting of multiple
layers of neurons.
By 2012, the success of deep neural networks
developed by Hinton and other pioneers was
impossible to ignore. “It became clear that this
is actually working amazingly well and just
transforming the whole tech industry,”
Zdeborová said. The generative AI models many
of us now interact with every day, including
large language models such as ChatGPT and
image-generation models such as Midjourney,
are all deep neural networks. They can trace
their success back to curious physicists in the
1970s who refused to let the “useless” properties
of spin glasses go unexplained.
Hopfield networks aren’t just part of AI’s past,
however. Thanks to new ideas, these old models
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 21/26

22. could be making a comeback.
In 2016, Hopfield and Dmitry Krotov of IBM
Research realized that Hopfield networks
weren’t just one model, but a whole family of
models with different memory storage
capacities. Then, in 2020, another team showed
that a key part of the transformer architecture,
the blueprint of most modern successful AI
models, was a member of that extended
Hopfield network family.
Armed with that insight, Krotov and his
colleagues recently developed a new deep
learning architecture called the energy
transformer. Typical AI architectures are
usually found by trial and error. But Krotov
thinks energy transformers could be designed
more intentionally with a specific energy
landscape in mind, like a more complex take on
a Hopfield network.
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 22/26

23. Though Hopfield networks were originally
designed to remember, researchers are now
exploring how they can be used to create. Image
generators such as Midjourney are powered by
“diffusion models,” which are themselves
inspired by the physics of diffusion. To train
them, researchers add noise to the training data
— say, pictures of cats — and then teach the
model to remove the noise. That’s a lot like what
a Hopfield network does, except instead of
always landing on the same cat picture, a
diffusion model removes “non-cat” noise from
a noisy, random starting state to produce a new
cat.
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 23/26

24. Dmitry Krotov, a computer
scientist at IBM Research, has
shown that some of the most
advanced AI models in use today
follow the same basic principle
that Hopfield networks
employed from the start.
Kim Martineau
It turns out that diffusion models can be
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 24/26

25. understood as a particular kind of modern
Hopfield network, according to Krotov and his
colleagues, including Benjamin Hoover, Yuchen
Liang and Bao Pham. And that approach can be
used to predict aspects of these networks’
behavior. Their work suggests that feeding a
modern Hopfield network more and more data
doesn’t just saturate its memory. Instead, the
model’s energy landscape gets so rugged that it
is more likely to settle on a made-up memory
than a real one. It becomes a diffusion model.
That a simple change in quantity — in this case,
the amount of training data — can trigger an
unexpected change in quality isn’t anything
new for physicists. As the condensed matter
physicist Philip Anderson wrote back in 1972,
“more is different.” In collective systems, simply
scaling up networks of interactions between
parts can add up to surprising new behaviors.
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 25/26

26. “The fact that [a neural network] works is an
emergent property,” Mézard said.
Emergence in a deep learning architecture — or
a brain — is as captivating as it is puzzling;
there’s no universal theory of emergence.
Perhaps statistical physics, which provided the
first tools for understanding collective behavior,
will be the key not just to using but also to
understanding the inscrutable machine
intelligences changing our world.
6/6/25, 11:34 PM The Strange Physics That Gave Birth to AI | Quanta Magazine
https://www.quantamagazine.org/the-strange-physics-that-gave-birth-to-ai-20250430/ 26/26