Kevin Hartnett by June 10, 2021
Collected at: https://www.quantamagazine.org/the-mystery-at-the-heart-of-physics-that-only-math-can-solve-20210610/
The accelerating effort to understand the mathematics of quantum field theory will have profound consequences for both math and physics.
ver the past century, quantum field theory has proved to be the single most sweeping and successful physical theory ever invented. It is an umbrella term that encompasses many specific quantum field theories — the way “shape” covers specific examples like the square and the circle. The most prominent of these theories is known as the Standard Model, and it is this framework of physics that has been so successful.
“It can explain at a fundamental level literally every single experiment that we’ve ever done,” said David Tong, a physicist at the University of Cambridge.
But quantum field theory, or QFT, is indisputably incomplete. Neither physicists nor mathematicians know exactly what makes a quantum field theory a quantum field theory. They have glimpses of the full picture, but they can’t yet make it out.
“There are various indications that there could be a better way of thinking about QFT,” said Nathan Seiberg, a physicist at the Institute for Advanced Study. “It feels like it’s an animal you can touch from many places, but you don’t quite see the whole animal.”
Mathematics, which requires internal consistency and attention to every last detail, is the language that might make QFT whole. If mathematics can learn how to describe QFT with the same rigor with which it characterizes well-established mathematical objects, a more complete picture of the physical world will likely come along for the ride.
“If you really understood quantum field theory in a proper mathematical way, this would give us answers to many open physics problems, perhaps even including the quantization of gravity,” said Robbert Dijkgraaf, director of the Institute for Advanced Study (and a regular columnist for Quanta).
Every other idea that’s been used in physics over the past centuries had its natural place in mathematics. This is clearly not the case with quantum field theory.
Nathan Seiberg, the Institute for Advanced Study
Nor is this a one-way street. For millennia, the physical world has been mathematics’ greatest muse. The ancient Greeks invented trigonometry to study the motion of the stars. Mathematics turned it into a discipline with definitions and rules that students now learn without any reference to the topic’s celestial origins. Almost 2,000 years later, Isaac Newton wanted to understand Kepler’s laws of planetary motion and attempted to find a rigorous way of thinking about infinitesimal change. This impulse (along with revelations from Gottfried Leibniz) birthed the field of calculus, which mathematics appropriated and improved — and today could hardly exist without.
Now mathematicians want to do the same for QFT, taking the ideas, objects and techniques that physicists have developed to study fundamental particles and incorporating them into the main body of mathematics. This means defining the basic traits of QFT so that future mathematicians won’t have to think about the physical context in which the theory first arose.
The rewards are likely to be great: Mathematics grows when it finds new objects to explore and new structures that capture some of the most important relationships — between numbers, equations and shapes. QFT offers both.
“Physics itself, as a structure, is extremely deep and often a better way to think about mathematical things we’re already interested in. It’s just a better way to organize them,” said David Ben-Zvi, a mathematician at the University of Texas, Austin.
For 40 years at least, QFT has tempted mathematicians with ideas to pursue. In recent years, they’ve finally begun to understand some of the basic objects in QFT itself — abstracting them from the world of particle physics and turning them into mathematical objects in their own right.
Yet it’s still early days in the effort.
“We won’t know until we get there, but it’s certainly my expectation that we’re just seeing the tip of the iceberg,” said Greg Moore, a physicist at Rutgers University. “If mathematicians really understood [QFT], that would lead to profound advances in mathematics.”
Fields Forever
It’s common to think of the universe as being built from fundamental particles: electrons, quarks, photons and the like. But physics long ago moved beyond this view. Instead of particles, physicists now talk about things called “quantum fields” as the real warp and woof of reality.
These fields stretch across the space-time of the universe. They come in many varieties and fluctuate like a rolling ocean. As the fields ripple and interact with each other, particles emerge out of them and then vanish back into them, like the fleeting crests of a wave.
“Particles are not objects that are there forever,” said Tong. “It’s a dance of fields.”
To understand quantum fields, it’s easiest to start with an ordinary, or classical, field. Imagine, for example, measuring the temperature at every point on Earth’s surface. Combining the infinitely many points at which you can make these measurements forms a geometric object, called a field, that packages together all this temperature information.
In general, fields emerge whenever you have some quantity that can be measured uniquely at infinitely fine resolution across a space. “You’re sort of able to ask independent questions about each point of space-time, like, what’s the electric field here versus over there,” said Davide Gaiotto, a physicist at the Perimeter Institute for Theoretical Physics in Waterloo, Canada.
Quantum fields come about when you’re observing quantum phenomena, like the energy of an electron, at every point in space and time. But quantum fields are fundamentally different from classical ones.
While the temperature at a point on Earth is what it is, regardless of whether you measure it, electrons have no definite position until the moment you observe them. Prior to that, their positions can only be described probabilistically, by assigning values to every point in a quantum field that captures the likelihood you’ll find an electron there versus somewhere else. Prior to observation, electrons essentially exist nowhere — and everywhere.
“Most things in physics aren’t just objects; they’re something that lives in every point in space and time,” said Dijkgraaf.
A quantum field theory comes with a set of rules called correlation functions that explain how measurements at one point in a field relate to — or correlate with — measurements taken at another point.
Each quantum field theory describes physics in a specific number of dimensions. Two-dimensional quantum field theories are often useful for describing the behavior of materials, like insulators; six-dimensional quantum field theories are especially relevant to string theory; and four-dimensional quantum field theories describe physics in our actual four-dimensional universe. The Standard Model is one of these; it’s the single most important quantum field theory because it’s the one that best describes the universe.
There are 12 known fundamental particles that make up the universe. Each has its own unique quantum field. To these 12 particle fields the Standard Model adds four force fields, representing the four fundamental forces: gravity, electromagnetism, the strong nuclear force and the weak nuclear force. It combines these 16 fields in a single equation that describes how they interact with each other. Through these interactions, fundamental particles are understood as fluctuations of their respective quantum fields, and the physical world emerges before our eyes.
It might sound strange, but physicists realized in the 1930s that physics based on fields, rather than particles, resolved some of their most pressing inconsistencies, ranging from issues regarding causality to the fact that particles don’t live forever. It also explained what otherwise appeared to be an improbable consistency in the physical world.
“All particles of the same type everywhere in the universe are the same,” said Tong. “If we go to the Large Hadron Collider and make a freshly minted proton, it’s exactly the same as one that’s been traveling for 10 billion years. That deserves some explanation.” QFT provides it: All protons are just fluctuations in the same underlying proton field (or, if you could look more closely, the underlying quark fields).
But the explanatory power of QFT comes at a high mathematical cost.
“Quantum field theories are by far the most complicated objects in mathematics, to the point where mathematicians have no idea how to make sense of them,” said Tong. “Quantum field theory is mathematics that has not yet been invented by mathematicians.”
Too Much Infinity
What makes it so complicated for mathematicians? In a word, infinity.
When you measure a quantum field at a point, the result isn’t a few numbers like coordinates and temperature. Instead, it’s a matrix, which is an array of numbers. And not just any matrix — a big one, called an operator, with infinitely many columns and rows. This reflects how a quantum field envelops all the possibilities of a particle emerging from the field.
“There are infinitely many positions that a particle can have, and this leads to the fact that the matrix that describes the measurement of position, of momentum, also has to be infinite-dimensional,” said Kasia Rejzner of the University of York.
And when theories produce infinities, it calls their physical relevance into question, because infinity exists as a concept, not as anything experiments can ever measure. It also makes the theories hard to work with mathematically.
“We don’t like having a framework that spells out infinity. That’s why you start realizing you need a better mathematical understanding of what’s going on,” said Alejandra Castro, a physicist at the University of Amsterdam.
The problems with infinity get worse when physicists start thinking about how two quantum fields interact, as they might, for instance, when particle collisions are modeled at the Large Hadron Collider outside Geneva. In classical mechanics this type of calculation is easy: To model what happens when two billiard balls collide, just use the numbers specifying the momentum of each ball at the point of collision.
When two quantum fields interact, you’d like to do a similar thing: multiply the infinite-dimensional operator for one field by the infinite-dimensional operator for the other at exactly the point in space-time where they meet. But this calculation — multiplying two infinite-dimensional objects that are infinitely close together — is difficult.
“This is where things go terribly wrong,” said Rejzner.
Smashing Success
Physicists and mathematicians can’t calculate using infinities, but they have developed workarounds — ways of approximating quantities that dodge the problem. These workarounds yield approximate predictions, which are good enough, because experiments aren’t infinitely precise either.
“We can do experiments and measure things to 13 decimal places and they agree to all 13 decimal places. It’s the most astonishing thing in all of science,” said Tong.
One workaround starts by imagining that you have a quantum field in which nothing is happening. In this setting — called a “free” theory because it’s free of interactions — you don’t have to worry about multiplying infinite-dimensional matrices because nothing’s in motion and nothing ever collides. It’s a situation that’s easy to describe in full mathematical detail, though that description isn’t worth a whole lot.
“It’s totally boring, because you’ve described a lonely field with nothing to interact with, so it’s a bit of an academic exercise,” said Rejzner.
But you can make it more interesting. Physicists dial up the interactions, trying to maintain mathematical control of the picture as they make the interactions stronger.
This approach is called perturbative QFT, in the sense that you allow for small changes, or perturbations, in a free field. You can apply the perturbative perspective to quantum field theories that are similar to a free theory. It’s also extremely useful for verifying experiments. “You get amazing accuracy, amazing experimental agreement,” said Rejzner.
But if you keep making the interactions stronger, the perturbative approach eventually overheats. Instead of producing increasingly accurate calculations that approach the real physical universe, it becomes less and less accurate. This suggests that while the perturbation method is a useful guide for experiments, ultimately it’s not the right way to try and describe the universe: It’s practically useful, but theoretically shaky.
“We do not know how to add everything up and get something sensible,” said Gaiotto.
We’ve been using QFT as an outside stimulus, but it would be nice if it were an inside stimulus.
Dan Freed, the University of Texas, Austin
Another approximation scheme tries to sneak up on a full-fledged quantum field theory by other means. In theory, a quantum field contains infinitely fine-grained information. To cook up these fields, physicists start with a grid, or lattice, and restrict measurements to places where the lines of the lattice cross each other. So instead of being able to measure the quantum field everywhere, at first you can only measure it at select places a fixed distance apart.
From there, physicists enhance the resolution of the lattice, drawing the threads closer together to create a finer and finer weave. As it tightens, the number of points at which you can take measurements increases, approaching the idealized notion of a field where you can take measurements everywhere.
“The distance between the points becomes very small, and such a thing becomes a continuous field,” said Seiberg. In mathematical terms, they say the continuum quantum field is the limit of the tightening lattice.
Mathematicians are accustomed to working with limits and know how to establish that certain ones really exist. For example, they’ve proved that the limit of the infinite sequence 12 + 14 +18 +116 … is 1. Physicists would like to prove that quantum fields are the limit of this lattice procedure. They just don’t know how.
“It’s not so clear how to take that limit and what it means mathematically,” said Moore.
Physicists don’t doubt that the tightening lattice is moving toward the idealized notion of a quantum field. The close fit between the predictions of QFT and experimental results strongly suggests that’s the case.
“There is no question that all these limits really exist, because the success of quantum field theory has been really stunning,” said Seiberg. But having strong evidence that something is correct and proving conclusively that it is are two different things.
It’s a degree of imprecision that’s out of step with the other great physical theories that QFT aspires to supersede. Isaac Newton’s laws of motion, quantum mechanics, Albert Einstein’s theories of special and general relativity — they’re all just pieces of the bigger story QFT wants to tell, but unlike QFT, they can all be written down in exact mathematical terms.
“Quantum field theory emerged as an almost universal language of physical phenomena, but it’s in bad math shape,” said Dijkgraaf. And for some physicists, that’s a reason for pause.
“If the full house is resting on this core concept that itself isn’t understood in a mathematical way, why are you so confident this is describing the world? That sharpens the whole issue,” said Dijkgraaf.
Outside Agitator
Even in this incomplete state, QFT has prompted a number of important mathematical discoveries. The general pattern of interaction has been that physicists using QFT stumble onto surprising calculations that mathematicians then try to explain.
“It’s an idea-generating machine,” said Tong.
At a basic level, physical phenomena have a tight relationship with geometry. To take a simple example, if you set a ball in motion on a smooth surface, its trajectory will illuminate the shortest path between any two points, a property known as a geodesic. In this way, physical phenomena can detect geometric features of a shape.
Now replace the billiard ball with an electron. The electron exists probabilistically everywhere on a surface. By studying the quantum field that captures those probabilities, you can learn something about the overall nature of that surface (or manifold, to use the mathematicians’ term), like how many holes it has. That’s a fundamental question that mathematicians working in geometry, and the related field of topology, want to answer.
“One particle even sitting there, doing nothing, will start to know about the topology of a manifold,” said Tong.
In the late 1970s, physicists and mathematicians began applying this perspective to solve basic questions in geometry. By the early 1990s, Seiberg and his collaborator Edward Witten figured out how to use it to create a new mathematical tool — now called the Seiberg-Witten invariants — that turns quantum phenomena into an index for purely mathematical traits of a shape: Count the number of times quantum particles behave in a certain way, and you’ve effectively counted the number of holes in a shape.
“Witten showed that quantum field theory gives completely unexpected but completely precise insights into geometrical questions, making intractable problems soluble,” said Graeme Segal, a mathematician at the University of Oxford.
Another example of this exchange also occurred in the early 1990s, when physicists were doing calculations related to string theory. They performed them in two different geometric spaces based on fundamentally different mathematical rules and kept producing long sets of numbers that matched each other exactly. Mathematicians picked up the thread and elaborated it into a whole new field of inquiry, called mirror symmetry, that investigates the concurrence — and many others like it.
“Physics would come up with these amazing predictions, and mathematicians would try to prove them by our own means,” said Ben-Zvi. “The predictions were strange and wonderful, and they turned out to be pretty much always correct.”
But while QFT has been successful at generating leads for mathematics to follow, its core ideas still exist almost entirely outside of mathematics. Quantum field theories are not objects that mathematicians understand well enough to use the way they can use polynomials, groups, manifolds and other pillars of the discipline (many of which also originated in physics).
For physicists, this distant relationship with math is a sign that there’s a lot more they need to understand about the theory they birthed. “Every other idea that’s been used in physics over the past centuries had its natural place in mathematics,” said Seiberg. “This is clearly not the case with quantum field theory.”
I like to say the physicists don’t necessarily know everything, but the physics does.
David Ben-Zvi, the University of Texas, Austin
And for mathematicians, it seems as if the relationship between QFT and math should be deeper than the occasional interaction. That’s because quantum field theories contain many symmetries, or underlying structures, that dictate how points in different parts of a field relate to each other. These symmetries have a physical significance — they embody how quantities like energy are conserved as quantum fields evolve over time. But they’re also mathematically interesting objects in their own right.
“A mathematician might care about a certain symmetry, and we can put it in a physical context. It creates this beautiful bridge between these two fields,” said Castro.
Mathematicians already use symmetries and other aspects of geometry to investigate everything from solutions to different types of equations to the distribution of prime numbers. Often, geometry encodes answers to questions about numbers. QFT offers mathematicians a rich new type of geometric object to play with — if they can get their hands on it directly, there’s no telling what they’ll be able to do.
“We’re to some extent playing with QFT,” said Dan Freed, a mathematician at the University of Texas, Austin. “We’ve been using QFT as an outside stimulus, but it would be nice if it were an inside stimulus.”
Make Way for QFT
Mathematics does not admit new subjects lightly. Many basic concepts went through long trials before they settled into their proper, canonical places in the field.
Take the real numbers — all the infinitely many tick marks on the number line. It took math nearly 2,000 years of practice to agree on a way of defining them. Finally, in the 1850s, mathematicians settled on a precise three-word statement describing the real numbers as a “complete ordered field.” They’re complete because they contain no gaps, they’re ordered because there’s always a way of determining whether one real number is greater or less than another, and they form a “field,” which to mathematicians means they follow the rules of arithmetic.
“Those three words are historically hard fought,” said Freed.
In order to turn QFT into an inside stimulus — a tool they can use for their own purposes — mathematicians would like to give the same treatment to QFT they gave to the real numbers: a sharp list of characteristics that any specific quantum field theory needs to satisfy.
A lot of the work of translating parts of QFT into mathematics has come from a mathematician named Kevin Costello at the Perimeter Institute. In 2016 he coauthored a textbook that puts perturbative QFT on firm mathematical footing, including formalizing how to work with the infinite quantities that crop up as you increase the number of interactions. The work follows an earlier effort from the 2000s called algebraic quantum field theory that sought similar ends, and which Rejzner reviewed in a 2016 book. So now, while perturbative QFT still doesn’t really describe the universe, mathematicians know how to deal with the physically non-sensical infinities it produces.
“His contributions are extremely ingenious and insightful. He put [perturbative] theory in a nice new framework that is suitable for rigorous mathematics,” said Moore.
Costello explains he wrote the book out of a desire to make perturbative quantum field theory more coherent. “I just found certain physicists’ methods unmotivated and ad hoc. I wanted something more self-contained that a mathematician could go work with,” he said.
By specifying exactly how perturbation theory works, Costello has created a basis upon which physicists and mathematicians can construct novel quantum field theories that satisfy the dictates of his perturbation approach. It’s been quickly embraced by others in the field.
“He certainly has a lot of young people working in that framework. [His book] has had its influence,” said Freed.
Costello has also been working on defining just what a quantum field theory is. In stripped-down form, a quantum field theory requires a geometric space in which you can make observations at every point, combined with correlation functions that express how observations at different points relate to each other. Costello’s work describes the properties a collection of correlation functions needs to have in order to serve as a workable basis for a quantum field theory.
The most familiar quantum field theories, like the Standard Model, contain additional features that may not be present in all quantum field theories. Quantum field theories that lack these features likely describe other, still undiscovered properties that could help physicists explain physical phenomena the Standard Model can’t account for. If your idea of a quantum field theory is fixed too closely to the versions we already know about, you’ll have a hard time even envisioning the other, necessary possibilities.
“There is a big lamppost under which you can find theories of fields [like the Standard Model], and around it is a big darkness of [quantum field theories] we don’t know how to define, but we know they’re there,” said Gaiotto.
Costello has illuminated some of that dark space with his definitions of quantum fields. From these definitions, he’s discovered two surprising new quantum field theories. Neither describes our four-dimensional universe, but they do satisfy the core demands of a geometric space equipped with correlation functions. Their discovery through pure thought is similar to how the first shapes you might discover are ones present in the physical world, but once you have a general definition of a shape, you can think your way to examples with no physical relevance at all.
And if mathematics can determine the full space of possibilities for quantum field theories — all the many different possibilities for satisfying a general definition involving correlation functions — physicists can use that to find their way to the specific theories that explain the important physical questions they care most about.
“I want to know the space of all QFTs because I want to know what quantum gravity is,” said Castro.
A Multi-Generational Challenge
There’s a long way to go. So far, all of the quantum field theories that have been described in full mathematical terms rely on various simplifications, which make them easier to work with mathematically.
One way to simplify the problem, going back decades, is to study simpler two-dimensional QFTs rather than four-dimensional ones. A team in France recently nailed down all the mathematical details of a prominent two-dimensional QFT.
Other simplifications assume quantum fields are symmetrical in ways that don’t match physical reality, but that make them more tractable from a mathematical perspective. These include “supersymmetric” and “topological” QFTs.
The next, and much more difficult, step will be to remove the crutches and provide a mathematical description of a quantum field theory that better suits the physical world physicists most want to describe: the four-dimensional, continuous universe in which all interactions are possible at once.
“This is [a] very embarrassing thing that we don’t have a single quantum field theory we can describe in four dimensions, nonperturbatively,” said Rejzner. “It’s a hard problem, and apparently it needs more than one or two generations of mathematicians and physicists to solve it.”
But that doesn’t stop mathematicians and physicists from eyeing it greedily. For mathematicians, QFT is as rich a type of object as they could hope for. Defining the characteristic properties shared by all quantum field theories will almost certainly require merging two of the pillars of mathematics: analysis, which explains how to control infinities, and geometry, which provides a language for talking about symmetry.
“It’s a fascinating problem just in math itself, because it combines two great ideas,” said Dijkgraaf.
If mathematicians can understand QFT, there’s no telling what mathematical discoveries await in its unlocking. Mathematicians defined the characteristic properties of other objects, like manifolds and groups, long ago, and those objects now permeate virtually every corner of mathematics. When they were first defined, it would have been impossible to anticipate all their mathematical ramifications. QFT holds at least as much promise for math.
“I like to say the physicists don’t necessarily know everything, but the physics does,” said Ben-Zvi. “If you ask it the right questions, it already has the phenomena mathematicians are looking for.”
And for physicists, a complete mathematical description of QFT is the flip side of their field’s overriding goal: a complete description of physical reality.
“I feel there is one intellectual structure that covers all of it, and maybe it will encompass all of physics,” said Seiberg.
Now mathematicians just have to uncover it.