Saturday, June 27, 2026

When AI Models of Physical Systems Fail

Current-generation AIs are particularly good at finding and generating corresponding patterns. This can be useful when classifying images as cats or dogs (classifiers) or creating new sentences or images (genAI). But those patterns in physical systems arise from the behavior of physical laws. Oscillators are a good example -- a pendulum's behavior is due to its own potential and kinetic energy, momentum, and the restoring force of gravity. An AI that could derive the laws describing the behavior of pendulums was demonstrated more than fifteen years ago. That's great, and at the time was touted as heralding a new automation of science.

In the last few years, though, AIs have been tasked with predicting the weather. Three years ago, there were claims that DeepMind could outperform European weather models on some tasks. This is great when the task involves finding a pattern. What happens when the system deviates from the pattern? We know that happens whenever there is sensitive dependence on initial conditions (chaos). We also know that there are limitations to the accuracy of computer arithmetic, even at the most fundamental level.

If we become too dependent on systems that mainly look for patterns, we might find deeper patterns than those that we already knew about, but we might miss out on finding new physical phenomena, especially emergent behavior.

To me, this is a pretty fundamental concern about replacing mathematical modeling with AIs. Just recently, it was shown that physics-based models still outperform AIs on certain weather tasks, and I think this is why.

So, let's be careful about simply handing over both the thinking and the computation to AI, okay?

No comments: