The AIGS Pod

The AIGS
  • 5.0 (1)
  • Natural Sciences
  • Updated Biweekly

The AIGS podcast tackling AI in earth system modeling and more.

  1. 3d ago

    How AI weather models reinvented physics

    Could it be that AI weather models are solving physical equations, although they may not be the equations used by conventional NWP models? The authors compute correlations of forecast skill and Centered Kernel Alignment, providing evidence that different AI weather models represent the atmosphere in similar ways, despite differences in architecture and capacity. They argue that the architecture and training of the AI models constrains the form of the physical laws that they might simulate. In particular, they propose that the models implement a particle description of the atmosphere, where the latent variables at each mesh point correspond to the position of a particle in the high dimensional latent space. They hypothesize that the movement of the particles follows a gradient flow in the latent space towards a minimum of a learned free energy functional. Analysis of the GraphCast and Aurora models show that they make changes on large spatial scales in the early processor layers and move to smaller scale with increasing layer depth, consistent with the gradient flow hypothesis. Paper: https://arxiv.org/abs/2605.23778

    How AI weather models reinvented physics

  2. Sep 13

    How AI predicts weather without physics

    Artificial Intelligence (AI) weather models are improving rapidly, and their forecasts are already competitive with long-established traditional Numerical Weather Prediction (NWP). To build confidence in this new methodology, it is critical to understand how these predictions are generated. This is a huge challenge as these AI weather models remain largely black boxes. In other areas of Machine Learning (ML), mechanistic interpretability has emerged as a framework for understanding ML predictions by analysing the building blocks responsible for them. Here the authors present an open-source, highly adaptable tool which incorporates concepts from mechanistic interpretability. The tool organises internal latent representations from the model processor and allows for initial analyses, including cosine similarity and Principal Component Analysis (PCA), enabling the user to identify directions in latent space potentially associated with meteorological features. Applying the tool to the graph neural network GraphCast, they present preliminary case studies for mid-latitude synoptic-scale waves and specific humidity. These demonstrate the tool's ability to identify linear combinations of latent channels that appear to correspond to interpretable features. Paper: https://arxiv.org/abs/2604.20467

    How AI predicts weather without physics

  3. Sep 12

    How AI weather models actually think

    Artificial Intelligence (AI) weather models are improving rapidly, and their forecasts are already competitive with long-established traditional Numerical Weather Prediction (NWP). To build confidence in this new methodology, it is critical to understand how these predictions are generated. This is a huge challenge as these AI weather models remain largely black boxes. In other areas of Machine Learning (ML), mechanistic interpretability has emerged as a framework for understanding ML predictions by analysing the building blocks responsible for them. Here the authors present an open-source, highly adaptable tool which incorporates concepts from mechanistic interpretability. The tool organises internal latent representations from the model processor and allows for initial analyses, including cosine similarity and Principal Component Analysis (PCA), enabling the user to identify directions in latent space potentially associated with meteorological features. Applying the tool to the graph neural network GraphCast, they present preliminary case studies for mid-latitude synoptic-scale waves and specific humidity. These demonstrate the tool's ability to identify linear combinations of latent channels that appear to correspond to interpretable features. Paper: https://arxiv.org/abs/2604.20467

    How AI weather models actually think

  4. Sep 8

    How AI weather models understand physics

    Deep learning weather prediction models achieve remarkable predictive skill yet remain largely opaque: little is known about how they represent physical climate phenomena internally. Mechanistic interpretability through Sparse Autoencoders (SAEs) offers a principled route to decomposing these representations, but existing SAEs assume strictly linear feature superposition - a constraint ill-suited for the highly nonlinear atmospheric dynamics encoded in modern transformers. The authors introduce KAN-SAE, a sparse autoencoder whose encoder replaces the standard ReLU with learnable per-feature B-spline activations drawn from Kolmogorov-Arnold Networks (KANs), allowing each latent dimension to develop its own nonlinear gating profile. Applied to Sonny, KAN-SAE discovers 975 alive features (vs. 566 for a linear baseline, a 72% improvement) with 20% lower inter-feature redundancy and comparable reconstruction fidelity. Without any climate supervision, KAN-SAE identifies an interpretable European heatwave feature spatially concentrated over western Europe, and a western Pacific typhoon tracker confirmed by causal steering experiments. The results demonstrate that nonlinear activations are essential for mechanistic interpretability of deep learning weather prediction models, recovering climate features that remain invisible to linear baselines. Paper: https://arxiv.org/abs/2605.17493

    How AI weather models understand physics

  5. Aug 30

    AI weather models independently learned physics

    Large data-driven physics models like DeepMind's weather model GraphCast have empirically succeeded in parameterizing time operators for complex dynamical systems with an accuracy reaching or in some cases exceeding that of traditional physics-based solvers. Unfortunately, how these data-driven models perform computations is largely unknown and whether their internal representations are interpretable or physically consistent is an open question. Here, the authors adapt tools from interpretability research in Large Language Models to analyze intermediate computational layers in GraphCast, leveraging sparse autoencoders to discover interpretable features in the neuron space of the model. They uncover distinct features on a wide range of length and time scales that correspond to tropical cyclones, atmospheric rivers, diurnal and seasonal behavior, large-scale precipitation patterns, specific geographical coding, and sea-ice extent, among others. They further demonstrate how the precise abstraction of these features can be probed via interventions on the prediction steps of the model. As a case study, they sparsely modify a feature corresponding to tropical cyclones in GraphCast and observe interpretable and physically consistent modifications to evolving hurricanes. Such methods offer a window into the black-box behavior of data-driven physics models and are a step towards realizing their potential as trustworthy predictors and scientifically valuable tools for discovery. Paper: https://arxiv.org/abs/2512.24440

    AI weather models independently learned physics

  6. Aug 29

    GraphCast learns the laws of physics

    Large data-driven physics models like DeepMind's weather model GraphCast have empirically succeeded in parameterizing time operators for complex dynamical systems with an accuracy reaching or in some cases exceeding that of traditional physics-based solvers. Unfortunately, how these data-driven models perform computations is largely unknown and whether their internal representations are interpretable or physically consistent is an open question. Here, the authors adapt tools from interpretability research in Large Language Models to analyze intermediate computational layers in GraphCast, leveraging sparse autoencoders to discover interpretable features in the neuron space of the model. They uncover distinct features on a wide range of length and time scales that correspond to tropical cyclones, atmospheric rivers, diurnal and seasonal behavior, large-scale precipitation patterns, specific geographical coding, and sea-ice extent, among others. They further demonstrate how the precise abstraction of these features can be probed via interventions on the prediction steps of the model. As a case study, they sparsely modify a feature corresponding to tropical cyclones in GraphCast and observe interpretable and physically consistent modifications to evolving hurricanes. Such methods offer a window into the black-box behavior of data-driven physics models and are a step towards realizing their potential as trustworthy predictors and scientifically valuable tools for discovery. Paper: https://arxiv.org/abs/2512.24440

    GraphCast learns the laws of physics

  7. Aug 16

    Walrus masters physics from stars to oceans

    Foundation models have reshaped language and vision, but physical simulation has resisted the same playbook: heterogeneous data, unstable long-term rollouts, and mismatched resolutions and dimensionalities make it hard to train one model across many kinds of physics. This paper introduces Walrus, a transformer-based foundation model for fluid-like continuum dynamics, built around a harmonic-analysis-based stabilization method, load-balanced distributed 2D and 3D training strategies, and compute-adaptive tokenization. Walrus is pretrained on nineteen diverse scenarios spanning astrophysics, geoscience, rheology, plasma physics, acoustics, and classical fluids, treating Earth's atmosphere and oceans as just one flavor of continuum dynamics among many. Across short- and long-horizon forecasts on downstream tasks, and across the full breadth of pretraining domains, Walrus outperforms prior foundation models, while ablation studies confirm that the stabilization, distributed training, and tokenization choices each meaningfully improve forecast stability, training throughput, and transfer performance. Code and weights are released for community use. Paper: https://arxiv.org/abs/2511.15684

    Walrus masters physics from stars to oceans

  8. Aug 9

    Walrus AI unifies nineteen physics scenarios

    Foundation models have reshaped language and vision, but physical simulation has resisted the same playbook: heterogeneous data, unstable long-term rollouts, and mismatched resolutions and dimensionalities make it hard to train one model across many kinds of physics. This paper introduces Walrus, a transformer-based foundation model for fluid-like continuum dynamics, built around a harmonic-analysis-based stabilization method, load-balanced distributed 2D and 3D training strategies, and compute-adaptive tokenization. Walrus is pretrained on nineteen diverse scenarios spanning astrophysics, geoscience, rheology, plasma physics, acoustics, and classical fluids, treating Earth's atmosphere and oceans as just one flavor of continuum dynamics among many. Across short- and long-horizon forecasts on downstream tasks, and across the full breadth of pretraining domains, Walrus outperforms prior foundation models, while ablation studies confirm that the stabilization, distributed training, and tokenization choices each meaningfully improve forecast stability, training throughput, and transfer performance. Code and weights are released for community use. Paper: https://arxiv.org/abs/2511.15684

    Walrus AI unifies nineteen physics scenarios
See All (23)

About

The AIGS podcast tackling AI in earth system modeling and more.

Information

  • Creator
    The AIGS
  • Years Active
    2026
  • Episodes
    23
  • Rating
    Clean
  • Show Website
    The AIGS Pod