Citation: Henn, B., Bretherton, C. S., Kodunov, N., Lessig, C., Molina, M. J., Arcomano, T., Watt-Meyer, O., Couairon, G., Singh, R., Brunstein, R., Hasson, Y., Jost, A., Brenowitz, N., Manshausen, P., Cresswell-Clay, N., Durran, D., Hall, K. J. C., Yuval, J., Kochkov, D., Hoyer, S., & Lopez-Gomez, I. (2026). AIMIP Phase 1: systematic evaluations of AI weather and climate models. arXiv preprint. A New Benchmarking Era for AI Climate Models: AIMIP Phase 1 establishes the first systematic intercomparison framework for artificial intelligence weather and climate models (AIWCMs). It defines a common experimental protocol, standardizes CMIP-compatible output formats, and provides an open dataset to evaluate how different AI architectures influence long-term climate simulation behaviors.Standardized Historical Simulation Protocol: Under the Phase 1 protocol, participating models are trained exclusively on historical ERA5 atmospheric reanalysis data from 1979 to 2014 and run through a 10-year out-of-sample test period (2015–2024). To prevent overfitting, models are forced only by specified sea surface temperatures (SST) and sea ice concentrations (SIC), with direct greenhouse gas inputs (like CO2 concentrations) strictly excluded.Excellent Representation of Baseline Climate and ENSO: The initial evaluations of the eight participating AI models demonstrate that they represent time-mean climate averages and natural variability patterns—such as the El Niño-Southern Oscillation (ENSO)—just as well as, or in some cases with lower systematic biases than, conventional physically-based climate models like the NOAA GFDL-CM4.The Out-of-Sample Warming Gap: A primary weakness identified across several AI models is their struggle to accurately replicate global warming trends during the out-of-sample test period (2015–2024). Because greenhouse gases like CO2 are omitted as direct predictors to avoid overfitting, some models fail to translate rising ocean temperatures into the full magnitude of observed atmospheric warming.Extreme Extrapolation Remains a Challenge: When subjected to extreme, highly out-of-sample sensitivity experiments where sea surface temperatures are uniformly raised by +2 K and +4 K, the AI models diverge significantly. They produce highly inconsistent and sometimes physically implausible responses (such as simulated cooling over land), highlighting that projecting unseen future climates remains a key development challenge for the AI climate modeling community.