Introduction to Climate Modeling
Environmental scientists are increasingly using enormous artificial intelligence models to make predictions about changes in weather and climate. However, a new study by MIT researchers shows that bigger models are not always better. The team demonstrates that, in certain climate scenarios, much simpler, physics-based models can generate more accurate predictions than state-of-the-art deep-learning models.
The Problem with Benchmarking Techniques
The researchers’ analysis also reveals that a benchmarking technique commonly used to evaluate machine-learning techniques for climate predictions can be distorted by natural variations in the data, like fluctuations in weather patterns. This could lead someone to believe a deep-learning model makes more accurate predictions when that is not the case. The researchers developed a more robust way of evaluating these techniques, which shows that, while simple models are more accurate when estimating regional surface temperatures, deep-learning approaches can be the best choice for estimating local rainfall.
Climate Emulators
The researchers used these results to enhance a simulation tool known as a climate emulator, which can rapidly simulate the effect of human activities onto a future climate. Because the Earth’s climate is so complex, running a state-of-the-art climate model to predict how pollution levels will impact environmental factors like temperature can take weeks on the world’s most powerful supercomputers. Scientists often create climate emulators, simpler approximations of a state-of-the art climate model, which are faster and more accessible.
Comparing Emulators
The MIT researchers performed a study comparing a traditional technique called linear pattern scaling (LPS) with a deep-learning model using a common benchmark dataset for evaluating climate emulators. Their results showed that LPS outperformed deep-learning models on predicting nearly all parameters they tested, including temperature and precipitation. However, some initial results seemed to fly in the face of the researchers’ domain knowledge. The powerful deep-learning model should have been more accurate when making predictions about precipitation, since those data don’t follow a linear pattern.
A New Evaluation Method
The researchers found that the high amount of natural variability in climate model runs can cause the deep learning model to perform poorly on unpredictable long-term oscillations, like El Niño/La Niña. This skews the benchmarking scores in favor of LPS, which averages out those oscillations. From there, the researchers constructed a new evaluation with more data that address natural climate variability. With this new evaluation, the deep-learning model performed slightly better than LPS for local precipitation, but LPS was still more accurate for temperature predictions.
Implications and Future Research
Based on these results, the researchers incorporated LPS into a climate emulation platform to predict local temperature changes in different emission scenarios. They hope their results emphasize the need to develop better benchmarking techniques, which could provide a fuller picture of which climate emulation technique is best suited for a particular situation. Ultimately, more accurate benchmarking techniques will help ensure policymakers are making decisions based on the best available information.
Conclusion
The study by MIT researchers serves as a cautionary tale about the risk of deploying large AI models for climate science. While deep-learning models have shown incredible success in domains such as natural language, climate science contains a proven set of physical laws and approximations, and the challenge becomes how to incorporate those into AI models. The researchers hope others build on their analysis, perhaps by studying additional improvements to climate emulation methods and benchmarks.
FAQs
Q: What is a climate emulator?
A: A climate emulator is a simpler approximation of a state-of-the-art climate model, which is faster and more accessible.
Q: What is the problem with using deep-learning models for climate predictions?
A: Deep-learning models can be distorted by natural variations in the data, like fluctuations in weather patterns, which can lead to inaccurate predictions.
Q: What is the benefit of using linear pattern scaling (LPS) for climate predictions?
A: LPS can generate more accurate predictions than state-of-the-art deep-learning models for certain climate scenarios, particularly for estimating regional surface temperatures.
Q: What is the importance of developing better benchmarking techniques for climate emulation?
A: Better benchmarking techniques can provide a fuller picture of which climate emulation technique is best suited for a particular situation, ensuring policymakers make decisions based on the best available information.
