• Physics 17, 110
Machine learning frameworks predict when complex systems such as ecosystems and power grids will undergo significant change.
The world is full of sudden changes that are difficult to predict in advance. For example, ecologists have shown that gradual (rather than sudden) changes in the environment can lead to sudden, regional mass extinctions of species. Over the past two decades, analysis of data from ecology, epidemiology, and other fields has identified several statistical markers that can predict such sudden changes in advance (Figure 1). 1While these “early warning signals” are qualitatively successful, they rarely can accurately predict when a sudden change will occur. Liu Zijia of Tongji University in China and his colleagues have devised an innovative machine learning method that does just that. [1]Based on relatively short timescale observations of complex systems (e.g., populations of different species in an ecosystem), their predictive framework quantitatively predicts when sudden changes will occur across different types of dynamics, networks and scenarios.
Ecosystems, power grids, and living organisms are all complex systems that can experience regime shifts, in which global parameters such as species populations or power output suddenly change in value. These shifts, also known as tipping points or critical transitions, can occur even when the surrounding environment is relatively stable. The study of these regime shifts uses tools from statistical physics, dynamical systems theory, and other fields to analyze observational data and predict sudden regime shifts before they occur. Previous research has produced a list of data markers that can warn that a system is about to shift. [2]The sample variance and lag-1 autocorrelation (the correlation between data points taken one time period apart) of an observed time series are among the most commonly used early warning signals: in typical complex systems, they increase as the system approaches an imminent regime shift.
But these signals don’t always work well in networked systems – systems made up of seemingly independent but interacting entities. [3–5]Moreover, finding early warning signals that warn of impending regime shifts is less challenging than finding signals that quantitatively predict when a regime shift will occur, and recent research in ecology has addressed the latter problem by developing predictive models. [6]However, this approach has not yet been generalized. Another trend in early warning signal research is to use machine learning, which is becoming more common in physics research. [7]For example, machine learning algorithms, when fed data from a variety of complex systems, were able to identify types of regime shifts and provide early warning signals. [8]However, previous machine learning methods have not yet been able to quantitatively predict regime shifts of different types of dynamics on networks, which is what Liu’s team has now achieved. [1].
The researchers evaluated the performance of different machine learning models and decided to use a neural network architecture consisting of a layer of so-called graph isomorphic networks (GINs) followed by a layer of so-called gated recurrent unit (GRU) neural networks. The GIN layers take as input the time series data observed at the different nodes of the network. For example, each node is a geographic location and the data might track the number of organisms or the amount of precipitation at that location over time. The GRU neural network layer takes the output of the GIN layer and detects recurring patterns in the time series data. Recurrent neural networks are generally well suited for time series data. In this way, the GIN-GRU neural network predicts when the networked system will undergo a regime shift.
Liu and colleagues validated the effectiveness of the GIN-GRU forecasting method on numerical simulations of dynamical systems, such as the synchronous transitions of coupled oscillators, and on real data from observations, such as the vegetation changes occurring in Central Africa as the annual mean precipitation gradually declines. They also performed robustness tests of the predictor and further demonstrated its transferability, which is the ability to use knowledge gained in a previous task to improve performance on a related task. Such transfer learning ability is important because when we want to predict turning points in new situations, long-term observational data may not be available. In such situations, pre-training the neural network predictor on different, but related, and sufficiently abundant data can prime the algorithm to succeed well with a relatively small amount of data from the target system.
What comes next? The researchers discussed the goal of reducing the required data length, which is currently set at 20 time points per node. This is an important research direction because in certain environments, only a few data points per node may be available before gradually transitioning to another state. Furthermore, some nodes may be more useful than others for building early warning signals. Future work also includes improving transfer learning between different networks, such as those with different numbers of nodes or dynamics. It will also be interesting to see how their predictor performs on real data and how it contributes to applications outside of physics, for example by collaborating with experts in ecology or psychiatry.
References
- Z. Liu etc“Early prediction of the onset of important transitions in networked dynamical systems” Physics Revision X 14031009 (2024).
- M. Schaefer etc“Anticipating important transitions” Science 338344 (2012).
- A.C. Patterson etc“When and where to find early warning signals when species-diverse systems are approaching tipping points: insights from theory” America. Nat. 198E12 (2021).
- N.G. McLaren etc“Early warning of multi-stage transitions in network dynamics” JR Society, Interface 2020220743 (2023).
- Noriaki Masuda etc“Predicting regime changes by blending early warning signals from different nodes” National Communications 151086 (2024).
- H. Chan etc“Comparative Estimation of Distance to Tipping Points in Interdependent Systems Using Scales of Recovery Rates” National Institute for Environmental Studies 61524 (2022).
- G. Carleo etc“Machine Learning and Physical Sciences” Rev. Mod. Phys. 91045002 (2019).
- TM Berry etc“Deep Learning for Early Warning Signals of Tipping Points” Proceedings of the National Academy of Sciences 118e2106140118 (2021).