[20240624_LabSeminar_Huy]Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective​.pptx

Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective​

1. Quang-Huy Tran Network Science Lab Dept. of Artificial Intelligence The Catholic University of Korea E-mail: [email protected] 2024-06-24 Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective Binwu Wang et al. AAAI-2024: Proceedings of the Thirty-Eight Conference on Artificial Intelligence

2. OUTLINE MOTIVATION INTRODUCTION METHODOLOGY EXPERIMENT & RESULT CONCLUSION

3. MOTIVATION Spatial-temporal graph prediction has emerged as an essential task in the intelligent transportation systems . The prevailing approaches: GCN for spatial correlation and sequence modules for temporal correlation. Overview Most of them are evaluated using shor t-term dataset and portray the underlying graph as static and unchanging. When considering a longer time frame, the distribution of the graph can undergo substantial evolution over time - a dynamic spatial-temporal graph: the underlying structure of the graph would change over time. the distribution feature of the original nodes in the graph would also evolve over time.

4. MOTIVATION Overview: Previous Approach and Limitation Existing works rely on two main approaches: the retraining method. continuous learning strategies. Limitation of these methods. assume that there is sufficient data available from the updated graph for at least one month . very ideal and oversimplifies the dynamic graph scenario.

5. INTRODUCTION reframe the problem of spatial-temporal prediction within the context of data streaming and tackle this problem using dynamic spatial-temporal graph learning . proposed a decoupled learning framework (DLF): a disentangled spatial-temporal graph network (DSTG) and a decoupled training strategy STG decouples temporal correlation into seasonal and trend patterns. employ a decoupled training strategy that alternately updates these two patterns, facilitating efficient and effective dynamic graph learning.

6. METHODOLOGY Graph Construction and Problem Definition A dynamic spatial-temporal graph represents the graph during the - th month. is the node feature matrix with feature dimension F of nodes in the past time steps.   Problem: learn a model function to predict future graph signals   Strategies for retraining method: requires collected data from for training to encompass period ( ) months   Strategies for online learning method: collect half of the month’s data in - th month to fine-tune the archive model with parameters  

7. METHODOLOGY Main Architecture

8. METHODOLOGY Spatial-Temporal Graph Learning (DSTG) Input module: input signal is separated into seasonal factors and trend factors incorporate time position information into the model. Disentangled spatial-temporal module : two GCNs and a disentangled temporal layer. GCN: Given the input at - th layer . Disentangled temporal module: given a node representation , decompose into a trend and a seasonal by a moving average kernel in the input layer. seasonal is applied self-attention and a position-wise feedforward layer: trend is input into Temporal Convolutional Network to capture short-term patterns. Combine both.   Output module: CNNs is used to generate predictions.

9. METHODOLOGY Decoupled Training Strategy involves updating two patterns in an alternating manner. Training for seasonal pattern: utilize three-month data to thoroughly train the model for seasonal patterns masked autoencoder mechanism: start with a continuous long-term series as the input. divide this input sequence into P patches where P is set to a large value (shape P: , where is length of the input sequence). create a challenging self-supervised task by randomly masking a subset of patches with a masking ratio up to 75% - reduce computational complexity while providing sufficient long-term information.  

10. METHODOLOGY Decoupled Training Strategy Fine-tuning for new knowledge: select a subset of each graph to continuously fine-tune the weights of trend. Detect evolved nodes whose patterns have changed significantly for expanding unseen patterns and replay nodes whose patterns are consistent for reinforcing learned patterns. select the half-month data from training archived model and current update data . sum these two sequences along the time dimension and obtain daily average flow vectors and , is time-steps of one-day data and and represent the number of nodes. calculate the similarity between and of node based on Wasserstein distance. sample the top of nodes with high distances, as well as newly added nodes that appear in the archived data, along with their N-hop neighbors.  

11. EXPERIMENT AND RESULT EXPERIMENT SETTINGs Dataset: Traffic dataset PeMS in California . Baselines: Retraining learning: TCN, STGCN[1], STNN[2], ST-GAM[3], and DSTG+AD (DSTG with all available data). Continuous learning: DSTG+SK(STKEC)[4], DSTG+TS(Traffic Stream)[5], and DSTG+NN(new nodes with N-hop neighbors to fine-tune all parameters. Online learning: DSTG-Static(no fine-tuning). [1] Yu, B., Yin, H., & Zhu, Z. (2017). Spatio -temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875. [2] Yang, S., Liu, J., & Zhao, K. (2021, December). Space meets time: Local spacetime neural network for traffic flow forecasting. In 2021 IEEE International Conference on Data Mining (ICDM) (pp. 817-826). IEEE. [3] Wang, P., Zhu, C., Wang, X., Zhou, Z., Wang, G., & Wang, Y. (2022). Inferring intersection traffic patterns with sparse video surveillance information: An st-gan method. IEEE Transactions on Vehicular Technology, 71(9), 9840-9852. [4] Wang, B., Zhang, Y., Wang, P., Wang, X., Bai, L., & Wang, Y. (2023, April). A Knowledge-Driven Memory System for Traffic Flow Prediction. In International Conference on Database Systems for Advanced Applications (pp. 192-207). Cham: Springer Nature Switzerland. [5] Chen, X., Wang, J., & Xie, K. (2021). TrafficStream : A streaming traffic flow forecasting framework based on graph neural networks and continual learning. arXiv preprint arXiv:2106.06273. Measurement : Mean Absolute Error(MAE), Mean Absolute Percentage Error (MAPE), and Root Mean Square Error (RMSE).

12. EXPERIMENT AND RESULT RESULT – Overall Perfor mance

13. EXPERIMENT AND RESULT Result: Generalization of Models Evaluate it on the Knowair dataset (different domain): the air pollution feature PM2.5 and 18 meteorological features.

14. CONCLUSION proposing a decoupled learning framework on spatial-temporal graph prediction with a dynamic scenario. a disentangled spatial-temporal graph convolutional network (DSTG) and a decoupled training strategy. DSTG decomposes temporal correlations into seasonal and trend patterns. The training strategy updates these patterns alternately to facilitate dynamic spatial-temporal graph learning.