[20240617_LabSeminar_Huy]Long-term Spatio-Temporal Forecasting via Dynamic Multiple-Graph Attention.pptx

Long-term Spatio-Temporal Forecasting via Dynamic Multiple-Graph Attention

1. Quang-Huy Tran Network Science Lab Dept. of Artificial Intelligence The Catholic University of Korea E-mail: [email protected] 2024-06-17 Long-term Spatio -Temporal Forecasting via Dynamic Multiple-Graph Attention Wei Shao et al. IJCAI-2022: Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence

2. OUTLINE MOTIVATION INTRODUCTION METHODOLOGY EXPERIMENT & RESULT CONCLUSION

3. MOTIVATION Long-term spatio-temporal forecasting (LSTF) are giving benefit to prediction tasks: long-term dependency structure between the spatial and temporal domains with the contextual information . i.e. transportation, air quality, etc. Overview One main way for LSTF: multi-graph neural networks (MGNNs) [1] . [1] Wang, C., Zhu, Y., Zang, T., Liu, H., & Yu, J. (2021, March). Modeling inter-station relationships with attentive temporal graph convolutional network for air quality prediction. In Proceedings of the 14th ACM international conference on web search and data mining (pp. 616-634).

4. MOTIVATION Overview: LSTF Limitation Most existing GNN studies consider only the spatial similarity of nodes: i.e the distance similarity and neighborhood correlation. insufficient to represent correlations among nodes. Fusing different graph models is challenging: nodes in different graphs are associated with different spatio -temporal information. can not simply merge by using weighted sum or other averaging approaches. Existing multi-graph fusion approaches rely heavily on specific models: assuming specific structures. Henc e, it induces various difficulties in examining the importance of each graph to find a better combination of each module. Long-term spatio -temporal dependency is not considered: Due to limited data source, previous works only considered the static spatial information.

5. INTRODUCTION Propose a heuristic graph represent the long-term spatio -temporal dependency from historical data or human insights and can be widely used for various graph neural networks. Design a novel graph model fusion module to integrate various graph models with graph attention and spatial attention mechanisms a iming to align nodes within graphs and across different graphs . construct a learnable weight tensor for each node to flexibly capture the dynamic correlations between nodes.

6. METHODOLOGY Main Architecture - LSTF 3 main components: Graph construction. ST-GNN module: any SOTA STGNN. Dynamic multi-graph fusion.

7. METHODOLOGY Graph Construction A graph set Include: heuristic graph , functionality graph , distance graph , neighbor graph , and temporal pattern similarity graph .   A distance graph : a thresholded Gaussian kernel   A neighbor graph :  

8. METHODOLOGY Graph Construction A functionality graph : a Pearson correlation coefficients for similarity between functional locations: factories, schools, hospital, etc.   A heuristic graph : a histogram where each bin indicates a predefined temporal range, and the bar height measures the number of data records that fall into each bin. Apply function to obtain fitted parameter and of each node.   K is total number of functions and F i is vector of frequency of these functions in node i .

9. METHODOLOGY Graph Construction A temporal pattern similarity graph : a Pearson correlation coefficients for finding similarity between node over a length time series. vector of time series of node , where is length of series, and is the time-series data value of the node i at time step p.  

10. METHODOLOGY Multi-graph Spatial Embedding (MGSE) and Dynamic Multi-graph Attention Block MGSE: to preserve the graph structure information and represent the relationships of the nodes in different graphs . Employ 2-layer Fully-connected to transform graph into a vector. Fuse two types of embedding by:   Dynamic Multi-graph Attention Block: capture adaptively the correlations among the node in different graphs Spatial Attention block. Graph Attention block.

11. METHODOLOGY ST-ATT: Spatial Attention Capturing the contextual correlations of nodes by proposing a spatial attention mechanism. The next hidden state of graph is ReLU activation function of m-head attention.   Concentrating with MGSE to extract both spatial and graph features. Multi-head attention mechanism:

12. METHODOLOGY ST-ATT: Graph Attention and Gated Fusion employ graph attention to obtain the self-correlations of a node in different graphs . To extract the correlations of nodes on different graphs . the gated fusion method.

13. EXPERIMENT AND RESULT EXPERIMENT SETTINGs Measurement: Mean absolute error (MAE). Root mean square error (RMSE). Dataset: Parking dataset: Melbourne City Council in 2019, contains 42,672,743 parking events recorded by the in-ground sensors. Air Quality datasets: 92 air quality monitoring stations, to assess the hourly PM2.5 concentration in Jiangsu province in 2020. ST-GNN module: STGCN [1], ASTGCN [2], MSTGCN [3], ST-MGCN [4], and Graph WaveNet [5]. [1] Yu, B., Yin, H., & Zhu, Z. (2017). Spatio -temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875. [2] Guo, S., Lin, Y., Feng, N., Song, C., & Wan, H. (2019, July). Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence (Vol. 33, No. 01, pp. 922-929). [3] Guo, S., Lin, Y., Feng, N., Song, C., & Wan, H. (2019, July). Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence (Vol. 33, No. 01, pp. 922-929). [4] Geng , X., Li, Y., Wang, L., Zhang, L., Yang, Q., Ye, J., & Liu, Y. (2019, July). Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. In Proceedings of the AAAI conference on artificial intelligence (Vol. 33, No. 01, pp. 3656-3663). [5] Wu, Z., Pan, S., Long, G., Jiang, J., & Zhang, C. (2019). Graph wavenet for deep spatial-temporal graph modeling. arXiv preprint arXiv:1906.00121.

14. EXPERIMENT AND RESULT RESULT – Overall Perfor mance

15. EXPERIMENT AND RESULT Result: Visualization

16. CONCLUSION Proposed two new graphs to extract heuristic knowledge and contextual information from spatiotemporal data . designed a heuristic graph to capture the long-term pattern of the data. and functional graph similarity between two areas. To align nodes in graphs and timestamps: designed a dynamic graph multi-graph fusion module.