250302_HW_LabSeminar[Towards Foundation Models for Knowledge Graph Reasoning].pptx

Towards Foundation Models for Knowledge Graph Reasoning

1. K im Hyun W oo Network Science Lab The Catholic University of Korea E-mail : [email protected] TOWARDS FOUNDATION MODELS FOR KNOWLEDGE GRAPH REASONING

2. Contents Introduction Difficulty on making foundation model in KG Previous work Method ULTRA Experiment Conclusion

3. Knowledge graph Knowledge graph is connection based on entity and relation in the world By the knowledge graph, we can understand how world was organized and will be organized Not only directly, knowledge graph can also do deep inference Introduction

4. Difficulty on making foundation model in KG Modern machine learning applications increasingly rely on the pre training and fine tuning paradigm GPT, BERT . etc Bu t in graph domain, especially KG has difficulty on making foundation model The key problem is that different KGs typically have different entity and relation vocabularies Most of KGs are not transferable Knowledge graph

5. Previous work There are previous works effort to make inductive learning method RED-GNN, GRAIL, NBFNet , NodePiece The models can generalize to graphs with new nodes, but not to new relation types RMPI The model employ graphs of relations to genetalize to unseen domains but have suffer scalability and computational issue as subgraph samling method INGRAM The model use degree but because of that, if graphs don’t have similar relational distribution it failed Knowledge graph

6. Method In this paper. The authors divide relation in graph by 4 fundamental relations Head to head Head to tail Tail to head Tail to tail Then make relation graph that is constructed by the fundamental relation Learning on the relation graph Knowledge graph

7. Experiments Knowledge graph

8. Knowledge graph

9. Conclusion In this paper, the authors propose a model can learn universal and transferable graph representation With pre-training strategy ULTRA achive SOTA performance in several dataset(not all) the authors remain the problem as open question Knowledge graph