[NS][Lab_Seminar_251013]Multimodal Cancer Survival Analysis via Hypergraph Learning with Cross-Modality Rebalance.pptx

Multimodal Cancer Survival Analysis via Hypergraph Learning with Cross-Modality Rebalance

1. Multimodal Cancer Survival Analysis via Hypergraph Learning with Cross-Modality Rebalance Tien-Bach-Thanh Do Network Science Lab Dept. of Artificial Intelligence The Catholic University of Korea E-mail: os fa19730 @catholic.ac.kr 2025/10/13 Mingcheng Qu et al. IJCAI 2025

2. Introduction Survival prediction is forecasting events of cancer progression and mortality in prognostic patients Multimodal analysis has gained increasing prominence, exemplified by integration of whole slide images (WSIs) and genomic profiles in clinical oncology WSIs with giga-level resolution are segmented into multiple smaller patches Multi-instance learning is employed to aggregate patch-level features, assuming a bag is positive if it contains at least one positive instance It distinguishes tumor from non-tumor regions, but remain suboptimal for survival analysis and fail to capture crucial contextual and hierarchical information required for precise prognostic assessment, leading information loss Integrating information from different modalities is a key research direction Employ late fusion by combining 2 modalities, but overlook interconnections between genetic and pathologic data Explore early or mid fusion strategies like leveraging cross-attention Still struggle to address the modality imbalance

3. Introduction Figure 1: Left: Compared to MIL, hypergraph learning activates more patch regions and better captures contextual and hierarchical details. Right: examples reveal the pathology-genomics imbalance, where pathology features dominate the overall survival prediction

4. Introduction Proposed MRePath (Multi-Modal Rebalance for Pathology-Genomic Survival Prediction) to leverage hypergraph learning and modality rebalance for survival analysis Introduce sheaf hypergraphs to promote information exchange between nodes and hyperedges, preserving contextual details within patches and hierarchical relationships Employ dynamic weighting that first assess the mono-confidence of each modality's reliability and calculates holo-confidence by interactions

5. Related Works Survival analysis Rely on MIL to aggregate patch-level features Extract global features by embedding, attention weights, graph-based modeling Multimodal approaches with genomic data for survival analysis They focused on late fusion, like vector concatenation, modality-level alignment, and bilinear pooling Graph-based pathology analysis Capture intricate relationships in pathology-related tasks Early studies represented patches as nodes, constructed adjacency-based or fully-connected graphs

6. Proposed Method Figure 2: Overview of MRePath

7. Methods Overview - Preliminary Given                                          is cohort of n subjects, each subject represented as tuple                                 denote pair of pathology-genomics features P i represent WSI G i represent genomic profile                          represent label of i-th subject, with event status  Survival prediction is estimating hazard function              , predict instantaneous incidence rate of interest event at specific time point t Instead of estimating patient's survival time, they train model F to predict probability that patient's survival exceed t using survival function

8. Methods Overview MRePath 3 stages: feature extraction, hypergraph learning, modality rebalance P, G are extracted from paired pathology and genomics data Pathology feature extraction WSI into multiple 256*256 pixel patches at 20X magnification ResNet50 is used to extract d-dimensional features WSI represented Genomic feature extraction Selected genes are categorized into 6 functional groups: Tumor Suppression Oncogenesis Protein Kinases Cellular Differentiation Transcription

9. Methods Hypergraph Learning Construction Hypergraph Hyperedges are generated using 2 complementary: Topological-based: formed by grouping each patch with its neighboring patches based on Euclidean distance  Feature-based: created based on similarity between patch features

10. Methods Hypergraph Learning Sheaf hypergraph Pathology feature P Sheaf Laplacian: Create information flow space for nodes and hyperedges, capture both local contextual features witnin individual patches and global hierarchical relationships

11. Methods Modality Rebalance Dynamic weighting Regulate contributions of pathologic and genomic by w p and w g 2 confidence measures: Mono-confidence: assess reliability of each modality by estimating probability of true class label Holo-confidence: extend mono-confidence by incorporating cross-modal interactions Final weights through linear operation

12. Methods Modality Rebalance Interactive alignment fusion Modality-specific co-attention mechanism to refine feature representations Enhance selection of pathological features based on genomic information, gene-guided co-attention layer  Genomic feature

13. Experiments Datasets Bladder Urothelial Carcinoma (BLCA) (n=384) Breast Invasive Carcinoma (BRCA) (n=968) Colon and Rectum Adenocarcinoma (CO-READ) (n=298) Head and Neck Squamous Cell Carcinoma (HNSC) (n=392) Stomach Adenocarcinoma (STAD) (n=317) Metrics C-Index Implementation details Adam optimizer LR = 1*10 -4 Weight decay 1*10 -5 30 epochs

14. Experiments Table 1: Comparison of MRePath  with 5 datasets

15. Experiments Table 2: Ablation study on hypergraph learning

16. Experiments Figure 3: Ablation study on hyperedge construction threshold k

17. Experiments Figure 4: Visualization for low and high-risk in BRCA (top) and Kaplan-Meier curves (Bottom)

18. Experiments Table 3: Ablation study on modality rebalance Table 4: Ablation study on pathology feature encoder

19. Conclusion Propose a multimodal framework MRePath for cancer survival prediction Address MIL-based information loss by using sheaf hypergraph and pathology-genomics imbalance by employing a dynamic rebalance Limitations: Consider WSI with varying numbers of patches, same k-value in hypergraph leads to different scopes, imposing a potential incosistency in pathology slides Challenges to obtain complete paired pathology and genomic data In some cases, low-quality data or missing modalities can significantly imact modality rebalancing process, need for greater robustness