Deep Learning-based Application Development for Automated Cephalometric Analysis

deep learning-based application development for automated cephalometric analysis

1. 1 Ref: https://www.linkedin.com/pulse/what-artificial-intelligence-definition-uses-types-aishwarya-v-lyoac

2. 2 AI stands for Artificial Intelligence, which is the ability of machines to perform tasks that typically require human intelligence like: Learning: AI can learn from experience and improve its performance. Problem-solving: AI can solve problems and act to achieve a specific goal. Decision-making: AI can make recommendations and decisions. Creativity: AI can write poems and create original images and text. Communication: AI can understand and translate spoken and written language. Data analysis: AI can analyze data and make data-based predictions

3. Applications Of Artificial Intelligence In Medicine 05 04 03 02 01 06 07 08 09 10 Disease diagnosis Accurate illness identification through symptom analysis and data. Drug discovery Speeds research, predicts properties, finds potential treatments. Medical imaging analysis Improves diagnostics, detects anomalies in scans effectively. Personalized treatment plans Tailors therapies, utilizes patient data, genetic information. Virtual health assistants Provides care, schedules, offers medical guidance remotely. Medical research Speeds discoveries, analyzes data, informs treatments efficiently. Predictive analytics Forecasts outcomes, identifies risks, optimizes care plans effectively. Health informatics Manages records, analyzes data, enhances healthcare delivery. Precision surgery Guides surgeons, boosts accuracy, minimizes invasiveness ,. Remote monitoring systems Tracks health data, alerts changes, enhances remote care.

4. Deep Learning-based Application Development for Automated Cephalometric Analysis Dr. Khandaker Reajul Islam MS (Orthodontics), BSMMU, Dhaka MSc (Physiology), UKM, Malaysia Lecturer Dhaka Dental College, Mirpur, Dhaka 4

5. Introduction Cephalometric analysis is a vital and mandatory part of orthodontic diagnosis and treatment planning and needs precise identification of landmarks. (Dinesh et al., 2020) Manual cephalometric analysis : 5 Time-consuming Prone to human error Inconsistent. (Song et al., 2022) Photo Curtesy: Department of Orthodontics, BSMMU

6. 6 Deep learning Artificial intelligence (AI): The proposed Deep learning framework, Self- CephaloNet , leverages high-resolution representation learning and incorporates a self-attention mechanism for enhanced accuracy in cephalometric landmark localization. Automates landmark detection Ensure more accuracy and consistency Introduction

7. Rationale 7 Currently, no study exists on AI-based Application development for Automated Cephalometric Analysis in Bangladesh. As Cephalometric analysis is crucial in orthodontic diagnosis, and traditionally it is subjective and time-consuming. The study created an AI based application for automated landmark detection and computation, which will significantly, aiding the field of Orthodontics in Bangladesh by precise diagnosis and appropriate treatment planning.

8. General Objective : To develop a deep learning-based application for automatic cephalometric analysis and evaluate its performance on internal and external validation sets. Specific Objectives : To develop a novel deep learning (DL) framework for automatic detection of anatomical landmarks within the lateral cephalogram. Then to extract patch as a region of interest (ROI) based on the initial landmarks. Then to develop a second refinement DL model for improving the accuracy and precision of landmark detection. And To evaluate the performance of the two-stage framework will be carried out on the internal and external validation sets. And lastly, To create a local (private)/webpage based (public) application for performing automatic cephalometric analysis. Objectives 8

9. Database Description: ISBI 2015 Challenge Data 9 Publicly available International Symposium on Biomedical Imaging (ISBI) 2015 challenge dataset Sample: Cephalometric images of 400 patients Tasks: Landmark detection and Clinical diagnosis using a 2 mm precision range. Ref: https://www.nature.com/articles/srep33581/figures/1

10. For each image, the resolution of image is 1935 by 2400 pixels in Tag Image File Format (TIFF) format. Database Description 10 Ref: https://www.researchgate.net/figure/Cephalometric-landmarks-used-in-the-cephalometric-analysis-S-Sella-turcica-N-nasion_fig1_276044508

11. Database for External Validation: PKU Cephalometric Dataset 11 Publicly available Peking University (PKU) Cephalometric Dataset 2021 Sample: Cephalograms of 102 patients Tasks: Landmark detection and Clinical diagnosis using a 2 mm precision range. Ref: https://www.nature.com/articles/srep33581/figures/1

12. Materials And Methods Type of study: It was a cross-sectional study. Place of the study : Department of orthodontics BSMMU, Dhaka. Study Duration : This study was conducted from April 2023 to March 2024. Study population: Participant of the ISBI 2015 challenge database, comprises 400 cephalometric x-rays of individuals. Study sample: Cephalograms of the dataset. Sampling technique: According to ISBI 2015 challenge, specific 150 images are used to train the model, and the rest of the data are divided in test set 1 (150) and test set 2 (100) to validate the model. Sample size: Total sample size was 400 . 12

13. Machine Learning And Deep-Learning Techniques Ref.- https://www.dynam.ai/wp-content/uploads/2020/06/Screen-Shot-2020-06-26-at-11.04.18-AM-e1593376707667.png Machine learning (ML) is a type of artificial intelligence (AI) that enables computer programs to improve their predictive abilities without requiring explicit instructions on how to do so. 13

14. A wide range of classical machine learning models were employed. Such as: L ogistic regression (LR), S upport vector machine (SVM), XGBoost Random Forest (RF) K-nearest Neighbors (KNN) Ada Boost, etc. A small amount of Deep learning techniques were also used, which includes: Convolutional Neural Network (CNN) Yolo ( You only look once) variants HR(High Resolution)Net variants etc. Classical ML and Deep Learning Algorithms 14

15. Facial landmark detection Models Deep learning based anatomical landmark detection model. How Anatomical Landmark Can Be Detected? 15 Ref: https://freight.cargo.site/t/original/i/06a8cc9ead42d1523753ceb788adf25391bd77a19289345800955b13898c5e8e/1_kmu_t1iNbOipwozjxCG6og.jpeg

16. A ll the anatomical landmarks, angels, ratios, extracted from the X-ray images. They are: Cephalometric Analysis ANB angle SNA angle SNB angle Overbite depth indicator (ODI) Anteroposterior dysplasia indicator (APDI) Facial height index(FHI) Frankfurt mandibular angle (FMA) Modified Wits Appraisal (MW) 16 Ref: https://www.nature.com/articles/srep33581/figures/1

17. Normalized Mean Error (NME) : It is the average deviation from the true placements of landmarks ( gi ) as predicted by the algorithm (pi), scaled by the inter-ocular distance (d). The NME formula is defined as follows:   Performance Metrics 17 Mean Radial Error (MRE ): Mean Radial Error (MRE) is a metric used to measure the accuracy of landmark detection in image j for a specific landmark i . It is calculated by taking the Euclidean distance between the estimated landmark coordinates and the manually annotated landmark coordinates . This is represented by the radial error (RE) formula: The MRE for a specific landmark is then calculated by averaging the radial error values across all images j :   The SD for landmark is determined using the formula:  

18. Performance Metrics (cont.) Success Detection Rate (SDR): The Success Detection Rate (SDR) is the fraction of landmarks correctly detected, relative to the total number of predictions, for which the radial error between the predicted and ground truth landmark positions is less than or equal to a threshold δ: Here, represents the count of predicted landmark positions that satisfy the condition of having a radial error less than or equal to nd denotes the cardinality of the set , which represents all the predictions made across all images.   18

19. Study Design with DL Models 19 Cropped according to the center location and resized to 512x512 Predicted heatmap after 2 nd stage Input Image Cropped according the center Location and resized to 256x256 Predicted heatmap Patch Images Extraction 1 st Stage SelfCephaloNet Model SelfCephaloNet Model 2 nd Stage

20. Study Design with DL Models 20

21. Results Success detection rate (SDR)% Test1 Dataset Test2 Dataset Landmarks MRE ± SD <2.0mm < 2.5mm < 3.0mm < 4.0mm MRE ± SD < 2.0mm <2.5mm <3.0mm <4.0mm L1 1.26 ± 0.84 88.00% 94.67% 97.33% 98.00% 1.19 ± 0.65 86.00% 98.00% 99.00% 100.00% L2 1.63 ± 1.04 74.00% 84.67% 90.67% 95.33% 1.42 ± 0.86 84.00% 90.00% 94.00% 99.00% L3 1.64 ± 0.90 69.33% 85.33% 95.33% 98.00% 2.32 ± 1.19 40.00% 59.00% 74.00% 92.00% L4 2.22 ± 1.12 47.33% 58.67% 78.00% 93.33% 2.10 ± 1.74 60.00% 74.00% 83.00% 89.00% L5 1.95 ± 1.14 56.00% 73.33% 82.67% 94.67% 1.62 ± 0.74 71.00% 89.00% 96.00% 99.00% L6 1.32 ± 0.71 82.67% 92.00% 98.67% 100.00% 2.88 ± 1.36 30.00% 39.00% 48.00% 80.00% L7 1.33 ± 0.82 80.67% 90.00% 96.00% 100.00% 1.04 ± 0.69 92.00% 97.00% 99.00% 99.00% L8 1.16 ± 0.76 92.67% 96.00% 98.00% 98.67% 1.06 ± 0.55 98.00% 99.00% 100.00% 100.00% L9 1.08 ± 0.64 90.00% 97.33% 98.67% 100.00% 0.94 ± 0.46 96.00% 100.00% 100.00% 100.00% L10 2.11 ± 1.27 53.33% 67.33% 75.33% 92.67% 1.66 ± 0.99 66.00% 77.00% 86.00% 100.00% L11 1.37 ± 0.74 82.00% 92.00% 96.00% 100.00% 1.51 ± 0.84 74.00% 88.00% 96.00% 99.00% L12 1.35 ± 0.77 85.33% 93.33% 96.00% 98.67% 1.35 ± 0.80 85.00% 90.00% 95.00% 99.00% L13 1.29 ± 0.60 87.33% 96.00% 100.00% 100.00% 2.44 ± 0.91 30.00% 53.00% 74.00% 94.00% L14 1.12 ± 0.60 90.67% 97.33% 98.67% 100.00% 1.77 ± 0.91 67.00% 81.00% 86.00% 98.00% L15 1.52 ± 0.85 73.33% 87.33% 94.00% 100.00% 1.32 ± 0.68 88.00% 92.00% 97.00% 100.00% L16 1.40 ± 0.85 82.00% 90.67% 94.00% 97.33% 4.28 ± 1.45 5.00% 10.00% 21.00% 45.00% L17 1.36 ± 0.78 80.00% 94.00% 98.67% 99.33% 1.68 ± 0.76 73.00% 87.00% 91.00% 100.00% L18 1.87 ± 1.14 60.00% 74.00% 86.00% 96.67% 1.84 ± 0.94 62.00% 78.00% 87.00% 97.00% L19 2.05 ± 1.24 51.33% 71.33% 81.33% 92.00% 1.81 ± 1.07 63.00% 75.00% 85.00% 96.00% Average 1.53 ± 0.89 75.05% 86.07% 92.39% 97.61% 1.80 ± 0.93 66.84% 77.68% 84.79% 94.00% Table 1: Individual landmark localization results(1 st Stage) employing SDR%, MRE, and SD for test1 and test2 21

22. SDR% Method 2.0 mm 2.5 mm 3.0 mm 4.0 mm Ours 70.94 81.87 88.59 95.8 SCN (Payer et al., 2019) 73.33 78.76 83.24 89.75 Localization U-Net (2015) 72.15 77.83 82.04 88.8 Arık et al. (2017) ( Arık et al., 2017) 72.29 78.21 82.24 86.8 Urschler et al. (2018) ( Urschler et al., 2018) 70.21 76.95 82.08 89.01 Lindner and Cootes (2015) (Lindner and Cootes , 2015) 70.65 76.93 82.17 89.85 Ibragimov et al. (2015) (Ibragimov, 2014) 68.13 74.63 79.77 86.87 Table 2: Comparison of the 1st stage mean results of SDR% of some recent methods. Results 22

23. Success detection rate (SDR)% Test1 Dataset Test2 Dataset Landmarks MRE ± SD <2.0mm < 2.5mm < 3.0mm < 4.0mm MRE ± SD < 2.0mm <2.5mm <3.0mm <4.0mm L1 0.60 ± 0.97 98.00% 98.67% 98.67% 99.33% 0.49 ± 0.29 100.00% 100.00% 100.00% 100.00% L2 1.18 ± 1.91 90.00% 93.33% 94.67% 96.00% 0.89 ± 0.79 91.00% 94.00% 97.00% 99.00% L3 1.31 ± 0.83 82.00% 88.00% 95.33% 99.33% 2.10 ± 1.16 56.00% 70.00% 81.00% 97.00% L4 1.73 ± 1.35 68.67% 73.33% 82.00% 92.67% 1.62 ± 1.81 73.00% 82.00% 84.00% 92.00% L5 1.69 ± 1.06 68.67% 77.33% 86.67% 97.33% 1.45 ± 0.93 76.00% 85.00% 92.00% 98.00% L6 1.26 ± 0.86 84.00% 92.67% 96.67% 99.33% 2.89 ± 1.55 31.00% 45.00% 58.00% 79.00% L7 0.79 ± 0.57 94.67% 99.33% 100.00% 100.00% 0.71 ± 0.81 98.00% 99.00% 99.00% 99.00% L8 0.86 ± 0.63 94.67% 98.00% 99.33% 99.33% 0.71 ± 0.43 98.00% 100.00% 100.00% 100.00% L9 0.66 ± 0.45 99.33% 100.00% 100.00% 100.00% 0.55 ± 0.39 100.00% 100.00% 100.00% 100.00% L10 1.91 ± 1.33 63.33% 77.33% 83.33% 92.67% 1.71 ± 1.27 71.00% 83.00% 88.00% 94.00% L11 0.68 ± 0.78 94.00% 95.33% 97.33% 98.67% 0.71 ± 0.69 96.00% 97.00% 99.00% 99.00% L12 0.50 ± 0.51 97.33% 98.67% 98.67% 100.00% 1.09 ± 1.03 87.00% 93.00% 97.00% 98.00% L13 1.01 ± 0.55 96.67% 98.67% 98.67% 99.33% 2.31 ± 0.57 30.00% 67.00% 89.00% 100.00% L14 0.75 ± 0.41 99.33% 99.33% 100.00% 100.00% 1.48 ± 0.53 86.00% 96.00% 99.00% 100.00% L15 0.77 ± 0.66 96.67% 97.33% 97.33% 99.33% 1.07 ± 0.71 92.00% 94.00% 98.00% 99.00% L16 1.12 ± 0.89 92.00% 96.67% 96.67% 98.67% 4.13 ± 1.33 4.00% 9.00% 19.00% 48.00% L17 0.77 ± 0.67 94.67% 96.67% 98.00% 99.33% 1.01 ± 0.60 93.00% 98.00% 99.00% 100.00% L18 1.30 ± 1.27 82.67% 88.67% 93.33% 96.00% 1.06 ± 0.70 91.00% 96.00% 98.00% 99.00% L19 1.62 ± 1.40 72.00% 81.33% 89.33% 93.33% 1.33 ± 1.49 84.00% 91.00% 93.00% 96.00% Average 1.08 ± 0.90 87.82% 92.14% 95.05% 97.93% 1.44 ± 0.90 76.68% 84.16% 88.95% 94.58% Table 3: Individual landmark localization results(2 nd Stage) employing success detection rate (SDR), mean radial error (MRE), and standard deviation (SD) for test1 and test2 Results 23

24. SDR % Method 2.0 mm 2.5 mm 3.0 mm 4.0 mm Ours 82.25 88.15 92 96.25 (Oh et al., 2020) 82.08 88.06 92.34 96.92 (Zeng et al., 2021) 76.82 84.97 90 95.58 SCN Pay(Payer et al., 2019) 73.33 78.76 83.24 89.75 Localization U-Net (2015) 72.15 77.83 82.04 88.8 Arık et al. (2017) ( Arık et al., 2017) 72.29 78.21 82.24 86.8 Urschler et al. (2018)) (Urschler et al., 2018) 70.21 76.95 82.08 89.01 (Lindner and Cootes, 2015) 70.65 76.93 82.17 89.85 Ibragimov et al. (2015a) ( Ibragimov , 2014) 68.13 74.63 79.77 86.87 Table 4: Comparison of the 2nd stage mean results of SDR% to some recent final stage results. Results 24

25. Test1 Dataset Classes Ours Oh et al. (Oh et al., 2020) Zeng et al. (Zeng et al., 2021) Arık et al. ( Arık et al., 2017) Ibragimov et al. ( Ibragimov ., 2014) ANB 89.44 78.8 78.84 61.47 59.42 SNB 88.09 83.92 81.46 70.11 71.09 SNA 78.23 66.33 71.08 63.57 59 ODI 84.87 83.34 84.88 75.04 78.04 APDI 87.14 84.01 86.22 82.38 80.16 FHI 90.16 74.3 90.32 65.92 58.97 FMA 84.55 79.62 85.11 73.9 77.03 MW 90.18 91.1 84.19 81.31 83.94 Average 86.75 80.18 82.76 71.71 70.84 Table 5: Comparison of classification success rate (percent) for anatomical type classification on Test 1 Datasets. Results 25

26. Test2 Dataset Classes Ours Oh et al. (Oh et al., 2020) Zeng et al. (Zeng et al., 2021) Arık et al. ( Arık et al., 2017) Ibragimov et al. (Ibragimov, 2014) ANB 87.3 84.05 82.06 77.31 76.64 SNB 85.26 87.2 89.69 69.81 75.24 SNA 83.52 72.96 64.75 66.72 70.24 ODI 78.2 72.52 71.47 72.28 63.71 APDI 88.76 89.37 88.9 87.18 79.93 FHI 79.58 94.75 71.86 69.16 86.74 FMA 87.42 83.03 83.5 78.01 78.9 MW 80.89 82.67 81.9 77.45 77.53 Average 83.87 83.94 79.27 74.74 76.12 Table 6: Comparison of classification success rate (percent) for anatomical type classification on Test 2 Datasets Results 26

27. SDR% MRE ± SD 2 mm 2.5 mm 3 mm 4 mm 2nd stage 1.26 ± 0.90 82.25 88.15 92 96.25 1st stage 1.66 ± 0.90 70.94 81.87 88.59 95.8 Table 7 : First and second stage MRE comparison with SD and SDR Impact of individual stages: Results 27

28. Grad-CAM Visualization 28

29. External Validation We conducted an additional assessment involving external validation using the PKU cephalogram dataset, initially introduced by Zeng et al. (Zeng et al., 2021) SDR% MRE ± SD 2 mm 2.5 mm 3 mm 4 mm Ours 2.79 ± 1.87 75.95 83.49 88.49 91.8 Zeng et.al (Zeng et al., 2021) 2.02 ± 1.89 64.88 73.84 81.73 89.78 Table 8: Results of the Success Detection Rate (SDR) comparison on the PKU cephalometric landmark dataset. 29

30. CephaloNet (BSMMU Server) Doctor’s Portal Cephalometric Image Acquisition System X-ray Operator X-ray Images X-ray Images Results Results X-ray Images Web-Interface 30

31. Our Active Web Page Active link: https://orthoai.netlify.app/ 31

32. 32 Demonstration of Ortho AI

33. Generated Result by Our Apps 33

34. Study showcases significant progress, surpassing recent studies and previous benchmarks across precision ranges and suggesting clinical application potential. Despite a slight performance decrease for specific types in the Test2 dataset, our model exhibits high accuracy rates in classifying anatomical types, surpassing alternative methods. Our model demonstrates a notable enhancement in SDR at 2 mm precision and a reduction in the MRE, ensuring improved accuracy and precision in landmark detection. GradCAM visualization reveals the effectiveness of CNNs in cephalometric landmark detection. External validation demonstrates the robustness and adaptability of our trained model. Consistently high accuracy and superior performance metrics make our model a promising tool, for accurate and reliable cephalometric analyses in clinical settings. Discussion And Conclusion 34

35. Self- CepahloNet exhibits potential for cephalometric landmark detection but faces challenges, such as: Relying on a single dataset limits generalizability. Adaptability to new tasks is uncertain. Exploration of additional metrics is required. Further model refinement with a diverse dataset is necessary for real-world applicability. Limitations 35

36. To enhance Self- CepahloNet's robustness and accuracy, we need to do the following: Diversify training data and integrate external datasets. Continuously refine it based on clinical feedback and DL algorithm advancements. Validate its clinical relevance through collaboration with orthodontic professionals. Recommendations 36

37. REFERENCES Dinesh, A., Mutalik , S., Feldman, J. and Tadinada , A., 2020. Value-addition of lateral cephalometric radiographs in orthodontic diagnosis and treatment planning. The Angle Orthodontist , 90 (5), pp.665-671. Song, Y., Ren, S., Lu, Y., Fu, X. and Wong, K.K., 2022. Deep learning-based automatic segmentation of images in cardiac radiography: a promising challenge. Computer Methods and Programs in Biomedicine , 220 , p.106821 Payer, C., Štern , D., Bischof, H. and Urschler , M., 2019. Integrating spatial configuration into heatmap regression based CNNs for landmark localization. Medical image analysis , 54 , pp.207-219. Arık , S.Ö., Ibragimov , B. and Xing, L., 2017. Fully automated quantitative cephalometry using convolutional neural networks. Journal of Medical Imaging , 4 (1), pp.014501-014501 Urschler , M., Ebner, T. and Štern , D., 2018. Integrating geometric configuration and appearance information into a unified framework for anatomical landmark localization. Medical image analysis , 43 , pp.23-36. Lindner, C. and Cootes , T.F., 2015, April. Fully automatic cephalometric evaluation using random forest regression-voting. In IEEE International Symposium on Biomedical Imaging (Vol. 13). Citeseer . Ibragimov , B., Likar , B., Pernus , F. and Vrtovec , T., 2014, April. Automatic cephalometric X-ray landmark detection by applying game theory and random forests. In Proc. ISBI Int. Symp . on Biomedical Imaging (pp. 1-8). © Springer‐Verlag Berlin Heidelberg 2014. Oh, K., Oh, I.S. and Lee, D.W., 2020. Deep anatomical context feature learning for cephalometric landmark detection. IEEE Journal of Biomedical and Health Informatics , 25 (3), pp.806-817. Zeng, M., Yan, Z., Liu, S., Zhou, Y. and Qiu, L., 2021. Cascaded convolutional networks for automatic cephalometric landmark detection. Medical image analysis , 68 , p.101904. Mahto, Ravi Kumar, Dashrath Kafle , Abhishek Giri, Sanjeev Luintel , and Arjun Karki. "Evaluation of fully automated cephalometric measurements obtained from web-based artificial intelligence driven platform." BMC Oral Health 22, no. 1 (2022): 1-8. Zeng, Minmin , Zhenlei Yan, Shuai Liu, Yanheng Zhou, and Lixin Qiu. "Cascaded convolutional networks for automatic cephalometric landmark detection." Medical Image Analysis 68 (2021): 101904. Yu, H. J., S. R. Cho, M. J. Kim, W. H. Kim, J. W. Kim, and J. Choi. "Automated skeletal classification with lateral cephalometry based on artificial intelligence." Journal of dental research 99, no. 3 (2020): 249-256. Wang, Ching-Wei, Cheng-Ta Huang, Jia-Hong Lee, Chung- Hsing Li, Sheng-Wei Chang, Ming- Jhih Siao , Tat-Ming Lai et al. "A benchmark for comparison of dental radiography analysis algorithms." Medical image analysis 31 (2016): 63-76. Lindner, Claudia, and Tim F. Cootes . "Fully automatic cephalometric evaluation using random forest regression-voting." In IEEE International Symposium on Biomedical Imaging (ISBI) 2015–Grand Challenges in Dental X-ray Image Analysis–Automated Detection and Analysis for Diagnosis in Cephalometric X-ray Image. 2015. 37

38. Thank You 38