Maryam Sajid Proposal defense and technology.pptx

this is about the improved classification of underwater ship engine using siamese network

1. AN IMPROVED CLASSIFICATION OF UNDERWATER SHIP-ENGINE AUDIOS USING SIAMESE NETWORK Department of Computer Science, Faculty of Engineering & Computer Science, National University of Modern Languages, Islamabad. ---, 2023 MS Research Proposal Presented By: Maryam Sajid Supervisor: Dr. Hina Ashraf

2. Outline Introduction Literature Review Problem Statement Research Questions & Objectives Proposed Solution References 2

3. Introduction 3 In recent years, with the development of science and technology. Vital issue in the field of underwater acoustic signal processing. Underwater acoustic ship-engine classification based on ship radiated noise helps in observing ships at sea, the search for underwater ships, detecting opponent’s ship, marine search and rescue and so on. Underwater environment causes distortion in radiated noise, more accurate underwater acoustic classification methods must be investigated Scholars have put through comprehensive research on ship-engine classification methods and the research mainly emphasis on feature extraction [5]. Inspired by human auditory perception, chromatogram, Mel frequency Cepstral coefficients (MFCC), Mel-Spectrogram are widely used for underwater classification [1-3]

4. Introduction (Cont... ) 4 Data Augmentation Ship-Radiated Noise Classifier Result Feature Extraction audio Mel spectrograms, Chromatogram GAN, up sampling, translational movements Resnet Ships

5. Introduction (Cont... ) 5 Ship-Radiated Noise Feature Extraction Classifier Result

6. Introduction (Cont... ) 6 Motivation Underwater classification of Ship-engine is performed by domain specialists [6]. Long-time work and weather conditions. More accurate underwater acoustic classification methods must be investigated. Deep learning methods are created for assistance of human experts. Example, monitors or tests to assist doctors. Underwater audio dataset is limited, usually expensive or not available due to security [5]. There are two ways to overcome this problem: Data Augmentation Diversify enough the features to accurately identify classes.

7. Introduction (Cont... ) 7 Motivation Previous researchers had randomly divided the underwater audio into; 80% training, 20% testing Vaz, G., Correia, A., Vicente, M., Sousa, J., Cruz, E., & Dommergues, B. Marine Acoustic Signature Recognition Using Convolutional Neural Networks. Available at SSRN 4119910. Class A Class B

8. 8 Literature Review Convolutional Neural Networks [1] The proposed model is based on CNN as the classifier Feature Extraction method: Mel-spectrogram and Mel-spectrogram +1 st derivate +2 nd derivative Results 88.8% Strength Unique feature of human hearing & converts the Hertz into mel scale. Widely used in classification, instrument detection in music etc. Limitation Window based comparison, so biasness in training and testing dataset.( training: 90 , testing:10) Vaz, Guilherme, Alexandre Correia, Miguel Vicente, Joao Sousa, Erica Cruz, and Benedicte Dommergues. "Marine Acoustic Signature Recognition Using Convolutional Neural Networks." Available at SSRN 4119910. (2022) Windowing Scaling

9. 9 Literature Review Vaz, Guilherme, Alexandre Correia, Miguel Vicente, Joao Sousa, Erica Cruz, and Benedicte Dommergues. "Marine Acoustic Signature Recognition Using Convolutional Neural Networks." Available at SSRN 4119910. (2022) Joint convolutional neural network and a long short-term memory network [2] Feature extraction method Results 92.14% Strength LSTM performs better on sequential data MFCC Increases the energy in higher frequency Limitation Window based comparison, so biasness in training and testing dataset. ( training: 90 , testing:10) Extract the mel-spectrogram , MFCCs , c hromatogram, spectral contrast , tonnetz

10. 10 Underwater Acoustic Target Recognition with a Residual Network [6] ResNet as classifier Mel Spectrogram+ Mel Frequency Cepstral Coefficient(MFCC) + Chroma Results 94.3% Strength Extracted features from integration of different feature extraction techniques leads to higher accuracy Limitation Window based comparison, so biasness in training and testing dataset.( training: 90 , testing:10) Hong, Feng, Chengwei Liu, Lijuan Guo, Feng Chen, and Haihong Feng. "Underwater acoustic target recognition with a residual network and the optimized feature extraction method." Applied Sciences 11, no. 4 (2021): 1442. Literature Review

11. 11 Underwater Acoustic Target Recognition Method Based on Spectrograms with Different Resolutions [5] The proposed model is based on CNN as the classifier Multi-window spectral analysis as feature extraction A Generative adversarial network for data augmentation. Results 92.91% (separate audio for testing) 96.32% (random selection) Strength MWSA performs multiple STFT processing on a piece of data to generate multiple spectrograms of the same size with different resolutions, which improves the ability to extract features from the original signal Used 1 audio for training and another audio for testing Vaz, G., Correia, A., Vicente, M., Sousa, J., Cruz, E., & Dommergues, B. Marine Acoustic Signature Recognition Using Convolutional Neural Networks. Available at SSRN 4119910. Literature Review

12. 12 Ref Proposed Model Results Dataset Limitation [1] Feature extraction mel-sepctogram+ 1 st derivative+ 2 nd derivate Classification by CNN 88.8% ShipEars dataset including 11 types of ships, 5 classes Randomly selected data for training and testing which leads to biasness [2] Five Feature extraction techniques Classification using Joint CNN-LSTM 92.14% ShipEars dataset including 11 types of ships, 5 classes [6] Three step feature extraction techniques Classification using ResNet-18 94.6% [5] Multi-window spectral analysis as feature extraction Classification using CNN 92.91% 96.32% Literature Review

13. 13 Ref Proposed Model Results Dataset Limitations [3] Attention-based Neural Network 96% Ship A and Ship B audio recorded in South China Sea, consisting 1254 frames Randomly selected data for training and testing which leads to biasness [4] UATC- DenseNet Model 98.5% 11 UA signals, one with noisy blank signal, consisting 4096 samples Randomly divide the data set into 70% for training and 30% for testing. Literature Review

14. 14 Underwater audio data is limited and not available due to security reasons [5]. The scarcity of data used for classification can be addressed by either data augmentation or by extracting feature that are unique and diverse enough to recognize unseen data [2]. Researchers have used different feature extraction techniques [1-5]. More accurate underwater acoustic classification methods must be investigated [2]. This study aims to Use the integration of multimodal features to analyze the impact on accuracy for Ship-engine classification. Use Siamese Networks to diversify feature extracted for recognition. Problem Statement

15. Research Questions What is the impact of using features from Siamese network on accuracy? What is the impact of using multimodal features along with extracted from Siamese network on accuracy? Research Objectives To propose the modified Underwater acoustic ship-engine classification with Siamese based feature extraction technique. To evaluate the performance of the proposed model with multimodal feature extraction techniques. 15 Research Questions and Objectives

16. Research Methodology 16

17. Proposed Solution 17

18. Dataset 18 90 acoustic recordings from 11 different ship types [7] Time dur ation 1- to 2-minute About 5400 samples after splitting into 2-seconds equal chunks About 4500 will be after preprocessing Santos-Domínguez, David, Soledad Torres- Guijarro , Antonio Cardenal-López, and Antonio Pena-Gimenez. " ShipsEar : An underwater vessel noise database." Applied Acoustics 113 (2016): 64-69.

19. Dataset 19 Category Ship Types Images Details A Tugboats   A tug measures over 80 cm and can easily carry 15-20 people. B Motorboats Used for enjoyment of such sports as fishing, duck hunting, swimming etc. C Passenger ferry transport passengers and/or vehicles across a body of water on a regular, frequent basis D Ro-Ro vessels Ro-Ro vessels are characterized as horizontally loading vessels

20. Data Pre-Processing 20 Vaz, G., Correia, A., Vicente, M., Sousa, J., Cruz, E., & Dommergues, B. Marine Acoustic Signature Recognition Using Convolutional Neural Networks. Available at SSRN 4119910.

21. Validation 21

22. References [1] Vaz, G., Correia, A., Vicente, M., Sousa, J., Cruz, E., & Dommergues, B. Marine Acoustic Signature Recognition Using Convolutional Neural Networks.  Available at SSRN 4119910 . [2] Han, X. C., Ren, C., Wang, L., & Bai, Y. (2022). Underwater acoustic target recognition method based on a joint neural network.  PloS one, 17(4), e0266425. [3] Xiao, X., Wang, W., Ren, Q., Gerstoft , P., & Ma, L. (2021). Underwater acoustic target recognition using attention-based deep neural network. JASA Express Letters, 1(10), 106001. [4] Doan, V. S., Huynh-The, T., & Kim, D. S. (2020). Underwater acoustic target classification based on dense convolutional neural network. IEEE Geoscience and Remote Sensing Letters. [5] Luo, Xinwei , Yulin Feng, and Minghong Zhang. "An underwater acoustic target recognition method based on combined feature with automatic coding and reconstruction." IEEE Access 9 (2021): 63841-63854 [6] Hong, Feng, Chengwei Liu, Lijuan Guo, Feng Chen, and Haihong Feng. "Underwater acoustic target recognition with a residual network and the optimized feature extraction method." Applied Sciences 11, no. 4 (2021): 1442. [7] Santos-Domínguez, David, Soledad Torres- Guijarro , Antonio Cardenal-López, and Antonio Pena-Gimenez. " ShipsEar : An underwater vessel noise database." Applied Acoustics 113 (2016): 64-69. 22

23. Q & A Thankyou 23