Failure Prediction for APU on a Metro System

This contains the slides for the presentation given on failure prediction at pune university's Smart India Hackathon and other conferences, This was presented to various tech professionals and esteemed academics as well as members of government bodies.

This presentation was also used while presenting my thesis on predictive maintenance.

1. Failure prediction for APU's on a Metro System Aaryadev Ghosalkar

2. Agenda Introduction Problem Statement Literature Review Existing Systems Modelling Results Conclusion Future Scope Failure prediction for APU's on a Metro System 2

3. Introduction Air production unit (APU) is component on most modern metro systems which circulates compressed air through the metro, our research presents a comprehensive comparison of PdM models on APU failure detection. Our goal is to detect failures at least 2 hours in advance, the challenge with APU's in particular is that APU failures are rare events In the dataset there are 3 failure within 6 months of operation  Failure prediction for APU's on a Metro System 3

4. Problem Statement Detect failures at least 2 hours in advance Create a system that is customizable can expand to more than 2 hours if required (also keeping in mind the side effects that may arise) Provide a way to deploy the model for real time inference Failure prediction for APU's on a Metro System 4

5. Literature Review Taking a look at what other researchers have done

6. Literature Review Veloso et al performed the initial data collection, which involved converting data from the metro company database into a CSV file We used some of the ideas for data preprocessing presented by the AzureML team at microsoft A study on Davari et al motivated many of the algorithms that we used in this study, there work has been a great resource in our study Failure prediction for APU's on a Metro System 6

7. Literature Review continued Chaudhuri et al used SVM to classify vehicles into 3 distinct risk levels, focusing on model interpretability, model interpretability refers to how easy it is to understand the choices made by the model and results of the model Various other researchers have used CNN with GAF to convert timeseries data into images and used CNNs for classification most notable was done by Silva which reported an accuracy of 93% in their study Failure prediction for APU's on a Metro System 7

8. Literature Review continued Nguyen and Medjaher used LSTM to predicted the probability of failure in a given time window, we drew a lot of inspiration from their work in terms of balancing the dataset and the model used In terms of early research into RUL estimation a majority of the work revolves around using statistical models or models which assume linear degradation pattern, it was only after 2016 Deep learning models were used in this field  Failure prediction for APU's on a Metro System 8

9. Existing Systems Predictive maintenance has been used in a lot of domains such as elevators and Jet engines.

10. The Infrastructure abroad Uses NLP on text fields where engineers describe the problem and how the problem was solved.  Also predict if a failure is about to happen and which component will fail TFL expects to save  £3 million a year Failure prediction for APU's on a Metro System 10 TrainDNA collects and analyzes real-time data from more than 200 trains across Australia using IBM Maximo 51% Increase in realiablity after introducing the TrainDNA system Considers multiple failures Large scale system able to process 30 Million message every hour London Australia and NZL

11. Modelling Taking a look at how the data was processed and what ML models were tested Failure prediction for APU's on a Metro System 11

12. Data Preprocessing Most of the data collection and cleaning work on this data set was done by researchers that created the data set Thus this data set did not have any null values Failure prediction for APU's on a Metro System Linear Model are a bad idea There is no obvious pattern which we can see to distinguish each class, which further motivated use of deep learning Balance is key! Collection EDA Discretizing the data to make this more suitable for classification Generalizing the model using a parameter to control number of hours before warning. Preparation 12

13. Challenges in data processing The sheer volume of data presented a significant challenge, there were more than 10 000 000 (1Cr) rows in the dataset to solve this we performed carefully changed the data types of the features to minimize the loss of information this allowed us to significantly reduce size of the data with a 91% decrease in storage space and 77% decrease in RAM usage Failure prediction for APU's on a Metro System 13

14. Challenges in Data processing continued The highly imbalanced nature of the data also presented a huge challenge when training the models, since APU failures are a rare event and only 3% of the data constitutes of APU failure a model that predicts all data points as normal would mathematically have an accuracy of 97%, to combat this we used Near miss under sampling and a convenience sampling like strategy for the LSTM, inspired by Chen et al  Failure prediction for APU's on a Metro System 14

15. Machine Learning Models SVMs take very long to train Results are not satisfactory and most other models perform better Failure prediction for APU's on a Metro System Very quick training time The hyper parameters such as the criterion do not make any difference to the accuracy  SVM Decision Trees Provide greater accuracy than decision trees due to this being an ensemble model Training time is similar to decision trees Random Forests 15

16. Deep Learning Models Used a small neural network of 25k parameters with AdamW optimizer Best results from all tested model Does not take into account the time series nature of the data Requires CUDA for efficient deployment. Failure prediction for APU's on a Metro System 16 Can incorporate temporal patterns Large model almost 184K parameters which makes training difficult. Requires 3rd order tensors as input for training thus balancing data is hard Also requires CUDA for efficient training and deployment Neural Network LSTM Network

17. Results Accuracy Precision (Class 1)  Recall (Class 1) SVM 0.57 0.60 0.62 Decision Tree 0.66 0.69 0.70 Random Forest 0.70 0.69 0.78 Neural Network (Adam) 0.72 0.67 0.91 Neural Network ( AdamW ) 0.76 0.75 0.87 LSTM Network 0.76 0.64 0.85 Failure prediction for APU's on a Metro System 17

18. Conclusion Neural networks produce satisfactory results however the Random forest model can be used when deploying on low end hardware or considering an edge solution. LSTMs perform nearly identical to neural networks with less data Microservices can be considered when deploying as this will allow different parts of the model to be deployed separately Multi collinearity is present in the data which can makes it hard to fit any kind of linear model Failure prediction for APU's on a Metro System 18

19. Future Scope Transformer based architecture can be considered as they have shown promising results with sequential data in the case of NLP applications Perhaps due to the imbalanced nature and the rarity of APU failures on a real metro systems an anomaly detection approach would be better as this would not need the data to be balanced and a lot more of the existing data can be used Failure prediction for APU's on a Metro System 19

20. Question and Answer Aaryadev Ghosalkar [email protected] https://github.com/aaryadevg