Novel Optimized Models for Deep Learning

Need and Objectives for Novel Optimized Deep learning model for Resource Constrained Devices

1. Novel Optimized DL Models for Resource Constrained Devices

2. Need for Research Lots of Deep Learning models are being used in Industries for a variety of tasks such as Object Recognition, Dialogue Modelling Inferencing a deep learning model for a particular tasks requires more computational resources Few industries deploy their deep learning models at cloud by sacrificing the security and privacy requirements In many cases, deploying deep learning models at cloud incurs more costs despite chance for security threats This work proposes a utility-privacy balanced light weight deep learning model which can be deployed at the data acquisition site itself by leveraging the benefits of Edge Computing.

3. Related Works - Survey Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review, January 2023 Data transmission from Edge (Data Acquisition Site) to the cloud results in high latency, round-trip delay, security and privacy concerns, and the inability of real-time decisions. Quantization of the DL Models (both weights and biases) are incorporated to accelerate the performance of DL models at Edge. The amount of Quantization is static and fixed for each pruned light-weight DL models Comparative study on the performance of deep learning implementation in the edge computing, December 2022 Preparing a DL model for production at Edge devices is based on the Hardware-Compiler Codesign ( HCCo ) The model pruning has to be done for every hardware architecture despite the model structure and parameters are one and the same The acceleration of DL inferencing at edge varies across Hardware(Raspberry Pi, Jetson, Intel NCS2) and Software Architectures( TensorRT , Qualcomm Neural processing SDK, CEVA DNN)

4. Related Works FedNets : Federated Learning on Edge Devices Using Ensembles of Pruned Deep Neural Networks, February 2023 FedNets utilises G raph E mbedding theory to reduce the complexity of running Deep Neural Networks (DNNs) on resource-limited devices. Graph Embedding: Instead of sharing the learning parameters for all the layers in the DL network, this work proposed an ensemble learning from diverse light-weight models and perform parameter updation . This work integrates privacy preserving and also utilizes the power of light weight edge models. Updation of model parameters for heterogenous data is not explored in the edge device . A hybrid deep learning framework for privacy preservation in edge computing, June 2023 Deep Learning models incur a huge computation cost A deep learning network based on adversarial training to build a utility-privacy balanced, low computation solution. This work proposed a privacy preserving framework to protect data shared in Edge Computing (EC-IoT) T here is a scope for investigating low-weight adversarial model to avoid the porting into cloud.

5. Identified Research Gap Data collected at the Edge Devices are very much homogenous in nature – All Models currently available are for Homogenous data, New Models are to be investigated for Heterogeneous data The data representation in Backpropagation of the Network is in either in FP16 or INT8 – Automatic Mixed Arithmetic can be studied for model compression Mapping of Light-weight multiple ensemble DL models in a single Edge device is not explored – This can be further studied As a conclusion, DL Inference Accelerators are highly dependent on the Hardware and Software Platform

6. Conclusion A Novel optimized DL model is required for accelerating the Inference at Edge Devices (Edge Inference) The model should be Independent of the Hardware and Software Architectures and handle heterogeneous data. The model should perform model pruning(removing redundant layers) and quantization

7. Use cases for Edge AI Inference A wearable sunglass capable of running multiple DL models on edge for image recognition tasks such as CBIR, Visually Impaired Assistance 3D Object recognition at Edge for Autonomous driving vehicles Edge Inferencing with Heterogenous Sensor and Image data

8. Thank You