Face recognition based heart beat prediction

The project presentation ppt helps final year students

1. Geetha Shishu Shikshana Sangha(R) GSSS INSTITUTE OF ENGINEERING & TECHNOLOGY FOR WOMEN (Affiliated to VTU, Belagavi, Approved by AICTE, New Delhi & Govt. of Karnataka) K.R.S Road, Metagalli , Mysuru-570016 Accredited with ‘A’ grade by NAAC DEPARTMENT OF INFORMATION SCIENCE AND ENGINEERING Accredited by NBA, New Delhi, (Validity: 01.07.2023 to 30.06.2026 ) MINI PROJECT SYNOPSIS [21ISMP67] ON “Face Recognition based heart rate prediction” 1. R BHOOMIKA (4GW21IS037) Under the Guidance of 2. YASHASWINI D A (4GW21IS058) Anand M 3. INDUSHREE (4GW22IS402 ) Assistant Professor, Dept. Of ISE

2. contents Introduction • Proposed System with Objectives • Present System (if any similar system exists) • Literature Survey • Requirement Specifications Functional/Non functional Hardware/Software • System Architecture • Data Flow Diagrams • Module Description • Methodology • Implementation • Test Cases • Advantages • Applications • Conclusion

3. The heart rate, indicating the number of beats per minute, is crucial for assessing cardiovascular health and stress levels. HR values vary with age, medical history, and physical activity, with less active individuals typically having higher HRs due to their hearts working harder. . Real-time heart rate measurement is achieved with OpenCV in Python, allowing for the simultaneous monitoring of multiple people through object tracking and face detection . . Introduction

4. Proposed system Face Recognition Module : Camera : Captures the facial image of the user. Face Detection Algorithm : Identifies the presence of a face in the captured image. Feature Extraction : Extracts unique facial features for recognition . Database : Stores facial feature data of authorized users. Matching Algorithm : Compares the extracted features with the database to verify identity. Heart Rate Prediction Module : Sensors : Devices to measure heart rate, such as photoplethysmography (PPG) sensors or camera-based methods that use subtle changes in facial coloration . Signal Processing : Processes the raw heart rate data to ensure accuracy. Prediction Algorithm : Uses machine learning models to predict heart rate based on the data from the sensors.

5. objectives Face Detection Algorithm : Identifies the presence of a face in the captured image. Feature Extraction : Extracts unique facial features for recognition. Database : Stores facial feature data of authorized users. Matching Algorithm : To design a Non-Intrusive Heart Rate Monitoring System using video processing technology for comfort and sensor-free monitoring. To develop Facial Video Analysis to extract heart rate data from webcam images by processing RGB channels and converting to HSI for precise analysis. To implement a Real-Time Multi-Person Monitoring System for simultaneous detection and tracking of multiple faces for heart rate measurement. Support heart rate estimation for multiple simultaneous individuals in the camera's field of view .

6. Present System In the existing system, sensors are employed to measure the user's heart rate. These sensors provide real-time physiological data for identity verification. The system is designed to be budget-friendly, utilizing readily available sensors and technology to keep costs low. Various hybrid techniques are implemented to detect heart rate. However, these methods often fail to deliver precise results, affecting the reliability of the system. The integration of multiple techniques and sensors makes the system more complex, potentially leading to difficulties in maintenance and user operation.

7. Literature survey Title Author(s) References Description Limitations Year “ Self-Supervised Learning for Remote Heart Rate Estimation from Face Videos " Song et al. Song, R., Zhang, X., and Gao, S. (2023). Explored self supervised learning to robust rPPG signal extraction, reducing the datasets. Privacy issues and the potential for biometric spoofing. 2023 " A Multi-Scale Temporal Network for Remote Photoplethysmogra phy " Niu et al. Niu , X., Zhao, Z., and Li, X. (2021). Introduced MTTS CAN, utilizing multiple temporal scales to capture short-term and long term rPPG signal Complexity of the model might increase the difficulty of real-time implementation. 2021 Table1. Literature Survey

8. " DeepPhys: Video Based Physiological Measurement Using Convolutional Attention Networks " Chen and McDuff Chen, W., and McDuff, D.J. (2021). Proposed a convolutional attention network to improve heart rate estimation by focusing on relevant facial regions and features. Susceptible to performance drops in highly dynamic scenes or extreme lighting conditions. 2021 " Remote Photoplethysmogra phy with Spatiotemporal Networks " Yu et al. Yu, Z., Zhao, Z., and Li, X. (2020). Combined spatial feature extraction and temporal signal processing for remote photoplethysmograp hy May struggle with significant motion artifacts and lighting variations. 2020 “ Deep Learning for Heart Rate Estimation from Face Videos ” Chen et al. Chen, W., Wang, H., and Li, Y. (2019). Utilized deep CNNs to estimate heart rate from facial videos pre-trained models like VGGNet and a temporal spatial network. . Limited by the computational cost and the need for large labeled datasets. 2019 Table2. Literature Survey

9. Functional requirements: • The software should detect and locate the user's face accurately in each frame of the video • The software should extract the green channel from the forehead region and compute the average optical intensity overtime. • The heart rate estimation algorithm should be efficient and accurate, providing real time results. Non functional requirements: • The application's interface should be intuitive and easy to navigate, enabling users to start and stop heartrate estimation effortlessly. • The software should adhere to accessibility guidelines, ensuring it is usable by individuals with disabilities. • If applicable, the system should implement secure user authentication mechanisms to prevent unauthorized access to sensitive features. Requirement specifications

10. Hardware requirements: • Processor: 11th Gen Intel(R) Core(TM) • RAM: 1GB or more • Hard disk : 40GB Software requirements: • Operating System: Windows 7 and above • Coding Language: Python • Framework: Flask • Tools: Tensorflow • Libraries: Pandas, Numpy , Sklearn , Scikit • Editor: Jupyter Requirement specifications

11. SYSTEM ARCHITECTURE

12. IMPLEMENTATION class Tracker: Track a region of interest in a video. def _ init _(self, t, im , mi): self.mi = mi self.updat ROI(t, im , mi) ef update ROI(self, t, im , mi, weight=0.5): et the new ROI as weighted average of the given ROI with given weight) and old ROI. elf.lastTrackTime = self.lastRoiTime = t elf.mi += weight * (mi - self.mi ) racker= getattr (cv2, ' TrackerMedianFlow ', None) or\ etattr (cv2, ' legacy_TrackerMedianFlow ') elf.tracker = Tracker.create () elf.tracker.init ( im , tuple( self.mi )) elf.ok = True class FaceTracker (Op): Detect and track faces: (frame)-> (frame, faces) def _ init _(self, source=None): Op._ init _(self, source) path= os.path.join ( os.path.dirame (_file_), data', 'lbpcascade_frontalface_improved.xml’) elf.classifier = cv2.CascadeClassifier(path) elf.trackers = [] elf.t0 = 0.0 lFacetracker module :

13. An advanced non-contact, real-time heart rate monitoring system using facial video analysis. Expected Outcomes of the Project Healthcare accessibility utilizing widely available webcam for enhanced monitoring capabilities. Improved comfort and sensor-free monitoring of multiple individuals without physical contact.

14. 1. Chen, W., Wang, H., and Li, Y. (2019). "Deep Learning for Heart Rate Estimation from Face Videos." IEEE Transactions on Biomedical Engineering. 2. Yu, Z., Zhao, Z., and Li, X. (2020). "Remote Photoplethysmography with Spatiotemporal Networks." IEEE Transactions on Biomedical Engineering. 3. Chen, W., and McDuff, D.J. (2021). "DeepPhys: Video-Based Physiological Measurement using the method Convolutional Attention Networks." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4. Niu, X., Zhao, Z., and Li, X. (2021). "A Multi-Scale Temporal Network for Remote Photoplethysmography" IEEE Transactions on Biomedical Engineering. 5. Song, R., Zhang, X., and Gao, S. (2023). "Self-Supervised Learning for Remote Heart Rate Estimation from Face Videos." IEEE Transactions on Biomedical Engineering. references

15. 6. Monkaresi , H., et al. (2014). "Automated Detection of Engagement Using Video-Based Estimation of Facial Expressions and Heart Rate." IEEE Transactions on Affective Computing. 7. Tulyakov , S., et al. (2016). "Self-adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 8. Winkler, I., and Nascimento, E.G.S. (2022). "Machine Learning Models and Videos of Facial Regions for Estimating Heart Rate: A Review on Patents, Datasets, and Literature." Electronics. 9. https://ieeexplore.ieee.org/abstract/document/9303142. 10. https://ieeexplore.ieee.org/xpl/conhome/9303028/proceeding.

16. THANK YOU