Virtual Mouse using hand gesture recognition

This project is to develop a Virtual Mouse using Hand Gesture Recognition. Hand gestures are the most effortless and natural way of communication. The aim is to perform various operations of the cursor. Instead of using more expensive sensors, a simple web camera can identify the gesture and perform the action. It helps the user to interact with a computer without any physical or hardware device to control mouse operation.

1. Virtual Mouse using Hand Gesture Recognition Presented By - 53 Mukti G. Kalsekar 59 Shubhangi M. Shirke Guided By- Prof.Kumud Wasnik Usha Mittal Institute Of Technology May 2022

2. Contents Introduction Literature Survey Problem Statement Proposed System Methodology Classification Model Implementation Resources Future Scope References

3. Introduction This project is to develop a Virtual Mouse using Hand Gesture Recognition The hand gestures are the most effortless and natural way of communication. The aim is to perform various operations of the cursor. Instead of using more expensive sensors, a simple web camera can identify the gesture and perform the action. It helps the user to interact with a computer without any physical and hardware device to control mouse operation.

4. Sr.No Name of Research Paper Author Name Technology Used Advantages / Disadvantages 1 Virtual Mouse Control Using Colored Fingertips and Hand Gesture Recognition 11-12 Sep 2020 Vantukal Reddy , Thumma Dhyanchand , Galla Vamsi Krishna,Satish Maheshwaram Hand Gesture Recognition, Image processing,Neural Network Algorithm,Python, opencv,pyautogui Advantage : used neural network for hand gesture recognition. Disadvantage : Limitation of This Mouse they used Colored Fingertips 2 Virtual Mouse Implementation Using Opencv , 2019 Kollipara Sai Varun. I Puneeth, Dr.T.Prem Jacob Opencv,IP,Deep Learning, Pyautogui,numpy,Anaconda Advantage : used python for hand gesture recognition. Disadvantage : In this they used color variation techniques ,it's very complex to use 3 Hand Gesture - Virtual Mouse For Human Computer Interaction , 2018 Sherin Mohammed, V H preetha Matlab s/w ,Two cameras, Advantage : Get 90% Correct Detection for both views, under enough Light Condition Disadvantage : used two cameras and its cost is more Literature Survey

5. Literature Survey 4 Virtual Mouse, 12th march 2015 Ashish Mhetar.B K Srioop,Kavya AGS,Ramanath Nayak,Ravikumar Javali,Suma K V IR Camera, USB-HID,IR Pen , Teensy(ARM M4) Advantage : i ts work as a virtual marker , its effective to use while teaching . Disadvantage : They used a Hardware devices, and the Cost of this is More,. 5 Virtual Mouse Using Object Tracking 10th july 2020 Monali Shetty,Christina Daniel,Manthan Bhatkar,Ofrin Lopes HSV Technique,Python, Opencv,Ip Advantage : Accuracy of this System is in Plain Background :95% Disadvantage : Non-Plain Background :40% 6 Design and Development of Hand Gesture based Virtual Mouse 19 Dec 2019 Kabid Shibly,Samrat Dey,Aminul islam, Shahriar Showrav HCI Technology Advantage : use of latest technology. Disadvantage : In this System barrier is the lightning Condition That’s Why System Still Cant enough to replace the Mouse

6. Problem Statement To design a virtual mouse which detects hand gestures and performs mouse operations only using the fingers. we used different combinations of fingers to perform various operations of the mouse according to which particular combination of fingers is recognized. In Proposed system users don't have to color their fingers with a specific color and are not required to use any device or sensors. Easy to use and cost effective.

7. Proposed System

8. Fig2 Operations of cursor- Hand Landmark detection(fingers up(1) and down(0)) Move Left Click Double Click Right click Scroll Up Scroll down Volume Control Brightness Control Steps- It will detect the camera, video interface will be start The camera can extract and recognize human hand gestures from video interface Hand tracking functionality is done by mediaPipe After the recognition the cursor move accordingly, to perform various operations Fig 1. Block Diagram METHODOLOGY

9. Classification Model Deep Learning Deep Learning is a subset of machine learning. It is basically learning and improving on its own by examining other algorithm It works on artificial neural network that was designed to imitate human think and learn capabilities Mediapipe MediaPipe to recognize the hand and the hand key points. MediaPipe returns a total of 21 key points for each detected hand.

10. CONVOLUTION NEURAL NETWORK To recognize various features of image or video frame cnn makes use of layers. Cnn used layers to detect the features of input image. And apply convolution to each feature detection, and get the recognize output image. Fig 2. Flowchart

11. Implementation Hand Landmark Detection Cursor Move

12. Left Click Right Click Double Click

13. Scroll Up Scroll Down

14. Brightness Control Volume Control

15. Resources Software Requirements: 1 . Visual studio code Used Language/library: Python OpenCv Media Pipe Pyautogui pycaw Screen-brightness-control Hardware Requirement: Windows 10 webcam Minimum RAM 2GB Intel core i5 processor

16. Future Scope It reduces workspace and burden of extra hardware devices. it removes the burdens of devices, it brings the user and the workspace more closer

17. References Vantukal Reddy , Thumma Dhyanchand , Galla Vamsi Krishna,Satish Maheshwaram “Virtual Mouse Control Using Colored Fingertips and Hand Gesture Recognition”,11-12 Sep 2020 https://ieeexplore.ieee.org/document/9242677 Kollipara Sai Varun. I Puneeth, Dr.T.Prem Jacobi”Virtual Mouse Implementation Using Opencv “,2019 https://ieeexplore.ieee.org/document/8862764 Sherin Mohammed, V H preetha “Hand Gesture - Virtual Mouse For Human Computer Interaction ”,2018 https://ieeexplore.ieee.org/abstract/document/8748401/authors Ashish Mhetar.B K Srioop,Kavya AGS,Ramanath Nayak,Ravikumar Javali,Suma K V “Virtual Mouse” 12th March 2015 https://ieeexplore.ieee.org/document/7057759?arnumber=7057759 Monali Shetty,Christina Daniel,Manthan Bhatkar,Ofrin Lopes “Virtual Mouse Using Object Tracking”10th july 2020 https://ieeexplore.ieee.org/document/9137854 Kabid Shibly,Samrat Dey,Aminul islam, Shahriar Showrav”Design and Development of Hand Gesture based Virtual Mouse ”,19 Dec 2019 https://ieeexplore.ieee.org/document/8934612

18. Thank You