applications of Artifical Intillengence in biometrics.pptx

Commercial Applications
Access to computers, applications, websites
Access of entry to office, tracking attendance.
Control access to secure areas link banks, access to ATM, Credit card swipe.
Biometrics are also used in everyday applications such as virtual assistants controlled by voice like APPLE’S Siri, AMAZON’S Alexa.
2. Government Applications
• Airport entry
• Border control
• Defense and Government communications
Filing tax returns
• Updating Government issued identity card.

3. Law Enforcement Applications
• Criminal investigations
• Surveillance




























Handwriting Recognition


Handwriting of person is unique to every individual.

Handwriting recognition is the ability of a computer to receive and interpret intelligible handwritten input from sources such as paper documents , photographs, touch-screens and other devices.

On-line :about writing dynamics as the text is being written

Off-line: Deals with static information








Advantages of using Handwriting





High recognition rate as handwriting patterns are different for each person
Handwriting patterns do not change with time significantly.
Professionals cannot forge the handwriting style easily as it requires months of practice to copy style for every word and letter
Handwriting on a paper or tablet have similar comfort




Disadvantages of using Handwriting



It is limited only to people who can write.

Training a model require many samples.



Applications


Access to electronic systems like ATMs, Mobile phones, Tablets etc
Associating the historical records with writers
Attendance in the office
Complaints handling to resolve recurring issues in IT infrastructure in a company

Why HMM and CNN?(Motivation)





HMM seems to be the fastest method to deal with large datasets for a given hardware configuration.
Instead of using the pixel intensities directly in the model like it is done in many other methods, it is subjected to dimensionality reduction with various transformation methods like DCT or SVD.
Advantages of deep learning lies in its versatility, ease of use of libraries, flexibility to choose various techniques, standardization of code etc.
Purpose of this integration is to achieve speed, accuracy and ease of usage.







database

There are two data sets that are used in this research work to measure the performance . They are:

User Database – 880 Lines with 100 Writers - .PNG Format
IAM Dataset – 13,353 Lines with 657 Writers - .PNG Format




Each state can be represented with its pixel intensities as matrix M. Using singular value decomposition, singular values can be extracted.

M=USV
where, U matrix with elements U1,U2,…..UN , S matrix of singular values S1, S2,…….SN and V matrix with elements V1, V2………VN, can be used as features in the HMM.

These features are given to classifier for identification

results


HM MBW with User Database
Two hyper parameters of the model were varied to
optimize the mod

1. Technical Seminar on Applications of AI in Biometrics Presented by, Dr . Vinita Balbhim Patil Professor, Department of ECE Principal, Lingaraj appa Engineering college

2. Contents INTRODUCTION WRITER IDENTIFICATION WITH HMMBW WRITER IDENTIFICATION WITH MLPHMM AND CNNHMM WRITER IDENTIFICATION WITH HMMMLP CONCLUSION AND FUTURE SCOPE Handwriting Recognition Modu le I Module II Module I II Module I V Module VI Module V WRITER IDENTIFICATION WITH HMMCNN

3. Module I INTRODUCTION

4. Introduction What is Biometric? It refers to unique physical and behavioural characteristics of individuals such as fingerprint , facial features, voice or handwriting. Why we need Biometrics? Simpler , Faster and Stronger Authentication

5. Introduction Authentication is one of the important factors in order to grant access to a facility or service to certain people. To grant the authentication for access ,verification is important. Verification can be either manual or automatic.

6. Introduction Fig 1: Various Categories of biometrics

7. Applications of Biometrics 1. Commercial Applications Access to computers, applications, websites Access of entry to office, tracking attendance . Control access to secure areas link banks, a ccess to ATM, Credit card swipe . Biometrics are also used in everyday applications such as virtual assistants controlled by voice like APPLE’S Siri, AMAZON’S Alexa . 2. Government Applications • Airport entry • Border control • Defense and Government communications

8. Filing tax returns • Updating Government issued identity card . 3. Law Enforcement Applications • Criminal investigations • Surveillance

9. Handwriting R ecognition Handwriting of person is unique to every individual. Handwriting recognition is the ability of a computer to receive and interpret intelligible handwritten input from sources such as paper documents , photographs, touch-screens and other devices. On-line :about writing dynamics as the text is being written Off-line: D eals with static information

10. Handwriting Recognition Fig 2: Static and Dynamic Features of handwriting

11. General Biometric System Biometric sensor Feature Extraction Biometric sensor Feature Extraction Database if matches then verified Test sample Train samples

12. Handwriting Recognition Fig 3:Handwriting Recognition Framework

13. Advantages of using Handwriting High recognition rate as handwriting patterns are different for each person Handwriting patterns do not change with time significantly. Professionals cannot forge the handwriting style easily as it requires months of practice to copy style for every word and letter Handwriting on a paper or tablet have similar comfort

14. Disadvantages of using Handwriting It is limited only to people who can write. Training a model require many samples.

15. Applications Access to electronic systems like ATMs, Mobile phones, Tablets etc Associating the historical records with writers Attendance in the office Complaints handling to resolve recurring issues in IT infrastructure in a company

16. Why HMM and CNN?(Motivation) HMM seems to be the fastest method to deal with large datasets for a given hardware configuration. Instead of using the pixel intensities directly in the model like it is done in many other methods, it is subjected to dimensionality reduction with various transformation methods like DCT or SVD. Advantages of deep learning lies in its versatility, ease of use of libraries, flexibility to choose various techniques, standardization of code etc. Purpose of this integration is to achieve speed, accuracy and ease of usage.

17. Module II WRITER IDENTIFICATION USING HMM

18. Objective   To implement and measure the performance of HMMBW for handwriting recognition

19. Database There are two data sets that are used in this research work to measure the performance . They are: User Database – 880 Lines with 100 Writers - .PNG Format IAM Dataset – 13,353 Lines with 657 Writers - .PNG Format

20. Block diagram Fig 4:Block diagram of HMM model with singular values as features

21. Fig 5:HMM and states in Writer Identification In a text image with 1200 pixels (W) and 60 pixels (H), there are a total of 72,000 pixels. The text image can be divided into six non-overlapping blocks with 200 x 6 pixels. The blocks can be overlapping or non-overlapping

22. Each state can be represented with its pixel intensities as matrix M. Using singular value decomposition, singular values can be extracted. M=USV where, U matrix with elements U1,U2,…..UN , S matrix of singular values S1, S2,…….SN and V matrix with elements V1, V2………VN, can be used as features in the HMM. These features are given to classifier for identification

23. Results HMMBW with User Database Two hyper parameters of the model were varied to optimize the model. The parameters were: Number of samples used for training. Number of states considered as percentage of maximum possible overlapping states.

24. Varying the number of samples per writer in model training Number of Samples per writer Total number of training samples Total number of test samples Ratio of test to train sets Total Matches Total Mismatches Accuracy 1 100 780 7.80 130 650 16.67% 2 200 680 3.40 280 400 41.18% 3 300 580 1.93 300 280 51.72% 4 400 480 1.20 300 180 62.50% 5 500 380 0.76 270 110 71.05% 6 600 280 0.47 270 10 96.43% Table 1: Variation in accuracy with Sample Size

25. Fig 6: Variation in Accuracy of HMMBW with sample size Fig7: Variation in Accuracy of HMMBW with test –train ratio

26. Varying number of states in HMMBW The number of overlapping states, with a stride of S, can be a maximum of (W-F+2P/S)+1. W= Width of image in Pixels F=Width of block of each state P= Padding (Zero in this case) S= Number of strids For an image width (W) of 1000 pixels, if the stride (S) is 1 and for block width (F) of 50, Maximum number of overlapping states possible (N) = (1000 – 50)/1 + 1= 951

27. Number of states as percentage of maximum possible overlapping states HMMBW Accuracy 50% 96.43% 60% 96.90% 70% 97.60% 80% 98.10% 90% 99.20% 91% 99.90% 92% 99.30% 93% 99.10% 94% 98.90% 95% 98.90% 100% 98.40% Table 2: Variation in accuracy of HMMBW with number of states

28. Fig 8: Variation in Accuracy in HMMBW with Varying number of states

29. Benchmarking HMMBW with other methods on IAM Dataset Author Features  Classifier Accuracy Language Marti et al. Graphemes extracted from cursive handwriting Cosine similarity 95% English Hertel  and  Bunke Height of writing zones, Distance between connected components kNN 92% English Xing and Qiao Height of writing zones, Distance between connected components ANN 92% English HMMBW Raw pixels by CNN CNN 98.8% English This Research work States based HMMBW HMM 99.8% English Table 3: Accuracies of various models on IAM dataset

30. Benchmarking HMMBW with other HMM models Authors Database Features  Classifier Accuracy Language Schlapbach  et al. 100 writers with 20% skilled imposters States based on HMM requirements HMM 97% English Schlapbach et al. 100 writers Location coordinates HMM 96% English This research work IAM States based HMMBW HMM 99.8% English Table 4:Accuracies of HMM model on different dataset

31. Module IV WRITER IDENTIFICATION WITH MLPHMM AND CNNHMM

32. Objectives To implement and measure the performance of hybrid technique MLPHMM (Multilayer Perceptron Hidden Markov Model). To implement and measure the performance of hybrid technique CNNHMM ( Convolutional Neural Network Hidden Markov Model).

33. Block Diagram Fig 9: Block diagram of Proposed MLPHMM

34. Fig 10: Feature Extraction with MLP and classification with BW

35. Fig 11 : Feature extraction with MLP

36. Fig 12 : Feature Extraction with CNN and classification with BW

37. Fig 13: Feature Extraction with CNN

38. Results Number of Samples per writer Total number of training samples Total number of test samples Ratio of test to train sets Total Matches Total Mismatches MLPHMM Accuracy CNNHMM Accuracy 1 100 780 7.80 130 650 15.34% 17.23% 2 200 680 3.40 280 400 40.32% 43.21% 3 300 580 1.93 300 280 52.00% 51.35% 4 400 480 1.20 300 180 59.00% 64.78% 5 500 380 0.76 270 110 68.00% 72.87% 6 600 280 0.47 270 10 94.00% 95.58% Table 5 : Variation in Accuracy of MLPHMM and CNNHMM with Sample Size

39. Fig 14 : Variation in Accuracy of MLPHMM and CNNHMM with Sample Size Fig 15: Variation in Accuracy of MLPHMM and CNNHMM with test to train ratio

40. Number of states as percentage of maximum possible overlapping states MLPHMM Accuracy CNNHMM Accuracy 50% 94.00% 95.58% 60% 94.85% 95.65% 70% 96.35% 97.50% 80% 97.95% 98.30% 90% 98.50% 98.95% 91% 99.10% 99.25% 92% 99.20% 99.30% 93% 99.10 % 99.20% 94% 98.65% 98.95% 95% 98.60% 98.75% 100% 98.30% 98.90% Table 6: Variation in Accuracy of MLPHMM and CNNHMM with number of states

41. Fig 16 : Variation in Accuracy of HMMBW with number of states (% of maximum possible overlapping states)

42. Module V WRITER IDENTIFICATION WITH HMMMLP

43. Objective To implement and measure the performance of hybrid technique HMMMLP

44. Block diagram Fig 17: Block diagram of proposed HMMMLP model

45. Fig 18: Feature Extraction for classification with MLP

46. Fig 19: Classification Using MLP

47. Hyper parameters of the architecture used in HMMMLP Neurons in First hidden layer (activation function is ReLU ): 634 Neurons in Second hidden layer (activation function is ReLU ): 422 Neurons in Third hidden layer (activation function is ReLU ): 280 Neurons in Output layer (activation function is SOFTMAX): 100 Drop out (10%)

48. MLP Algorithm Weights are initialized randomly Error is computed in terms of Mean square error between actual and predicted output Derivative of error w.r.t weights is computed The derivative is back propagated to all the weights of all layers The derivatives are adjusted with learning rate and derivatives with gradient descent method. The inputs are again presented to the network and error is computed. This procedure is repeated until the error is less than 1e-04.

49. Results Number of Samples per writer Total number of training samples Total number of test samples Ratio of test to train sets HMM MLP Accuracy 1 100 780 7.80 20.5 % 2 200 680 3.40 46.25% 3 300 580 1.93 57.0% 4 400 480 1.20 66.3% 5 500 380 0.76 74.4% 6 600 280 0.47 97.30% Table 7: Variation in accuracy of HMMMLP with sample size

50. Results Number of states as percentage of maximum possible overlapping states HMMMLP Accuracy 50% 97.30% 60% 97.40% 70% 97.90% 80% 98.10% 90% 99.50% 91% 99.70% 92% 99.90% 93% 99.85% 94% 99.10% 95% 99.00% 100% 98.60% Table 8: Variation in accuracy of HMMMLP with number of states

51. Module VI WRITER IDENTIFICATION WITH CNN

52. Objective To implement and measure the performance of hybrid technique HMMCNN

53. Block Diagram Fig 20 : Block Diagram of Proposed HMMCNN

54. Fig 21: Feature Extraction for classification with CNN

55. Fig 22: Classification Using CNN

56. Fig 23: LeNet Architecture

57. Hyper parameters of the architecture used in HMMCNN Number of colvolutional and max pool layers:-2 Number of filters in First Convolutional Layer (activation function is ReLU ):- 20 First down sampling layer – Max pool layer Filter size – 3x3 Number of filters in Second Convolutional Layer (activation function is ReLU ):- 40 Second down sampling layer – Max pool layer Filter size – 3x3 Drop out – 20%

58. CNN Algorithm Weights of neuron and filters are initialized randomly Error is computed in terms of Mean square error between actual and predicted output in the fully connected layer Derivatives of error w.r.t weights of neurons and filters are computed The derivative is back propagated to all the weights of all layers and filters The derivatives are adjusted with learning rate and derivatives with gradient descent method. The inputs are again presented to the network and error is computed This procedure is repeated until the error is less than 1e-04.

59. Number of Samples per writer Total number of training samples Total number of test samples Ratio of test to train sets HMMCNN Accuracy 1 100 780 7.80 22.35% 2 200 680 3.40 50.15% 3 300 580 1.93 59.50% 4 400 480 1.20 68.10% 5 500 380 0.76 79.80% 6 600 280 0.47 98.50% Table 9: Variation in accuracy of HMMCNN with sample size

60. Number of states as percentage of maximum possible overlapping states HMMCNN Accuracy 50% 98.50% 60% 98.50% 70% 98.60% 80% 98.75% 90% 99.85% 91% 99.90% 92% 99.95% 93% 99.85% 94% 99.20% 95% 99.10% 100% 99.00% Table 10: Variation in accuracy of HMMCNN with number of states

61. Number of VTU-WRITER Samples per writer Total number of training samples Total number of test samples Computational Time (s) for Training Computational Time (s) for Testing 1 100 780 25 6.00 2 200 680 41 4.50 3 300 580 71 4.10 4 400 480 99 3.60 5 500 380 125 3.10 6 600 280 160 2.60 Table 11 : Computational Times for Training and Testing with HMMCNN on User database

62. Fig 24 :Computational times for Training vs Training Sample Size with HMMCNN on user database Fig 25 :Computational times for Testing vs training Sample Size with HMMCNN on user database

63. Total number of IAM training samples Total number of IAM test samples Computational Time (s) for Training Computational Time (s) for Testing 1517 11836 390 95.00 3035 10318 620 75.00 4552 8801 1100 64.00 6070 7283 1520 51.00 7587 5766 1927 43.00 9104 4249 2345 33.00 Table 12: Computational times for Training and Testing with IAM dataset

64. Fig 26:Computational times for Training vs Training Sample Size with HMMCNN on IAM dataset Fig 27:Computational times for Training vs Training Sample Size with HMMCNN on IAM dataset

65. Authors References Database Features  Classifier Accuracy Bensefia  et al. [36-39] IAM Graphemes extracted from cursive handwriting Cosine similarity 95.00% Marti et al. [40] IAM Height of writing zones, Distance between connected components kNN 92.00% Hertel and Bunke [41] IAM Height of writing zones, Distance between connected components ANN 92.00% Xing and Qiao [111] IAM Raw pixels by CNN CNN 98.80% This research work IAM States based HMMBW BW 99.80% IAM States based HMMBW MLP 99.90% IAM States based HMMBW CNN 99.95% Table 13: Accuracies of Various Models on IAM Dataset

66. Fig 28: Accuracies of Various Models on IAM Dataset

67. Module VII CONCLUSIONS AND SCOPE FOR FUTURE WORK

68. Total number of training samples Total number of test samples Ratio of test to train sets HMMBW Accuracy MLPHMM Accuracy CNNHMM Accuracy HMMMLP Accuracy HMMCNN Accuracy 100 780 7.80 16.67% 15.34% 17.23% 20.5% 22.35% 200 680 3.40 41.18% 40.32% 43.21% 46.25% 50.15% 300 580 1.93 51.72% 52.00% 51.35% 57.0% 59.50% 400 480 1.20 62.50% 59.00% 64.78% 66.3% 68.10% 500 380 0.76 71.05% 68.00% 72.87% 74.4% 79.80% 600 280 0.47 96.43% 97.30% 98.50% Table 14: Comparison of accuracies of all five models for different sample sizes of train datasets 94.00% 95.58%

69. Number of states as percentage of maximum possible overlapping states HMMBW Accuracy MLPHMM Accuracy CNNHMM Accuracy HMMMLP Accuracy HMMCNN Accuracy 50% 96.43% 94.00% 95.58% 97.30% 98.50% 60% 96.90% 94.85% 95.65% 97.40% 98.50% 70% 97.60% 96.35% 97.50% 97.90% 98.60% 80% 98.10% 97.95% 98.30% 98.10% 98.75% 90% 99.20% 98.50% 98.95% 99.50% 99.85% 91% 99.90% 99.10% 99.25% 99.70% 99.90% 92% 99.30% 99.20% 99.30% 99.95% 99.95% 93% 99.10% 99.10 % 99.20% 99.90% 99.85% 94% 98.90% 98.65% 98.95% 99.10% 99.20% 95% 98.90% 98.60% 98.75% 99.00% 99.10% 100% 98.40% 98.30% 98.90% 98.60% 99.00% Table 15: Variation in Accuracy of all five models with number of states

70. Sl. No Method Accuracy using User database Accuracy using IAM dataset 1. HMMBW 99.90 % 99.80% 2. MLPHMM 99.20 % 99.50% 3. CNNHMM 99.30% 99.60% 4. HMMMLP 99.95% 99.90% 5. HMMCNN 99.95% 99.95% Table 16: Accuracies of Five Models on user database and IAM dataset

71. Contributions of research work A baseline HMM model with singular value features was developed A MLP model was developed with features extracted from HMM as the input. This is the first of its kind in the application to address writer identification problem. A CNN model was developed to improve up on MLP with features extracted from HMM as the input. This is again the first of its kind in the application to address writer identification problem.

72. Scope for Future Work HMM model can be developed with other feature vectors like DCT, KLT or PCA coefficients for writer identification dataset. The HMMCNN model can be extended to other languages. Model need to be extended to dynamic characteristics. HMMCNN model can be improved with Google inception models. Measure the performance of HMMCNN on cloud environments like AWS or Microsoft Azure or GCP and compare with FPGA. Develop the code in FPGA directly using C or C++ so that the run times are much faster than MATLAB.

73. References [1] Byung-Gyu Kim, Han- Ju Kim and Dong-Jo Park, “New Enhancement Algorithm for Fingerprint Images”, IEEE, 2002, pp 1051-4651. [2] Chapa Martell and Mario Alberto, “Fingerprint image enhancement algorithm implemented on an FPGA”, University of Electro-communications, Tokyo, Japan,August 1, 2009, pp 1-6. [3].Lin Hong, Yifei Wan, and Anil Jain, “Fingerprint ImageEnhancement : Algorithm and Performance Evaluation”, IEEE Tractions on pattern analysis and machine intelligence, vol.20, No.8 August 1998, pp 777-789. [4] S.Gayathri and V.Sridhar , “FPGA Implementation of Normalization block of Fingerprint Recognition Process” Proceedings of International Conference on Recent Trends in Signal Processing, Image Processing and VLSI, February 21-22, 2014, pp 30-38. [5] Garcia M L, et. al, “FPGA implementation of a ridge extraction fingerprint algorithm based on microblaze and hardware coprocessor”, International conference, August 28-30 , 2006, pp 1-5. [6]. S.Gayathri and V.Sridhar “FPGA Implementation of Orientation Field Estimation Of Fingerprint Recognition Process “, International Journal in Recent trends in Engineering and Technology Vol. 11, No. 1, July 2014, pp 132- 143. [7] U. W. Vutipon Areekul , Kittiwat Suppasriwasuseth and Saward Tantaratana , " Seperable Gabor Filter Realization for Fast Fingerprint Enhancement," 2005. [8]. S.Gayathri and Dr V.Sridhar “Design and simulation of Gabor filter using verilog HDL”- International Journal of Latest Trends in Engineering and Technology, ISSN: 2278-621X, Volume 2, Issue 2, March 2013, pp 77-83.

74. [9] Sunny Arief Sudiro,”Thinning Algorithm for Image Converted in Fingeprint Recognition System”, National Seminar Soft Computing Intelligent Systems and Information Technology, 2005. [10]. A. Jain, L. Hong and S. Pankanti . “Biometric identification,” Communication of the ACM, vol. 43, no. 2, February 2000, pp. 90-98. [11]    Fornes , A.,  Llados , J., Sanchez, G.,  Bunke , H. (2008).Writer Identification in Old Handwritten Music Scores. In: 8 th  IAPR Workshop on Document Analysis Systems, 347—353. [12]    Sas , J. (2006) Handwriting Recognition Accuracy Improvement by Author Identification. In: L.  Rutkowski  et al. (eds.), ICAISC 2006, LNAI 4029, 682--691.Springer, Heidelberg. [13]    Chaudhry , R., Pant, S. K. (2004) Identification of authorship using lateral palm print—a new concept.  J.ForensicScience  International , volume (141), 49--57. [14]    Schomaker , L. (2007) Advances in Writer Identification and Verification. In: 9th International Conference on Document Analysis and Recognition (ICDAR‘07), volume (2), 1268--1273. [15]    Plamondon , R.,  Lorette , G.  (1989) ―Automatic Signature Verification and Writer Identification—The State of the Art,‖ Pattern Recognition, vol. 22, no. 2, pp. 107-131. [16]    Leclerc , F.,  Plamondon , R. (1994) Automatic signature verification: The state of the art 1989- 1993. In Progress in Automatic Signature Verification edited by R.  Plamandon , World Scientific Publ. Co., pp. 1319.   [17]   Gupta, S. (2008).Automatic Person Identification and Verification using Online Handwriting Master Thesis. International Institute of Information Technology Hyderabad, India

75. Publications [1] Vinitha Patil and Rajendra Patil -“Design of High Efficient and High Recognition Rate for Real Time Handwritten Recognition using HMM and ANN Classification, International Journal of Control Theory and Applications , vol-10 , Nov 24, 2017, ISSN : 0974-5572. [2] Vinita Patil and Rajendra R Patil - ” User Identification using HMM and ANN ” in International Journal of Computer Sciences and Engineering, ISSN: 2347-2693, volume 5, Issue 8. [3] Vinita Patil and Rajendra R Patil -“ Review on Handwriting Recognition Techniques” in Journal of Applied Science and Computations ISSN NO: 1076-5131,Volume V, Issue XII, December/20 [4] Vinita Patil and Rajendra R Patil “ Writer identification using hidden Markov model tool kit ” in 4 th International conference on signal processing and Communication on 8 Feb 2019. [5] Vinita Patil and Rajendra R Patil -“ Writer Identification with Hybrid Model using Hybrid HMM and ANN”, International Journal of Recent Technology and Engineering , ISSN: 2277-3878, Volume-8 Issue-3, September 2019 [6] Vinita Patil and Rajendra R Patil -“Comparative Study of various Methods for User Identification based on Handwriting ” NCCDS-2019 Conference , GSSIETW , Mysuru . [7] Vinita Patil and Rajendra R Patil -“ Protocol design for Handwriting recognition using FPGA ” International conference of smart energy and communication, March 2020 [8] Vinita Patil and Rajendra R Patil -“ Feature Extraction with MLP and CNN in Writer Identification” TEST Engineering and Management, ISSN 0193-4120, Page No:16813-16821, Mar-Apr 2020. [9]Vinita Patil and Rajendra R Patil -“ Handwriting Recognition using Support Vector Machine and Artificial Neural Network” International Conference on Recent Trends in Computer, Electronics and Electrical Engineering, H y derabad on 24 th Aug 2020.

76. Thank you