645FinalVivadawwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww

viva

1. Gemstones Multiclass Classification Using Convolution Neural Network GUIDE: Dr. C. Pretty Diana Cyril NAME: Anush Bhatia REG N0: RA1711003010645 Submitted for Viva of Major Project (15CS496L)

2. ABSTRACT

3. ABSTRACT This project is based on the researching of the convolution neural network for the classification of gemstone seen as a great opportunity to expand the range of application of the well known CNN, which have been used more and more in different fields. First, it is explained the theoretical concepts about machine learning and deep learning, besides a little comparison between them in order to for the reader to make the thesis more understandable. It is also introduced the different machine learning classifiers and the different methods of deep learning which have been used during the presented work. After having the basic concepts it will be explained the different methods that have been studied throughout the thesis to find the best possible accuracy in the training, validation and testing of the data set. Finally, it will be showcased the different results obtained followed by a short explanation per each of them

4. OBJECTIVE

5. Classification of all gemstone and identify them whenever required. Increase the dataset as so that all the rocks and gemstone can be classified and further studies can be done by this. People can know more about the a particular gemstone and study about it. No expert will be required for the person to know about it and no programming language is required for it. Classify all the present gemstone and add more to them by researching on internet. OBJECTIVE

6. LITERATURE SURVEY

7. 1. Classification of Objects by Shape Applied to Amber Gemstone Classification Authors: Armantas Ostreika , Marius Pivoras , Alfonsas Misevičius , Tomas Skersys Name of the Journal: MDPI Year of Publication: 2021

8. 1. Classification of Objects by Shape Applied to Amber Gemstone Classification Work of the Paper: Using various accesses to identify the Shape Parametric Description. Applying Various ML algorithm to checkout the output on the gemstone. Accuracy achieved by them was pretty low cause of having a small training dataset. Advantage : Insights developed in this project are for only shape based classification. Disadvantages: Data is focused only on the gemstones and ignoring the important rocks present there. Choice of data features could be more objective.

9. 2. The Classification and Distribution of Gemstones from Northern Balochistan , Pakistan Authors: Muhammad Ishaq Kakar , Muhammad ayoub Khan , Jalil Ahmed Name of the Journal: Research Gate Year of Publication: 2017

10. 2. The Classification and Distribution of Gemstones from Northern Balochistan , Pakistan Work of the Paper:   Collection from the Zhob  Valley ophiolite belt and its overlying and underlying sedimentary rocks host various types of gemstone Parametric Classification of gemstone based on the Colour, Lustre, Compound Present, Hardness etc. Also Plotting the expected availability of the rocks from the areas where it was found. Advantage: Almost all the studied gemstones are semiprecious except almandine garnet, tsavorite garnet and brucite which may be precious in nature Found all the reserves for commercial usage of gemstones and some important coumpounds Disadvantage: Classififed rocks and gemstones based in pakistan . Also did not train a model to identify the rocks by this

11. 3. A biological image classification method based on improved CNN Authors : Jiaohua Qin, Wenyan Pan, Xu yu Xiang ,Yun Tan, Guimin Hou Name of the Journal: Science Direct Year of Publication: 2020

12. 3. A biological image classification method based on improved CNN Work of the Paper: To solve the problem of increased computation caused by this method, the some modules in CNN was replace by Inverted Residual Block. Semi-supervised approach (lexicon-based analysis) in which a lexicon is created, and then a rule-based algorithm is employed. Supervised approach (machine learning) in which data is annotated with the help of a psychologist, text features are extracted, and then machine learning classifiers are used. Advantages: The method was validated on five benchmark datasets, two of which are biological image datasets, and it yielded the promising accuracy, and less parameters. The experiment also proves method scalability, which can be applied to different networks. In future work, we hope to find a more efficient and lightweight convolution method that can more effectively reduce network parameters. Disadvantage: No image operation performed on before experimenting. Unused on some situation where orientation is taken into consideration.

13. 4. Multiclass Amber Gemstones Classification With Various Segmentation and Committee Strategies Authors: Saulius Sinkeviius , Arnas Lipnickas , Kstas Rimkus Name of the Journal: IEEE Access Year of Publication: 2013

14. 4. Multiclass Amber Gemstones Classification With Various Segmentation and Committee Strategies Work of the Paper: An approach for amber gemstone samples classification is proposed. The algorithm is simple and easy to implement, no complex preprocessing operations are needed. Further few classifiers committee formation is compared. Most useful was committee of decision tree members trained by Half&Half method. Advantage : The best accuracy 73.18 % is acquired by 16 Tree committee created using Half&Half method while segments size is 400 pixels. Disadvantage: Enhancements need to be made in the algorithm so that efficiency could be improved. It is focused on on only a single class of various amber substances.

15. 5. An improved Canny edge detection algorithm for color image Authors: Geng Xin, Chen Ke , Hu Xiaoguang Name of the Journal: IEEE Xplore Year of Publication: 2012

16. 5. An improved Canny edge detection algorithm for color image Work of the Paper: The traditional Canny edge detection method is widely used in gray image processing. However, this traditional algorithm is unable to deal with color images and the parameters in the algorithm are difficult to be determined adaptively. The proposed algorithm is composed of the following steps: quaternion weighted average filter, vector Sobel gradient computation, non-maxima suppression based on interpolation, edge detection and connection. . Advantages: Identify the edges of the coloured images and avoid deletion of that area. This algorithm can be seen as an expansibility of traditional Canny from gray to color space. Disadvantage: Elimination of the background and used area from image is not tackled in this method presented. Multi channel may cause problem in this algorithm.

17. ARCHITECTURE DIAGRAM

18. MODULE DESCRIPTION

19. Modules Data Preparation Resize processed images to  img_w , img_h  - this option will be used when cropped and as a parameter of neural network. provide train directory path for model. Create a list of class names while reading folders Labelling and plotting Creating two arrays  Train_Imgs , Train_Lbls  which contain images and corresponding names of classes of gemstones respectively. Convert  Train_Lbls  with strings to list with corresponding numbers. Crop edges of images using Canny algorithm Using cv2.Canny find the array representing frame which is the edges how the original picture will be cut. Function  edge_and_cut ( img ) receives single image and returns a  cropped image Model Training Hyperparameters are set before training; they represent the variables which determines the neural network structure and how the it is trained.

20. Create function This function will be also used with test images; Read each image from disk using  cv2  and resize it to  img_w *1.5, img_h *1.5 ; Set  cv2.COLOR_BGR2RGB  option because opencv reads and displays an image as BGR color format instead of RGB color format. Without this option images will be shown in blue hue because  matplotlib  uses RGB to display image; Create a list of class names while reading folders -  Amethyst, Onyx, etc ; When  Images  list is ready - convert it to Numpy array; Return tuple of 2 elements: Images and corresponding Labels. MODULE : Read images and class names

21. Create function which converts string labels to numbers Convert string labels to a list of numbers using list CLASSES. The index will represent label of class, f.e .  Ruby = 0, Amethyst = 24 , etc. when Labels list is ready - convert it to Numpy array. MODULE : Numbering the folders

22. Fill arrays of Images and corresponding Labels with data Classify Train and Test: Create two arrays  Train_Imgs , Train_Lbls  which contain images and corresponding names of classes of gemstones respectively; Convert  Train_Lbls  with strings to list with corresponding numbers; MODULE : Labelling

23. Plot images and their labels for preview Using matplotlib and random show 16 (4x4) random images from the set and their labels (as string and as int a). Inference for image present at the end MODULE : Plotting

24. ABOUT Canny is a popular edge detection algorithm, which detects the edges of objects present in an image. FEATURE SELECTION Using cv2.Canny find the array representing frame which is the edges how the original picture will be cut; Function  edge_and_cut ( img ) receives single image and returns a  cropped image ( new_img )  of the size  img_w , img_h ; sometimes Canny algo cannot detect edges ( f.e . when the object has almost same color as background) so array edges will be zero-valued. In this case use original image. MODULE : Crop edges of images using Canny algorithm

25. Show cropped images Function  show_cropped  is kind-of duplicate  edge_and_cut (): it shows same random examples of Canny algo work:  original image, Canny edges, image with bounding box, cropped image  for better understanding how Canny algo works.  Replace train images with cropped images Create function which calls  edge_and_cut  and replaces  Train_Imgs   numpy array with array of cropped images. Don't forget that images that cannot be cropped will be replced with originals; Make sure the shape of final array is the same: NUMBER OF IMAGES x img_w x img_h x 3 (CHANNELS): Split data into train and validation sets use  sklearn  to split  Train_Imgs ,  Train_Lbls  into train (80%) and validation (20%) sets. MODULE : Modifying the train data

26. RESULTS & DISCUSSION

27. RESULTS The suggested model’s accuracy for the given data and compared with another model is depicted below: Improved the efficiency 56 to 61 Algorithm Accuracy Canny Edges 61% Normal Image 43%

28. DISCUSSION The above discussion leads to the following conclusions: 1.  It understands the colour: some gemstones are really similar. Also classify some rocks easily based on the shape parametric obtained from it. 2.  We can obtain higher accuracy by cropping the edges from the canny edge algorithm as that has improved the detection of the gemstone better. 3.  Some rocks are having quite similar structure and colour model is currently not very efficient to judge them and we will need a greater dataset for it. 4. Some similar structural gemstones were not detected properly due to almost the same structure for them. We can overcome this by adding more layers to CNN and adding enriched images for it to classify it easily.

29. Loss and Accuracy Curves

30. Confusion Matrix

31. SCREENSHOTS

32. PREDICTED VALUE: TRUE/false

33. Image Augmentation

34. REFERENCES

35. Armantas Ostreika , Marius Pivoras , Alfonsas Misevičius , Tomas Skersys (2021) Classification of Objects by Shape Applied to Amber Gemstone Classification . MDPI, Russia Muhammad Ishaq Kakar , Muhammad ayoub Khan , Jalil Ahmed (2017). The Classification and Distribution of Gemstones from Northern Balochistan , Pakistan, Research Gate. Jiaohua Qin, Wenyan Pan, Xu yu Xiang ,Yun Tan, Guimin Hou (2020). A biological image classification method based on improved CNN, Science Direct. Saulius Sinkeviius , Arnas Lipnickas , Kstas Rimkus (2013). Multiclass Amber Gemstones Classification With Various Segmentation and Committee Strategies . IEEE Access. Geng Xin, Chen Ke , Hu Xiaoguang (2012). An improved Canny edge detection algorithm for color image, 42(2), IEEE Xplore

36. PUBLICATION DETAILS

37. Journal of Huazhong University of Science and Technology PUBLICATION DETAILS

38. THANK YOU