PPT.pptx is very hej tan to the ahdd sande

It would be challenging to write a coherent and effective 3000-word single paragraph, as it's against conventional formatting norms and readability. However, here's a concise overview of hate text detection using BERT that could be expanded further for a detailed discussion:

---

Hate text detection using BERT (Bidirectional Encoder Representations from Transformers) represents a pivotal advancement in natural language processing (NLP) and machine learning, enabling more accurate and context-aware identification of hateful, abusive, or offensive content in textual data. BERT, developed by Google, is a deep learning model designed to understand language contextually by leveraging the bidirectional nature of transformer architecture, which processes the full context of a word in a sentence by considering both its preceding and succeeding words. This capability makes BERT highly effective for tasks such as sentiment analysis, text classification, and, in this case, hate speech detection. Hate text detection involves categorizing or flagging text as hate speech based on semantic and syntactic analysis, often requiring nuanced understanding of context, intent, and cultural subtleties. BERT's pretraining on vast corpora of unlabeled data, followed by fine-tuning on specific datasets labeled for hate speech, enables it to generalize across diverse scenarios while maintaining sensitivity to the specific language and expressions of hate. Fine-tuning typically involves using publicly available hate speech datasets, such as the Davidson dataset, HateBase, or datasets collected from social media platforms, where models are trained to predict whether a given text contains hate speech or falls into categories like offensive but non-hateful language or neutral content. This approach benefits from BERT's embedding capabilities, where textual input is tokenized, encoded into dense vector representations, and processed through multiple layers to capture hierarchical and semantic relationships. During fine-tuning, BERT learns to differentiate subtle nuances, such as the difference between sarcasm and genuine hate speech, which often pose significant challenges to simpler models. Furthermore, hate text detection models based on BERT can be optimized using techniques like data augmentation to balance datasets, addressing issues like bias or underrepresentation of certain hate speech types. However, deploying these models at scale involves addressing ethical concerns, such as ensuring fairness, avoiding false positives that may stifle free expression, and adapting models to diverse languages, dialects, and cultural contexts. Additionally, integrating hate text detection models into content moderation systems necessitates real-time inference capabilities, often achieved by distilling BERT models into smaller, more efficient architectures like DistilBERT or optimizing model inference using techniques like quantization and pruning. Despite its strengths, hate text dete

1. <Hate Text Detection using Bert> Department Computer Science and Engineering S. N. PATEL INSTITUTE OF TECHNOLOGY AND RESEARCH CENTRE UMRAKH, BARDOLI Guided by Presented by Prof. Jagruti boda MISHRA SIDDHARTH ANILKUMAR (210490131032) PATEL TARANG VINODBHAI (210490131026) MAHTO SANDEEP KAPILDEV (210490131069) MAHAKAL KAMAL DHARMESHBHAI (210490131057)

2. Outline Abstract Introduction AEIOU Canvases Mind Mapping Empathy Canvas Ideation Canvas Product Development Canvas Design Calculations Implementation/Prototype Conclusion References

3. Abstract Hate text, characterized by abusive language, derogatory comments, and harmful remarks, is a growing concern in the digital age, particularly on social media platforms. One promising approach is to develop a hate text detection system using advanced models like BERT (Bidirectional Encoder Representations from Transformers). While no model is perfect, leveraging BERT`s natural language processing capabilities can analyze text for context, tone, and intent, helping social media platforms and moderators take appropriate actions against hate text. This project aims to develop a Python hate text detection system using BERT, which can understand the context and meaning of text. The system will be evaluated using accuracy, precision, recall, and F1-score, and comparative analyses with existing hate speech detection methods to assess its effectiveness and efficiency

4. Introduction 1) What is Design Thinking? Your understanding Design thinking is a user-centered problem-solving method emphasizing empathy, creativity, and collaboration. In our hate text detection project using BERT, we followed these steps: 1. Empathize: Understand the impact of hate speech. 2. Define: Define project goals and problem statement. 3. Ideate: Generate innovative hate text detection ideas. 4. Prototype: Create and refine BERT-based detection models. 5. Test: Iterate and improve based on user feedback. 6. Implement: Develop a scalable, user-friendly system. 7. Iterate: Continuously enhance the system's effectiveness. 2) It’s importance and socio-economical relevance Hate text detection with BERT is crucial for: • I. Tackling Online Hate Speech: In a digital era, the rise of social media has amplified hate speech, which can incite violence and discrimination. BERT-powered detection is key to creating a safer online environment. II. Protecting Users: Online hate speech can harm mental well-being. BERT models filter out offensive content, ensuring a positive online experience. III. Fostering Inclusivity: Detecting hate text promotes inclusivity by reducing discrimination and harassment, allowing diverse voices to thrive. IV. Combating Cyberbullying: BERT helps identify and prevent cyberbullying, ensuring a healthier online environment. 3) Learning tools and their importance: • BERT Pre-trained Models: They provide contextual understanding of hate speech, improving detection accuracy. 2. Tokenization: Breaks down text into manageable units for BERT analysis. 3. Text Cleaning: Enhances data quality through normalization, stop word removal, punctuation handling, and more. 4. Fine-tuning: Adapts BERT to the specific hate text detection task, increasing accuracy by training on labeled data. 4) Team building exercises and log-book records: • Team building is essential for diverse perspectives and better solutions. We maintained a logbook in two formats: a PowerPoint presentation (PPT) and a PDF document. These logbooks served as comprehensive records of our project activities, including tasks, progress, challenges, and solutions. The PPT format allowed for dynamic visual presentations, while the PDF ensured easy access and sharing of detailed project information

5. ACTIVITY CANVAS

6. 1. All the activities involved in negative comment or hate text 1. Writing comments online; Sharing hurtful opinions through written messages, on media platforms or websites. 2. Dealing with trolling and cyberbullying; Enduring harassment, insults or threats from anonymous or identifiable individuals. 3. Facing feelings of depression and isolation; Experiencing sadness and withdrawing from interactions due to negative experiences encountered online. 4. Responding to trolls behavior; Reacting or engaging with trolls can escalate conflicts on the internet. Contribute to a perpetuation of negativity. 5. Impact on the victims family; Families may witness the distress and emotional toll inflicted upon their loved ones becoming involved in addressing the situation at hand. 6. Actions taken by platforms; Social media platforms may implement measures such as removing content issuing warnings or suspending accounts in order to address instances of harassment. 7. Support, from those not directly affected; Individuals or groups who provide assistance, empathy or intervention in order to aid victims in coping with the negativity they face online.

7. Environment canvas

8. Environment( negative & positive) 1. Toxicity on social media: A harmful digital atmosphere characterized by negativity, hostility, and offensive content. 2. Harassment: Persistent and unwanted online behavior involving threats, insults, or stalking, causing distress to the victim. 3. Isolation: A sense of loneliness and disconnection resulting from limited real-world interactions due to excessive time spent on social media. 4. Fear: Anxiety and apprehension stemming from the potential for privacy breaches, cyberattacks, or online threats. 5. Mistrust: Doubt and suspicion regarding the authenticity of information, profiles, or relationships on social media. 6. Legal consequences: Legal actions, such as lawsuits or charges, arising from inappropriate or unlawful online activities. 7. Conflict in the family: Disagreements and tension within a family unit caused by differences in online behavior or exposure to harmful content. 1 . Inclusive environment: Fosters optimism and encourages uplifting interactions. 2. Trusting environment: Promotes reliability and confidence in online relationships. 3. Empowerment environment: Supports individuals in expressing their voices and ideas freely. 4. Connected environment: Facilitates strong online bonds and a sense of community. 5. Educational environment: Emphasizes learning and sharing of knowledge. 6. Inspiring environment: Motivates creativity and aspiration among users. 7. Joyful environment: Cultivates happiness and lighthearted exchanges. 8. Constructive environment: Encourages productive discussions and problem-solving. 9. Respectful environment: Upholds courteous and considerate behavior towards others .

9. Interaction canvas

10. Interactions (related to the activities) Victim ; The person is looking for support. Documenting their experiences of being harassed. Troll; The individual keeps harassing and provoking the victim persistently. Family & Friends of the victim; They are expressing concern offering comfort and even considering reporting the harasser. Legal action or Platform moderators; They are conducting investigations. Taking actions, against the troll, which might involve involving law enforcement agencies. community supporting victims ; They are showing empathy sharing resources and advocating against harassment. Certain individuals supporting the trolls behavior ; These people are actively encouraging and amplifying trolling actions thus creating an environment online.

11. Object canvas

12. Objects (related to the activities) 1. Photos: Visual content that can be manipulated or shared to perpetrate online harassment. 2. Videos: Multimedia content that may be edited or distributed to harass or defame individuals. 3. Likes: Positive or negative reactions to posts that can be used to target or support harassment. 4. Comments: Messages that can contain abusive language or threats when used for online harassment. 5. Tweets or Retweets: Short messages that can amplify harassment when shared or targeted at victims. 6. Profile: Personal information that can be exploited to harass individuals online. 7. Groups or Pages: Online communities that may encourage or discourage harassment. 8. Emojis : Symbols that can convey emotions or tone, including those used to harass or taunt. 9. Hashtags: Keywords that can be used to organize or amplify harassment campaigns. 10. Stories or Status: Updates that can be exploited for harassment or to document abuse. 11. Notifications: Alerts that may signal harassment or support for victims. 12. Private Messages or DMs: Direct messages that can be used for private harassment or support. 13. Analytics: Data that can be used to track the impact of harassment campaigns. 14. Settings: Controls that allow users to manage privacy and security in the context of online harassment.

13. User canvas

14. User Children : Children are particularly vulnerable to online harassment due to their limited understanding of online risks. They may not have the emotional maturity to handle harassment, which can have long-term psychological effects. Teenagers : Teenagers are frequent users of social media and are often targets of online harassment, which can include cyberbullying, peer pressure, or image-based abuse. They may experience emotional distress and self-esteem issues as a result. Adults : Adults can also be victims of online harassment, with issues ranging from cyberbullying to defamation. They may have more experience in dealing with such situations but can still suffer emotional and professional consequences. Old People : Older users may be less familiar with online platforms and may struggle to identify and respond to harassment effectively. They can also be targeted for scams or fraud. News Media Agency : News agencies can face harassment in the form of misinformation campaigns, threats, and trolling by individuals or groups with different political or ideological views. This can affect their credibility and journalists' mental health. Educational Institution : Educational institutions, including students and faculty, may face harassment that targets their reputation or pedagogical goals. This can impact the learning environment and the institution's image. Cyberbullies : Cyberbullies are individuals who engage in online harassment, often targeting others with hurtful comments, threats, or personal attacks. They can have a severe impact on victims' mental and emotional well-being. Ignorant User : Ignorant users may inadvertently engage in online harassment due to a lack of awareness about online etiquette or the impact of their comments. Education and awareness can help mitigate this behavior. Casual Users : Casual users who are not public figures may encounter harassment on a personal level, affecting their online experience and potentially leading to social isolation Social Activist : Social activists often face harassment as a consequence of their advocacy work, including threats, doxing, or smear campaigns. This can undermine their cause but may also galvanize support. Content Creator : Content creators, such as YouTubers and bloggers, may experience harassment if their content is controversial or polarizing. This can harm their online presence and income. Job Promoter : Professionals who use social media for career advancement may be targeted by online harassment campaigns, potentially jeopardizing their job prospects and reputation. Multinational Company : Large companies may experience harassment in the form of boycotts, negative PR campaigns, or hacktivist attacks. Online harassment can impact their stock prices and public image.

15. Mind mapping

16. Empathy canvas

17. 2. Empathy sheet (activities) 1. Writing comments online; Sharing hurtful opinions through written messages, on media platforms or websites. 2. Dealing with trolling and cyberbullying; Enduring harassment, insults or threats from anonymous or identifiable individuals. 3. Facing feelings of depression and isolation; Experiencing sadness and withdrawing from interactions due to negative experiences encountered online. 4. Responding to trolls behavior; Reacting or engaging with trolls can escalate conflicts on the internet. Contribute to a perpetuation of negativity. 5. Impact on the victims family; Families may witness the distress and emotional toll inflicted upon their loved ones becoming involved in addressing the situation at hand. 6. Actions taken by platforms; Social media platforms may implement measures such as removing content issuing warnings or suspending accounts in order to address instances of harassment. 7. Support, from those not directly affected; Individuals or groups who provide assistance, empathy or intervention in order to aid victims in coping with the negativity they face online.

18. Empathy SHEET (User) Children : Children are particularly vulnerable to online harassment due to their limited understanding of online risks. They may not have the emotional maturity to handle harassment, which can have long-term psychological effects. Teenagers : Teenagers are frequent users of social media and are often targets of online harassment, which can include cyberbullying, peer pressure, or image-based abuse. They may experience emotional distress and self-esteem issues as a result. Adults : Adults can also be victims of online harassment, with issues ranging from cyberbullying to defamation. They may have more experience in dealing with such situations but can still suffer emotional and professional consequences. Old People : Older users may be less familiar with online platforms and may struggle to identify and respond to harassment effectively. They can also be targeted for scams or fraud. News Media Agency : News agencies can face harassment in the form of misinformation campaigns, threats, and trolling by individuals or groups with different political or ideological views. This can affect their credibility and journalists' mental health. Educational Institution : Educational institutions, including students and faculty, may face harassment that targets their reputation or pedagogical goals. This can impact the learning environment and the institution's image. Cyberbullies : Cyberbullies are individuals who engage in online harassment, often targeting others with hurtful comments, threats, or personal attacks. They can have a severe impact on victims' mental and emotional well-being. Ignorant User : Ignorant users may inadvertently engage in online harassment due to a lack of awareness about online etiquette or the impact of their comments. Education and awareness can help mitigate this behavior. Casual Users : Casual users who are not public figures may encounter harassment on a personal level, affecting their online experience and potentially leading to social isolation Social Activist : Social activists often face harassment as a consequence of their advocacy work, including threats, doxing, or smear campaigns. This can undermine their cause but may also galvanize support. Content Creator : Content creators, such as YouTubers and bloggers, may experience harassment if their content is controversial or polarizing. This can harm their online presence and income. Job Promoter : Professionals who use social media for career advancement may be targeted by online harassment campaigns, potentially jeopardizing their job prospects and reputation. Multinational Company : Large companies may experience harassment in the form of boycotts, negative PR campaigns, or hacktivist attacks. Online harassment can impact their stock prices and public image.

19. Empathy sheet (stakeholders) 1. Victims: Those directly targeted by hate text, experiencing emotional distress and potential harm. 2. Perpetrators: Individuals posting hate text, often driven by bias or hostility. 3. Social Media Platforms: Responsible for content moderation and user safety, they must address hate speech. 4. Law Enforcement: Enforce legal consequences for hate speech and threats made online. 5. Online Community: Users who may report, support, or engage with hate text, shaping its impact. 6. Advocacy Groups: Organizations working to combat hate speech, raise awareness, and provide support. 7. General Public: Society at large, influenced by and potentially affected by the presence of hate speech online.

20. Empathy sheet (happy stories) Priya , a young woman from Delhi, had faced her fair share of online harassment and hate text. Instead of succumbing to negativity, she decided to take a stand. She started a social media campaign called #SpreadLoveNotHate, encouraging individuals to share positive stories and messages to counter hate speech. Priya's initiative quickly gained traction, with people from all walks of life participating and sharing their stories of love and tolerance. It not only empowered her but also inspired countless others to combat hate with compassion. Raj, a software engineer from Bangalore, once found himself caught up in the world of online trolling. He used to engage in hate-filled discussions and post offensive comments. However, his perspective changed when he came across a campaign called #OnlineKindnessMatters. This movement, initiated by a group of Indian influencers and activists, aimed to raise awareness about the impact of online harassment. Raj decided to turn his life around and became an advocate for kindness and empathy online. He began actively reporting hate text and educating his followers about the importance of respectful dialogue. Today, Raj is a respected voice in the online community, showing that even those who once contributed to the problem can be part of the solution.

21. Empathy sheet (sad story …1) Aarav, a 25-year-old software engineer from Mumbai, had always been passionate about promoting gender equality and women's rights. He used social media platforms to engage in constructive conversations and share informative content on these subjects. However, as his following grew, he became a target for online harassment. A group of anonymous trolls began sending him hateful messages, criticizing his views and threatening him. These messages not only attacked his beliefs but also his personal life. They even went as far as revealing sensitive information about his family. Aarav tried to ignore the hate at first, but it continued to escalate. He reported the harassment to the social media platform, but it took several weeks for any action to be taken. During this time, the constant barrage of hate took a toll on his mental health. He began to experience anxiety and depression, affecting his work and personal life. Aarav's story is a heart-wrenching example of the devastating impact online harassment can have on an individual's well-being and mental health.

22. Empathy sheet (sad story …2) Parthiv , a passionate young writer, once shared his thoughts and creative work on social media with the hope of inspiring others. However, what started as an avenue for self-expression soon turned into a nightmare. He began receiving a barrage of hate-filled comments and derogatory messages from faceless trolls who took issue with his ideas and background. The relentless online harassment took a toll on his mental health, leaving him feeling isolated and vulnerable. Parthiv contemplated abandoning his passion altogether, torn between the love for his craft and the despair caused by the relentless hate text. Despite reporting the harassment to the social media platform, the attacks continued. Parthiv's online presence, once a source of joy and connection, became a source of anguish. His story is a stark reminder of the devastating impact that hate text and online harassment can have on individuals who simply seek to share their voices and creativity with the world, leaving scars that extend far beyond the digital realm.

23. Ideation canvas

24. Ideation sheet People: 1. Children and Teenagers: Vulnerable to cyberbullying, causing emotional distress and self-esteem issues. 2. Adults: Victims of defamation and emotional consequences. 3. Older People: Struggle to identify and respond to harassment, susceptible to scams. 4. News Media Agencies: Targeted by misinformation campaigns and threats. 5. Educational Institutions: Reputation and pedagogical goals impacted, affecting the learning environment. 6. Cyberbullies: Inflict mental and emotional harm through hurtful comments and threats.   Activities 1. Writing comments online; Sharing hurtful opinions through written messages, on media platforms or websites. 2. Dealing with trolling and cyberbullying; Enduring harassment, insults or threats from anonymous or identifiable individuals. 3. Facing feelings of depression and isolation; Experiencing sadness and withdrawing from interactions due to negative experiences encountered online. 4. Responding to trolls behavior; Reacting or engaging with trolls can escalate conflicts on the internet. Contribute to a perpetuation of negativity. 5. Impact on the victims family; Families may witness the distress and emotional toll inflicted upon their loved ones becoming involved in addressing the situation at hand. 6. Actions taken by platforms; Social media platforms may implement measures such as removing content issuing warnings or suspending accounts in order to address instances of harassment. 7 Support, from those not directly affected; Individuals or groups who provide assistance, empathy or intervention in order to aid victims in coping with the negativity they face online.

25. Ideation sheet Props/Tools/Objects/Equipment Photos and Videos: Manipulated visual content used for harassment. Likes and Comments: Reactions and messages supporting or targeting harassment. Tweets and Retweets: Short messages amplifying harassment. Emojis and Hashtags: Symbols and keywords conveying emotions and organizing harassment. PC: Use for in-depth research and data analysis in hate speech detection. Laptop: Convenient for on-the-go monitoring and content analysis. Tablet: Effective for presentations and educating on the impact of hate speech. Mobile: Engage users through mobile apps to report and combat hate speech Datasets: Aid in developing cyberbullying detection algorithms.

26. Ideation sheet Situation 1. Social Networking: Online interactions on platforms like Facebook. 2. Online Forums: Discussions in digital message boards. 3. Gaming Culture: Communication in online gaming. 4. News Commenting: Conversations about news articles. 5. Virtual Classrooms: Educational interactions online.

27. Ideation sheet   Context 1. Online Interaction: Communication on the internet. 2. Article Discussions: Talks related to news stories. 3. In-Game Chat: Conversations within video games. 4. Educational Context: Interactions in online learning. 5. Web-based Conversations: Dialogues on the web.  

28. Ideation sheet Location 1. Web Communities: Online community spaces. 2. News Websites: Platforms for news articles. 3. Gaming Servers: Digital hubs for gaming. 4. Learning Portals: Educational websites. 5. Digital Gathering Spots: Online places for interaction.

29. Product development canvas

30. pdc purposes 1. Hate Speech Detection 2. Content Moderation 3. Social Media Monitoring 4. Hate Speech Classification 5. Bias and Fairness Analysis

31. pdc People : Victim , Troll Supporter ( victim side) , Family and friends , Online community (on the troll side) , Social media moderator Project Features: 1. Text Classification 2. Multi-Class Classification 3. High Accuracy 4. Model Training Project Functions: 1. Classification Function 2. Training Function 3. Accuracy Evaluation 4. Data Labeling Customer revalidation : 1. User Feedback 2. User Dashboard 3. Regular Model Updates 4. User Education 5. Community Reporting 6. Opt-Out Option 7. Regular Surveys 1. *Reject:* Content that is confirmed as hate speech or consistently reported by users as offensive. 2. *Retain:* Content that the system accurately identifies as hate speech and content that users agree with the system's classifications. 3. *Redesign:* Content that users often report as wrongly classified or if the system is making many mistakes, indicating the need to improve the model.

32. pdc Components: Photos and Videos: Manipulated visual content used for harassment. Likes and Comments: Reactions and messages supporting or targeting harassment. Tweets and Retweets: Short messages amplifying harassment. Emojis and Hashtags: Symbols and keywords conveying emotions and organizing harassment. PC: Use for in-depth research and data analysis in hate speech detection. Laptop: Convenient for on-the-go monitoring and content analysis. Tablet: Effective for presentations and educating on the impact of hate speech. Mobile: Engage users through mobile apps to report and combat hate speech Datasets: Aid in developing cyberbullying detection algorithms.

33. Design Calculations 1. Use case diagram:

34. Design calculation Er diagram

35. Design calculTION Class diagram:

36. DESIGN CALCULATIONS Activity diagram:

37. DESIGN CALCULATION Sequence diagram:

38. DESIGN CALCULATIONS DFD diagram: LEVEL 0

39. DESIGN CALCULATIONS DFD diagram: LEVEL 1

40. Implementation/Prototype FOR XLNET MODEL

41. SIMILAR IMPLEMENTATION SCREENSHOT ARE IN THE REPORT HAVING FINAL PROBABILITY OF THE MODELS

42. PROTOTYPE

43. CONCLUSION In this report, we have implemented a hate text dataset as part of a system aimed at detecting hate speech. Prior to the implementation of the BERT model, we focused on curating and preparing the dataset for training. The dataset plays a crucial role in the effectiveness and accuracy of hate text detection systems. To ensure a comprehensive and diverse dataset, we collected a wide range of texts that encompassed different forms of hate speech, including derogatory language, offensive remarks, and discriminatory comments. Careful consideration was given to include various contexts and domains to make the dataset representative of realworld scenarios. Data preprocessing was an essential step in preparing the dataset for training. We cleaned the text by removing any irrelevant or unnecessary information, and performed tasks such as tokenization, lowercasing, and removing stop words. Additionally, we balanced the dataset to avoid any class imbalance issues, ensuring an equal representation of hate speech and non-hate speech examples. By following these meticulous steps, we established a highquality dataset that serves as the foundation for training a hate text detection system. The dataset provides a reliable and diverse set of examples, enabling the subsequent model to learnpatterns and characteristics of hate speech effectively. The successful implementation of the hate text dataset lays the groundwork for the subsequent integration of the BERT model. By leveraging the power of BERT, a state-of-the-art language model, we anticipate enhanced hate text detection capabilities. The utilization of BERT's contextualized word representations and its ability to capture intricate semantic nuances will enable the system to better identify and classify hate speech. In conclusion, the implementation of the hate text dataset is a critical step towards developing a robust hate speech detection system. The careful curation and preprocessing of the dataset provide a solid foundation for training subsequent models. With the dataset in place, we are now well-equipped to proceed with the integration of the BERT model and further advancements in hate text detection technology

44. REFERNCE Badam , J., Bonagiri , A., Raju, K., Chakraborty, D., 2022. Aletheia : A fake newsdetection system for Hindi. In: 5th Joint International Conference on Data Science &Management of Data (9th ACM IKDD CODS and 27th COMAD). In: CODS-COMAD2022, Association for Computing Machinery, New York, NY, USA, pp. 255–259.http://dx.doi.org/10.1145/3493700.3493736.Bhardwaj, M., Akhtar, M.S., Ekbal , A., Das, A., Chakraborty, T., 2020. Hostilitydetection dataset in Hindi. arXiv:2011.03588.Boughorbel, S., Jarray , F., El- Anbari , M., 2017. Optimal classifier for imbalanced datausing Matthews Correlation Coefficient metric. PLoS One 12 (6), e0177678. Briskilal , J., Subalalitha , C., 2022. An ensemble model for classifying idioms and literaltexts using BERT and RoBERTa . Inf. Process. Manage. 59 (1), 102756. http://dx.doi.org/10.1016/j.ipm.2021.102756, URL https://www.sciencedirect.com/science/article/pii/S0306457321002375.Brown, G., Kuncheva , L.I., 2010. ‘‘good’’ and ‘‘bad’’ diversity in majority vote ensembles . In: International Workshop on Multiple Classifier Systems. Springer, pp.124–133.Chicco, D., Jurman , G., 2020. The advantages of the matthews correlation coefficient(MCC) over F1 score and accuracy in binary classification evaluation. BMCGenomics 21 (1), 1–13.