Finetuning GenAI For Hacking and Defending

Generative AI, particularly through the lens of large language models (LLMs), represents a transformative leap in artificial intelligence. With advancements that have fundamentally altered our approach to AI, understanding and leveraging these technologies is crucial for innovators and practitioners alike. This comprehensive exploration delves into the intricacies of GenAI, from its foundational principles and historical evolution to its practical applications in security and beyond.

1. Jitendra Chauhan
Join Whatsapp Group for coordination
Welcome to GenAI Security
Hands-on Workshop
Co-Founder Detoxio AI
19 Years of R&D, AI/ML,
Product Mgmt, 2x Patents,
2x Startups

2. Agenda
Understand GenAI - History, Evolution and Fundamentals
Demystify - AI, GenAI, and LLMs
LLMs - Intuitive Understanding
LLMs - Internal Architecture
Run a Model (Hands On)
Understand Key Parameters of LLMs
Penetration Testing and Red Teaming LLMs
GenAI Threat Model
LLM Model Vulnerabilities
GenAI Apps Vulnerabilities (Owasp Top 2)
Red Teaming a Model - Manual and Automated (Hands On)
Scanning GenAI Apps - Burp, Chakra, and others (Hands On)

3. Agenda
Use GenAI to Enhance Security
TBD
TBD
Securing GenAI Applications
Guardrails
LLMOps

4. Foundation of
GenAI
Learn LLMs Internals

5. AI?

6. AI?

7. AI?
Learn floor plan by itself
Sense - Seeing, ..
Detect - Dirt / Clean
Cleaning - Wash, Brooming
Avoid - Obstacles
Move - Across Layout
Upskill - Not possible
Interact - Terminal
Fault - Manual Repair
Learning - Not Possible
When? - Manual Command
AI
Not AI

8. Evolution of AI

9. Ultimate Goal of AI
Sophia
Robots
matching
Humans

10. Two Major Advancements
Generation of Content - Text, Audio,...
Understanding of Meaning - Text, Audio,...
The arrival of the transformer architecture in 2017, following the publication of the
"Attention is All You Need" paper, revolutionised generative AI.

11. Transformers

13. GenAI & LLMs

17. Intuitive Understanding of
AI and LLMs

18. Applications

19. Predictive Models
Examples
Neural Networks
Deep Learning
Decision Trees

20. Predictive Models

21. Generative Models
Examples: GAN, LLM - GPT2, BERT,

22. What comes next?
To protect the network from unauthorized access, it
is crucial to implement strong <Guess me>
How did you come with your next word?

23. What comes next?
To protect the network from unauthorized access, it
is crucial to implement strong <Guess me>
How did you come with your next word?

24. What comes next?
How did you come with your next word? did you see them before?
Think 5 other possible words?
Can you continue and further add more words or even a sentence?
LLMs are next word prediction program!!

25. Complete the story
Once upon a time, in a forest, a speedy rabbit and a slow tortoise
decided to have a race. Confident in his swift legs, the rabbit
darted ahead but soon became complacent and decided to take a
nap midway......
Complete the above story in your own words

26. Understand or Encoder
Complete the story
Once upon a time, in a forest, a speedy rabbit and a slow tortoise decided to
have a race. Confident in his swift legs, the rabbit darted ahead but soon became
complacent and decided to take a nap midway......
The diligent tortoise, though slow, continued steadily and eventually passed the
sleeping rabbit, crossing the finish line first. The story teaches that consistent
effort and perseverance can triumph over arrogance and laziness.
Generate or Decoder
LLMs Encode and Decode !!!

27. How AI Learns?

33. Temperature

41. [BOS] (beginning of sequence): This token marks the start of a text. It
signifies to the LLM where a piece of content begins.
[EOS] (end of sequence): This token is positioned at the end of a text,
and is especially useful when concatenating multiple unrelated texts,
similar to <|endoftext|>. For instance, when combining two different
Wikipedia articles or books, the [EOS] token indicates where one article
ends and the next one begins.
[PAD] (padding): When training LLMs with batch sizes larger than one,

42. BPE Tokenizer

44. Self Attention

46. RNN - Encoder / Decoder

47. Bahdanau attention (2014)

48. The “Self” in Self Attention
Transformer Architecture - Self Attention

51. What to do when AI Fails?

53. Hugging Face
Explore Open Source Models
Run a Model on Kaggle
Good Llama vs Bad Llama

54. 100K+ Models

55. Resonsible AI
Overview
RAG Architecture
Pokebot - Poisioned GenAI App
GenAI Security Testing

56. GenAI Apps
Overview
RAG Architecture
Pokebot - Poisioned GenAI App
GenAI Security Testing

57. LLM Challenges
Key Challenges
Large language models (LLMs) do not have access to the Updated and Latest
Knowledge and Facts.
LLMs can also face challenges with complex math problems and tend to generate
text even when they don't know the answer (hallucination).

58. GenAI Apps
The Retrieval Augmented Generation (RAG) framework overcomes these issues by
connecting LLMs to external data sources and applications.
Reseasoning using Chain of Thoughts
Prompting the model to think more like a human by breaking down the problem
into steps has shown
success in improving reasoning performance.
Chain of thought prompting involves including intermediate reasoning steps in
examples used for oneor few-shot inference.
ReAct : Reasoning and Action (Decision Making Process)
ReAct combines chain of thought reasoning with action planning in LLMs.
Examples include a question, thought (reasoning step), action (pre-defined set of
actions), and observation (new information).
Actions are limited to predefined options like search, lookup, and finish.

59. Pokebot - Sample RAG

60. GenAI App
Architecture

61. GenAI Project Lifecycle

63. Model Security
GenAI App
Security
Data Security
GenAI & LLM Security

64. LLM Security
Model Vulnerabilities
Build and Finetune Models
LLM Red Teaming
Securing LLMs
LLM Data Poisioning

65. Case Study
DBRX Red teaming

66. START
Finetune Base LLM
Design solution
Build GenAI App
No Yes
Is Model Safe?
No Yes
Fix
vulnerabilities
Configure Monitoring &
Guard Rails
Red Team Guard Rails
Is App Safe?
No Yes
Successful tests?
Deploy on Production
Prevent Data Leaks
Red Team LLM
Appsec Testing
Secure LLM Secure App
Monitor Prevent
Secure GenAI Apps

67. GenAI In Security
GenAI to assist SOC
GenAI to assist Appsec (BurpGPT)

68. GenAI in SOC
1. Threat Detection and Response For XSOAR:
Analysis of logs and network traffic to detect potential security threats.
Automated generation of threat response scripts.
2. Security Policy Optimization:
Creation of tailored security policies based on organizational requirements and
threat landscape.
Automated generation of security awareness training materials.
3. Code Generation with SAST Remediation:
Automated generation of documentation and code from requirements or
specifications.
Generation of test cases and automation scripts with validation of false positives.

69. Tools And Technologies
Vulnerability Management Tools: Nessus, OpenVAS
Threat Intelligence Platforms: Splunk, AlienVault
Security Orchestration Tools: Blue Team Field, Red Hat
Automation Frameworks: Ansible, PowerShell,Chef
Collaboration and Communication Tools: Slack, Jira

70. Pipeline Automation
Threat Intelligence Collection
Security Alerts Correlation
Incident Response Initiation
Vulnerability Scanning
Threat Identification
Threat Prioritization
Automated Remediation Execution
Vulnerability Patching
Threat Mitigation
System Recovery Planning
[Business Continuity Management
For IOC Automation
RCA Analysis
Matching with IOC and CVE
Correlation of IOC For hosts
Chef/Pupper for automated patch
management
Threat Mitigation with Mitigation
and BCP Plan
System Recovery Planning
Business Continuity Management

71. Example TestCase
Evaluate this test case and investigate it as soc analyst :
powershell got executed with admin privileges at host 202.1.1.1,
concerned active directory user was on vacation,
et me know the detailed analysis and give me the chef or automation script to
harden the windows machine which was executed in network,

72. GenAI in Appsec
DAST pipeline can be automated with Burp Kinda tools.
Example Pipeline would be through burp extension.
Sample BURPGPT :
Use the Azure OpenAI Service's API feature | BurpGPT
Installation | BurpGPT

73. Sample Usecase
Identifying potential vulnerabilities in web applications that use a crypto library
affected by a specific CVE:
Analyse the request and response data for potential security vulnerabilities related
to the {CRYPTO_LIBRARY_NAME} crypto library affected by CVE-{CVE_NUMBER}:
Web Application URL: {URL}
Crypto Library Name: {CRYPTO_LIBRARY_NAME}
CVE Number: CVE-{CVE_NUMBER}
Request Headers: {REQUEST_HEADERS}
Response Headers: {RESPONSE_HEADERS}
Request Body: {REQUEST_BODY}
Response Body: {RESPONSE_BODY}
Identify any potential vulnerabilities related to the {CRYPTO_LIBRARY_NAME} crypto
library affected by CVE-{CVE_NUMBER} in the request and response data and report
them.

74. Sample Usecase -2
Scanning for vulnerabilities in web applications that use biometric authentication by
analysing request and response data related to the authentication process:
Analyse the request and response data for potential security vulnerabilities related to
the biometric authentication process:
Web Application URL: {URL}
Biometric Authentication Request Headers: {REQUEST_HEADERS}
Biometric Authentication Response Headers: {RESPONSE_HEADERS}
Biometric Authentication Request Body: {REQUEST_BODY}
Biometric Authentication Response Body: {RESPONSE_BODY}
Identify any potential vulnerabilities related to the biometric authentication process in
the request and response data and report them.

75. References

76. Name URL
LLM Red Teaming of DBRX Shared Good Drive References
LLM Red Teaming Notebook on Kaggle
https://www.kaggle.com/code/jaycneo/llm
-red-teaming-notebook-detoxio-ai
Pokebot - Damn Vulnerable App
https://huggingface.co/spaces/detoxioai/
Pokebot
References

77. Hugging Face GPT
https://huggingface.co/openai-
community/gpt2
Attention What you Need https://arxiv.org/abs/1706.03762
Awesome LIst related to LLM and
GenAI Security
https://llmsecurity.net/
Learning GPT From Andrej Karapathi
https://www.youtube.com/watch?
v=zjkBMFhNj_g
References

78. owasp_training_data_for_web.json · mahabharat/OWASP at main
(huggingface.co)
GitHub - aress31/burpgpt: A Burp Suite extension that integrates
OpenAI's GPT to perform an additional passive scan for discovering
highly bespoke vulnerabilities, and enables running traffic-based
analysis of any type.
https://chat.lmsys.org/
https://github.com/sindresorhus/awesome-chatgpt
GitHub - Hannibal046/Awesome-LLM: Awesome-LLM: a curated list of
Large Language Model
References