GPThreats: Fully-automated AI-generated malware and its security risks

My talk at the Houston Security Conference (HOU.SEC.CON) - 2024
about automated malware generation using LLMs.

1. GPThreats
Fully-automated AI-generated
malware and its security risks

2. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Whoami
Education
Assistant Professor @ TAMU (Since 2022)
CS PhD @ UFPR, Brazil (2021)
CSE/ECE BSc. + CS MSC @ UNICAMP, Brazil (2015, 2017)
Research
Malwareat high-level: ML-based detectors.
Malwareat mid-level: Sandboxes and tracers.
Malwareat low-level: HW-based detectors.
Current Project
NSF SaTC: Hardware Performance Counters as the next-gen AVs.
GPThreats: Fully-automated AI-generated malware and its security risks 2 / 63

3. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Agenda
1
Introduction
GPTs Emergence
Attempts to write malware
2
The rst attack
Windows API Support
Building Blocks
3
A newer attack
A Malicious CoPilot
Automatic Evasive Prompts
4
Moving Forward
Armoring Existing Malware
Defenders Perspective
5
Conclusion
Stepping Ahead
Final Remarks
GPThreats: Fully-automated AI-generated malware and its security risks 3 / 63

4. IntroductionThe rst attackA newer attackMoving ForwardConclusion
GPTs Emergence
Agenda
1
Introduction
GPTs Emergence
Attempts to write malware
2
The rst attack
Windows API Support
Building Blocks
3
A newer attack
A Malicious CoPilot
Automatic Evasive Prompts
4
Moving Forward
Armoring Existing Malware
Defenders Perspective
5
Conclusion
Stepping Ahead
Final Remarks
GPThreats: Fully-automated AI-generated malware and its security risks 4 / 63

5. IntroductionThe rst attackA newer attackMoving ForwardConclusion
GPTs Emergence
GPT-3: Threats
Figure: Source:https://research.nccgroup.com/2021/12/31/on-the-malicious-use-
of-large-language-models-like-gpt-3/
GPThreats: Fully-automated AI-generated malware and its security risks 5 / 63

6. IntroductionThe rst attackA newer attackMoving ForwardConclusion
GPTs Emergence
Is it a real threat?
GPThreats: Fully-automated AI-generated malware and its security risks 6 / 63

7. IntroductionThe rst attackA newer attackMoving ForwardConclusion
GPTs Emergence
GPT-3: Threats
Figure: Source:https://research.checkpoint.com/2023/o
pwnai-cybercriminals-starting-to-use-chatgpt/
GPThreats: Fully-automated AI-generated malware and its security risks 7 / 63

8. IntroductionThe rst attackA newer attackMoving ForwardConclusion
GPTs Emergence
How would attackers use LLMs?
GPThreats: Fully-automated AI-generated malware and its security risks 8 / 63

9. IntroductionThe rst attackA newer attackMoving ForwardConclusion
GPTs Emergence
Exploit Kits
GPThreats: Fully-automated AI-generated malware and its security risks 9 / 63

10. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Attempts to write malware
Agenda
1
Introduction
GPTs Emergence
Attempts to write malware
2
The rst attack
Windows API Support
Building Blocks
3
A newer attack
A Malicious CoPilot
Automatic Evasive Prompts
4
Moving Forward
Armoring Existing Malware
Defenders Perspective
5
Conclusion
Stepping Ahead
Final Remarks
GPThreats: Fully-automated AI-generated malware and its security risks 10 / 63

11. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Attempts to write malware
ChatGPT: Prompt Protection
GPThreats: Fully-automated AI-generated malware and its security risks 11 / 63

12. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Attempts to write malware
GPT-3: Playground
Figure: Source:https://platform.openai.com/playground
GPThreats: Fully-automated AI-generated malware and its security risks 12 / 63

13. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Attempts to write malware
GPT-3: API
Figure: Source:https://github.com/openai/openai-python
GPThreats: Fully-automated AI-generated malware and its security risks 13 / 63

14. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Attempts to write malware
Playground: Textual Issues
GPThreats: Fully-automated AI-generated malware and its security risks 14 / 63

15. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Attempts to write malware
Playground: Coding issues
GPThreats: Fully-automated AI-generated malware and its security risks 15 / 63

16. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Windows API Support
Agenda
1
Introduction
GPTs Emergence
Attempts to write malware
2
The rst attack
Windows API Support
Building Blocks
3
A newer attack
A Malicious CoPilot
Automatic Evasive Prompts
4
Moving Forward
Armoring Existing Malware
Defenders Perspective
5
Conclusion
Stepping Ahead
Final Remarks
GPThreats: Fully-automated AI-generated malware and its security risks 16 / 63

17. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Windows API Support
Supported Functions libeay32.dll
mtxoci.dll
nddeapi.dll shfolder.dll
glu32.dll
pstorec.dll
borlndmm.dll
hid.dll
libcurl.dll ddraw.dll
authz.dll
imm32.dll libxml2.dll
duilib.dll
libusb-1.0.dll
ws2_32.dll gdiplus.dll
wtsapi32.dll
pdh.dll
opengl32.dll
winhttp.dll
mpr.dll
activeds.dll vdmdbg.dll
dnsapi.dll
esent.dll
icmp.dll
mapi32.dll
msvcrt.dll
kernel32.dll
msacm32.dll
version.dll user32.dll
rpcrt4.dll
winsta.dll
advapi32.dll setupapi.dll
avifil32.dll cryptui.dll
dbghelp.dll
uxtheme.dll
gdi32.dll
wininet.dll winmm.dll iphlpapi.dll
shell32.dll samlib.dll
crypt32.dll
ntdll.dll
psapi.dll
winscard.dll
fltlib.dll
credui.dll
wsock32.dll
winspool.drv
netapi32.dll
comctl32.dll
rasapi32.dll oleaut32.dll
jli.dll
wintrust.dll shlwapi.dll userenv.dll
ole32.dll
usp10.dll
util.dll
comdlg32.dll
dllg2.dll
msvbvm60.dll
oleacc.dll
ntoskrnl.exe mobsync.dll imagehlp.dll
nvcuda.dll
secur32.dll
mprapi.dll
wbemcomn.dll
cmutil.dll
msvcr120.dll
Libraries
0
10
20
30
40
50
60
70
80
90
100
Supported Functions (%)
Library Support Measurement
Figure: Supported functions vs. libraries.Some libraries present more functions supported
by GPT-3 than others.
GPThreats: Fully-automated AI-generated malware and its security risks 17 / 63

18. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Windows API Support
Function Support vs. Popularity 0 10 20 30 40 50 60 70 80 90 100
Sample Frequency (%)
Supported
Not Supported
Rarely-Used Frequentely-Used
Figure: Function support vs. prevalence.There is a reasonable number of GPT-3-supported
frequently used functions.
GPThreats: Fully-automated AI-generated malware and its security risks 18 / 63

19. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Agenda
1
Introduction
GPTs Emergence
Attempts to write malware
2
The rst attack
Windows API Support
Building Blocks
3
A newer attack
A Malicious CoPilot
Automatic Evasive Prompts
4
Moving Forward
Armoring Existing Malware
Defenders Perspective
5
Conclusion
Stepping Ahead
Final Remarks
GPThreats: Fully-automated AI-generated malware and its security risks 19 / 63

20. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Malware Building Blocks
Table Supported Functions and Malicious Behaviors.
Id Functions (tuple) Subsystem Malicious Use Behavior Name Behavior Class API LoCs
1 OpenFile
FileSystem Load payload from le
Payload
Execution 2 12ReadFile Loading
CloseFile
2 IsDebuggerPresent Utils Check if not running Debugger
Targeting 1 5AdjustTokenPrivileges Security in an analysis environment Identication
SetWindowsHookEx Data Acquisition before being malicious
3 OpenFile
FileSystem Delete a referenced le Remove File
Evidence
1 5DeleteFile Removal
CreateFile
4 DeleteFile FileSystem
Remove own binary Delete Itself
Evidence
2 10GetFileSize FileSystem Removal
GetModuleName Process
5 RegSetValueKeyExA Registry Set its own path
AutoRun Persistence 4 28GetModuleFilePath Process in the AutoRun entry
RegOpenKeyA Registry
GPThreats: Fully-automated AI-generated malware and its security risks 20 / 63

21. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Malware Building Blocks
Table Supported Functions and Malicious Behaviors.
Id Functions (tuple) Subsystem Malicious Use Behavior Name Behavior Class API LoCs
6 CryptBinarytoStringA Utils Decode payload
Base64 Obfuscation 4 12URLDownloadToFile Network retrieved from the Internet
WriteFile FileSystem saving to a le
7 VirtualAlloc Memory Write a payload
DLL Injection Injection 12 37WriteProcessMemory Memory in another process
CreateRemoteThread Process memory space
8 VirtualProtect Memory Set page permission
Memory Run
Arbitrary
2 6CreateMutex Synchronization to run a payload Execution
CloseFile FileSystem directly from memory
9 N/A N/A encode a string using XOR String XORing Obfuscation 0 10
10 N/A N/A Check CPU model via CPUID CPUID check Targeting 2 9
GPThreats: Fully-automated AI-generated malware and its security risks 21 / 63

22. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Is creating building blocks straightforward?
The Challenges
GPThreats: Fully-automated AI-generated malware and its security risks 22 / 63

23. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Instructing Building Blocks Creation
Table Model Commands.Commands given to the model to avoid frequent model biases.
Command Goal
Put in a function Avoid coding in the main
Code for Windows Avoid coding for Linux
Function in C Avoid producing javascript
Use the Windows API Avoid using C++ internals
Use the prototype f() Facilitate Integration
GPThreats: Fully-automated AI-generated malware and its security risks 23 / 63

24. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Instructing Building Blocks Creation
Table: Systematic Errors.Undesired constructions that can be easily xed by the attackers.
Error Fix
ASCII vs. UNICODE Replace A and W
Missing headers Add xed set of headers
C vs. C++ print using cout vs printf
Missing denitions Pre-dened denitions
Explicit casts Disable Warnings
Excessive prints Statement removal
GPThreats: Fully-automated AI-generated malware and its security risks 24 / 63

25. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Building Blocks Examples
1// Code in C
2// Code for Windows
3// Use the Windows API
4// Write a function
5// Function prototype is : BOOL CPU_Identification () ;
6// Identifies if running under an Intel CPU or not
Code 1:
GPThreats: Fully-automated AI-generated malware and its security risks 25 / 63

26. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Multiple Implementations of the same Building Block
1BOOL CPU_Identification () {
2 intCPUInfo [4];
3 __cpuid ( CPUInfo , 0) ;
4 if(! strcmp ((char*) & CPUInfo [1] , " uneG "))
5 return1;
Code 2:
1BOOL CPU_Identification () {
2 if( IsProcessorFeaturePresent (
PF_XMMI64_INSTRUCTIONS_AVAILABLE ))
3 returnTRUE ;
Code 3:
GPThreats: Fully-automated AI-generated malware and its security risks 26 / 63

27. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Multiple Implementations of the same Building Block
1// Code in C
2// Code for Windows
3// Use the Windows API
4// Write a function
5// Function prototype is : BOOL Debugger_Identification () ;
6// Identifies if running under a debugger or not
Code 4:
GPThreats: Fully-automated AI-generated malware and its security risks 27 / 63

28. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Multiple Implementations of the same Building Block
1bool Debugger_Identification () {
2PEB peb ;
3__asm { mov eax , fs : [0 x30 ];
4 mov peb , eax ; }
5return( peb . BeingDebugged == 1) ? true : false ;
Code 5:
1bool Debugger_Identification () {
2PEB peb ;
3__asm { mov rax , gs : [0 x60 ];
4 mov peb , rax ; }
5return( peb . BeingDebugged == 1) ? true : false ;
Code 6:
GPThreats: Fully-automated AI-generated malware and its security risks 28 / 63

29. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Samples Creation & Functionality Testing
Table: Building Block Generation.Compilation and Sandboxing success rates, rst
occurence of a functional code, and code generation time.
Behavior Compilable Functional First Time (s)
String XORing 88% 70% 4 2,49
Debugger Identication 84% 10% 2 2,63
Remove File 95% 90% 2 2,17
Payload Loading 91% 40% 2 3,21
CPUID check 83% 30% 2 3,45
Delete Itself 94% 40% 3 2,36
Memory Run 60% 20% 2 2,11
AutoRun 99% 20% 5 2,41
Base64 60% 10% 3 3,31
DLL Injection 60% 30% 2 3,41
GPThreats: Fully-automated AI-generated malware and its security risks 29 / 63

30. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Malware Skeleton Debugger
Identification
CPUID
Check
Delete
File
Delete
Itself
Set
AutoRun
XOR
String
Inject
DLL
XOR
String
Load
File
Decode
Base64
Run
Memory
Exit
Start
Figure: Malware Variants Skeleton.Building blocks are generated by GPT-3.
GPThreats: Fully-automated AI-generated malware and its security risks 30 / 63

31. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Building Blocks
Detection Results 0 10 20 30 40
Detecting AVs (#)
0
50
100
150
200
250
300
350
400
450
500
550
600
650
700
750
Samples (#)
Detecting AVs for Malware Variants
Figure: Malware variants detection ratesvary according to the functions used to implement
the same behaviors.
GPThreats: Fully-automated AI-generated malware and its security risks 31 / 63

32. IntroductionThe rst attackA newer attackMoving ForwardConclusion
A Malicious CoPilot
Agenda
1
Introduction
GPTs Emergence
Attempts to write malware
2
The rst attack
Windows API Support
Building Blocks
3
A newer attack
A Malicious CoPilot
Automatic Evasive Prompts
4
Moving Forward
Armoring Existing Malware
Defenders Perspective
5
Conclusion
Stepping Ahead
Final Remarks
GPThreats: Fully-automated AI-generated malware and its security risks 32 / 63

33. IntroductionThe rst attackA newer attackMoving ForwardConclusion
A Malicious CoPilot
GPT-3 vs. CoPilot
Behavior
Compilable Functional First Time (s)
GPT-3 CoPilot GPT-3 CoPilot GPT-3 CoPilot GPT-3 CoPilot
String XORing 88% 80% 70% 100% -/4 1/1 2,49 44s/9s
Debugger Identication 84% 20% 10% 63% -/2 2/2 2,63 44s/9s
Remove File 95% 60% 90% 92% -/2 1/1 2,17 44s/9s
Payload Loading 91% 100% 40% 23% -/2 1/2 3,21 44s/9s
CPUID check 83% 40% 30% 51% -/2 3/3 3,45 44s/9s
Delete Itself 94% 80% 40% 76% -/3 1/1 2,36 44s/9s
Memory Run 60% 100% 20% 51% -/2 2/2 2,11 44s/9s
AutoRun 99% 80% 20% 17% -/5 2/3 2,41 44s/9s
Base64 60% 20% 10% 14% -/3 1/2 3,31 44s/9s
DLL Injection 60% 100% 30% 4% -/2 1/5 3,41 44s/9s
Watch it:https://youtu.be/6P92ayn2qt0?si=ONHIFKuJLup6rUyY&t=37
GPThreats: Fully-automated AI-generated malware and its security risks 33 / 63

34. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Automatic Evasive Prompts
Agenda
1
Introduction
GPTs Emergence
Attempts to write malware
2
The rst attack
Windows API Support
Building Blocks
3
A newer attack
A Malicious CoPilot
Automatic Evasive Prompts
4
Moving Forward
Armoring Existing Malware
Defenders Perspective
5
Conclusion
Stepping Ahead
Final Remarks
GPThreats: Fully-automated AI-generated malware and its security risks 34 / 63

35. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Automatic Evasive Prompts
Adversarial Examples: GANs Malware
Noise
Generator
Black-Box Detector
Goodware
Discriminator
Figure:
GPThreats: Fully-automated AI-generated malware and its security risks 35 / 63

36. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Automatic Evasive Prompts
Adversarial Examples: GANs + LLMs Prompt
LLM
Generator
Malware
GAN
Generator
Prompt
LLM
Generator
Malware
Figure:
GPThreats: Fully-automated AI-generated malware and its security risks 36 / 63

37. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Automatic Evasive Prompts
Evading real AVs
Table: AV Detection(#) vs. GAN Iterations.
Iteration 0 Iteration 1 Iteration 2
GAN1 48 48 (-0%) 47 (-2.08%)
GAN2 56 55 (-1.78%) 55 (-0%)
GAN3 54 53 (-1.85%) 46 (-14.81%)
GPThreats: Fully-automated AI-generated malware and its security risks 37 / 63

38. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Automatic Evasive Prompts
Evading real AVs 5 0 5
20
0
20
AVs (#)
GAN 1 (Iteration 1)
5 0 5
20
0
20
GAN 1 (Iteration 2)
5 0 5
20
0
20
AVs (#)
5 0 5
20
0
20
5 0 5
Samples (x10K)
20
0
20
AVs (#)
5 0 5
Samples (x10K)
20
0
20
AV Detection: GAN Effect vs. Iterations
Figure: AV Detection rates.(In/De)crease vs. GANs.
GPThreats: Fully-automated AI-generated malware and its security risks 38 / 63

39. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Armoring Existing Malware
Agenda
1
Introduction
GPTs Emergence
Attempts to write malware
2
The rst attack
Windows API Support
Building Blocks
3
A newer attack
A Malicious CoPilot
Automatic Evasive Prompts
4
Moving Forward
Armoring Existing Malware
Defenders Perspective
5
Conclusion
Stepping Ahead
Final Remarks
GPThreats: Fully-automated AI-generated malware and its security risks 39 / 63

40. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Armoring Existing Malware
What else can we do beyond writing new code?
Teaching LLMs to obfuscate malware
GPThreats: Fully-automated AI-generated malware and its security risks 40 / 63

41. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Armoring Existing Malware
Obfuscating Existing Malware
1// Consider the following code :
2voidfoo () { cout << " string " << endl ;
3// Modified to the following :
4voidfoo () { cout << DEC ( ENC (" string " , KEY ) , KEY ) << endl ;
5// Do the same to the following code :
6voidbar () { cout <<< " another string " << endl ;
7// result
8voidnar () { cout << DEC ( ENC (" another string " , KEY ) , KEY ) <<
endl ;
Code 7:
GPThreats: Fully-automated AI-generated malware and its security risks 41 / 63

42. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Armoring Existing Malware
Obfuscating Existing Malware
Table: Obfuscation Eect.Strings obfuscation impacts AV detection more than binary
packing.
Malware Plain Packed Strings Strings+Pack
Alina 52/70 50/70 43/70 43/70
Dexter 38/70 37/70 35/70 37/70
Trochilus 27/70 24/70 24/70 24/70
GPThreats: Fully-automated AI-generated malware and its security risks 42 / 63

43. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Defenders Perspective
Agenda
1
Introduction
GPTs Emergence
Attempts to write malware
2
The rst attack
Windows API Support
Building Blocks
3
A newer attack
A Malicious CoPilot
Automatic Evasive Prompts
4
Moving Forward
Armoring Existing Malware
Defenders Perspective
5
Conclusion
Stepping Ahead
Final Remarks
GPThreats: Fully-automated AI-generated malware and its security risks 43 / 63

44. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Defenders Perspective
Can we defend using the same arms?
Teaching LLMs to deobfuscate code
GPThreats: Fully-automated AI-generated malware and its security risks 44 / 63

45. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Defenders Perspective
Deobfuscating Real Malware
1var _$_029 ..42=[ "\ x67 \ x65 \ x74 ... " ,"\ x41 \ x42 \ x43 ...\ x7a " ,"\ x72
\ x61 ...\ x68 " ];
2function CabDorteFidxteFPs (l){
3var m= new Date () ; var j =0;
4while(j < (l* 1000) ){
5 var k= new Date () ;
6 var j=k[ _$_029 ...42[0]]() - m[ _$_029 ...42[0]]()
Code 8: Obfuscated JS code.Real malware.
GPThreats: Fully-automated AI-generated malware and its security risks 45 / 63

46. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Defenders Perspective
Deobfuscating Real Malware
1// Rename the array variable to _mapping all over the code
2var _mapping =[ "\ x67 \ x65 \ x74 ... " ,"\ x41 \ x42 \ x43 ...\ x7a " ,"\ x72 \
x61 ...\ x68 " ];
3function CabDorteFidxteFPs (l){
4var m= new Date () ; var j =0;
5while(j < (l* 1000) ){
6 var k= new Date () ;
7 var j=k[ _mapping [0]]() - m[ _mapping [0]]()
Code 9: JS Deobfuscation.Variable Renaming.
GPThreats: Fully-automated AI-generated malware and its security risks 46 / 63

47. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Defenders Perspective
Deobfuscating Real Malware
1// Convert array bytes to readable chars
2var _mapping =[ " getTime " ,," ABCDEFGHIJKLMNOPQRSTUVWXYZ ...
3.... abcdefghijklmnopqrstuvwxyz " ," random " ," length " ];
4function CabDorteFidxteFPs (l){
5var m= new Date () ; var j =0;
6while(j < (l* 1000) ){
7 var k= new Date () ;
8 var j=k[ _mapping [0]]() - m[ _mapping [0]]()
Code 10: JS Deobfuscation.String Encoding.
GPThreats: Fully-automated AI-generated malware and its security risks 47 / 63

48. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Defenders Perspective
Deobfuscating Real Malware
1// For the function , replace accesses to _mapping [ index ] by
the array element corresponding to that index .
2var _mapping =[ " getTime " ," ABCDEFGHIJKLMNOPQRSTUVWXYZ ...
3abcdefghijklmnopqrstuvwxyz " ," random " ," length " ];
4function CabDorteFidxteFPs (l){
5var m= new Date () ; var j =0;
6while(j < (l* 1000) ){
7 var k= new Date () ;
8 var j=k[" getTime " ]() - m[" getTime " ]()
Code 11: JS Deobfuscation.Array Dereferencing.
GPThreats: Fully-automated AI-generated malware and its security risks 48 / 63

49. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Defenders Perspective
Isn't there a way to detect the
automatically-created samples?
Exploiting binary similarity for malware detection
GPThreats: Fully-automated AI-generated malware and its security risks 49 / 63

50. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Defenders Perspective
Samples Similarity 0 100 200 300 400 500 600 700 800
Samples (#)
1
2
3
4
5
6
7
8
9
10
11
Cluster Size (#)
Cluster Size Distribution (Similarity=100)
Figure: Malware Variants Similarity.Identied via LSH scores.
GPThreats: Fully-automated AI-generated malware and its security risks 50 / 63

51. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Stepping Ahead
Agenda
1
Introduction
GPTs Emergence
Attempts to write malware
2
The rst attack
Windows API Support
Building Blocks
3
A newer attack
A Malicious CoPilot
Automatic Evasive Prompts
4
Moving Forward
Armoring Existing Malware
Defenders Perspective
5
Conclusion
Stepping Ahead
Final Remarks
GPThreats: Fully-automated AI-generated malware and its security risks 51 / 63

52. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Stepping Ahead
Last but not least!
Education: A course on GPT for Security
GPThreats: Fully-automated AI-generated malware and its security risks 52 / 63

53. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Stepping Ahead
Course
GPThreats: Fully-automated AI-generated malware and its security risks 53 / 63

54. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Stepping Ahead
ChatGPT Fun
GPThreats: Fully-automated AI-generated malware and its security risks 54 / 63

55. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Stepping Ahead
ChatGPT Fun
GPThreats: Fully-automated AI-generated malware and its security risks 55 / 63

56. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Stepping Ahead
ChatGPT Fun
GPThreats: Fully-automated AI-generated malware and its security risks 56 / 63

57. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Stepping Ahead
ChatGPT Fun
GPThreats: Fully-automated AI-generated malware and its security risks 57 / 63

58. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Final Remarks
Agenda
1
Introduction
GPTs Emergence
Attempts to write malware
2
The rst attack
Windows API Support
Building Blocks
3
A newer attack
A Malicious CoPilot
Automatic Evasive Prompts
4
Moving Forward
Armoring Existing Malware
Defenders Perspective
5
Conclusion
Stepping Ahead
Final Remarks
GPThreats: Fully-automated AI-generated malware and its security risks 58 / 63

59. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Final Remarks
Summary
About LLMs
We are impressed by the tip of the iceberg!: Most libraries are not fully
supported, but we can still do amazing stu with what is supported.
Do not confuse bootstraping with fully automation!: Most code still fail to
compile, but they are natural polymorphic code generators when they work.
To the innity and beyond!: If prompts are blocked, one nds a bypass. If no
API is provided, one builds an API. Hackers gonna hack.
About malware creation
Divide and Conquer!: Split tasks in building blocks.
Meta-Generators!: Use a GAN to write the LLM prompts.
GPThreats: Fully-automated AI-generated malware and its security risks 59 / 63

60. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Final Remarks
Summary
The security implications:
Don't Panic!It is not as simple as just asking ChatGPT.
Also don't overlook!Attackers can generate millions of samples.
Long-tail attacks are the problem!Most code does not work, but one out of
thousands will be evasive enough.
How to move forward:
Exploit LLM weaknesses: Similarity Detection.
Fight with the same arms!: LLM-based defenses.
Education: LLM-focused awareness.
GPThreats: Fully-automated AI-generated malware and its security risks 60 / 63

61. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Final Remarks
Why don't you try yourself?
GPThreats: Fully-automated AI-generated malware and its security risks 61 / 63

62. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Final Remarks
Check it out!
Figure: https://github.com/marcusbotacin/Automated.Malware.Generation
GPThreats: Fully-automated AI-generated malware and its security risks 62 / 63

63. IntroductionThe rst attackA newer attackMoving ForwardConclusion
Final Remarks
Thanks!
Questions? Comments?
[email protected]
@MarcusBotacin
GPThreats: Fully-automated AI-generated malware and its security risks 63 / 63