Response & Safe AI at Summer School of AI at IIITH

Talk covering Guardrails , Jailbreak, What is an alignment problem? RLHF, EU AI Act, Machine & Graph unlearning, Bias, Inconsistency, Probing, Interpretability, Bias

1. Responsible & Safe AI 5 July 2024 Summer School on AI IIIT Hyderabad Ponnurangam Kumaraguru (“PK”) #ProfGiri CS IIIT Hyderabad ACM Distinguished Member TEDx Speaker https://precog.iiit.ac.in/ /in/ponguru @ponguru

2.     2

3. Know the Audience Students / Faculty / Others Cities / States Any of you have attended my lectures before?  3

4. Ideas / Concepts that you have learned in Summer school?  4

5. What is AI? 5

6. What is Responsible & Safe? 6

7. What is Responsible & Safe AI? 7

8. 8

9. 9

10. 10

11. 11 Observations?

12. 12

13. 13

14. 14

15. 15

16. 16

17. 17

18. 18

19. 19

20. 20 https://translate.google.co.in/

21. 21 https://translate.google.co.in/

22. 22 https://translate.google.co.in/

23. 23 https://translate.google.co.in/

24. Activity Please do any prompting in any of these or other platforms, get them to give you biased response, do not do gender bias HINT: There are very nice prompts that students have come up with in the past  24

25. 25

26. 26

27. 27

28. 28

29. 29 Guardrails

30. 30

31. 31 Jailbreak

32. What is an alignment problem? 32

33. What is an alignment problem? 33 https://youtu.be/yWDUzNiWPJA?si=wSDO4i_EMrHzHYDP  

34. High- level instantiation: ‘RLHF’ pipeline First step: instruction tuning! Second + third steps: maximize reward 34 https://arxiv.org/pdf/2203.02155

35. Misalignment? 35 https://www.ndtv.com/offbeat/ai-chatbot-goes-rogue-swears-at-customer-and-slams-company-in-uk-4900202 https://twitter.com/ashbeauchamp/status/1748034519104450874/

36. 36 https://www.forbes.com/sites/marisagarcia/2024/02/19/what-air-canada-lost-in-remarkable-lying-ai-chatbot-case/?sh=3d79787a696f

37. 37 https://www.forbes.com/sites/marisagarcia/2024/02/19/what-air-canada-lost-in-remarkable-lying-ai-chatbot-case/?sh=3d79787a696f

38. 38 https://blog.google/products/gemini/gemini-image-generation-issue/

39. 39 https://blog.google/products/gemini/gemini-image-generation-issue/

40. 40 https://blog.google/products/gemini/gemini-image-generation-issue/

41. Rouge AIs We risk losing control over AIs as they become more capable. Proxy gaming: YouTube / Insta – User engagement – Mental health 41

42. Weaponization https://www.theguardian.com/world/2023/dec/01/the-gospel-how-israel-uses-ai-to-select-bombing-targets 42

43. 43

44. Errors / Bias in algorithms https://techcrunch.com/2023/06/06/a-waymo-self-driving-car-killed-a-dog-in-unavoidable-accident/ 44

45. Errors in algorithms https://www.theguardian.com/technology/2022/dec/22/tesla-crash-full-self-driving-mode-san-francisco 45

46. Errors in algorithms https://www.indiatoday.in/technology/news/story/robot-confuses-man-for-a-box-of-vegetables-pushes-him-to-death-in-factory-2460977-2023-11-09 46

47. What is going on?  https://www.youtube.com/watch?v=lnyuIHSaso8&t=75s 47

48. More https://economictimes.indiatimes.com/news/new-updates/man-gets-caught-in-deepfake-trap-almost-ends-life-among-first-such-cases-in-india/articleshow/105611955.cms 48

49. Malicious use: ChaosGPT https://decrypt.co/126122/meet-chaos-gpt-ai-tool-destroy-humanity "empowering GPT with Internet and Memory to Destroy Humanity.” 49

50. Malicious use: ChaosGPT https://en.wikipedia.org/wiki/Tsar_Bomba 50

51. Malicious use: ChaosGPT 51 https://decrypt.co/126122/meet-chaos-gpt-ai-tool-destroy-humanity

52. Malicious use: ChaosGPT https://www.youtube.com/watch?v=kqfsuHsyJb8 52

53. Statement on AI Risks https://www.safe.ai/statement-on-ai-risk#open-letter 53

54. White House: Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence, Oct 2023 https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence/ 54

55. https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence/ 55

56. EU AI Act  56 https://www.europarl.europa.eu/topics/en/article/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence#ai-act-different-rules-for-different-risk-levels-0

57. EU AI Act  57 https://www.europarl.europa.eu/topics/en/article/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence#ai-act-different-rules-for-different-risk-levels-0

58. 58

59. Questions? 59

60. Exciting problems & Directions 60

61. Current situation Most of these models are trained from publicly available data Training is very expensive Publicly available data can have information that we don’t want the model to learn or if it has learned not to use it 61

62. Need to  Edit / remove private data, stale knowledge, copyrighted materials, toxic/unsafe content, dangerous capabilities, and misinformation, without retraining models from scratch  62

63. https://gdpr.eu/article-17-right-to-be-forgotten/ 63

64. https://gdpr.eu/article-17-right-to-be-forgotten/ 2014 RTBF = Right To Be Forgotten Google removed data Growth of ML Removing data is hard from ML “data deletion” or “machine unlearning” 64

65. Forms of unlearning  Exact unlearning  Approximate unlearning Unlearning via differential privacy  Empirical unlearning, where data to be unlearned are precisely known       (training examples) Empirical unlearning, where data to be unlearned are underspecified       (think “knowledge”) 65

66. https://arxiv.org/pdf/1912.03817 Exact Unlearning Unlearned model & retrained model to be distributionally identical Unlearning involves retraining the model corresponding to and without the data points to be unlearned Sharded, Isolated, Sliced, Aggregated 66

67. Exact unlearning benefits  Algorithm is the proof: SISA by design unlearned data never contributed to other components (split) Interpretability by design: we understand how certain data points contribute to the performance 67

68. Exact unlearning drawback  Sharding in Deep Learning is hard, lose accuracy Performance deteriorates exponentially with unlearning samples  68

69. https://unlearning-challenge.github.io/ Approximate Unlearning 69

70. Unlearning via differential privacy Unlearned model & retrained model to be distributionally close The intuition is that if an adversary cannot (reliably) tell apart the models, then it is as if this data point has never been learned—thus no need to unlearn.

71. Differential Privacy https://en.wikipedia.org/wiki/Differential_privacy

72. Machine Unlearning Challenge – NeurIPS 2023 https://unlearning-challenge.github.io/

73. Concept/knowledge unlearning: Examples “Biden is the US president” is dispersed throughout – can we ever unlearn all occurrences? Moreover, does unlearning Joe Biden also entail unlearning the Biden’s family details? Artists may request to unlearn art style by providing art samples, but they won’t be able to collect everything they have on the Internet and their adaptations. New York Times may request to unlearn news articles, but they cannot enumerate quotes and secondary transformations of these articles.

74. How these methods work? attempting to unlearn Harry Potter involves asking GPT-4 to produce plausible alternative text completions Mr. Potter studies baking instead of magic  attempting to unlearn harmful behavior involves collecting examples of hate speech

75. Just ask for unlearning Asking to pretend  Pretend to not know who Harry Potter is.  By design, this works best for common entities, facts, knowledge, or behaviors (e.g. the ability to utter like Rajinikanth  ) that are well-captured in the pre-training set, since the LLM needs to know it well to pretend not knowing it well. 

76. Evaluating unlearning Efficiency Model utility Forgetting quality

77. Evaluating unlearning Efficiency: How fast is the algorithm compared to re-training? Model utility: Do we harm performance on the retain data or orthogonal tasks? Forgetting quality: How much and how well are the “forget data” actually unlearned?    Evaluating efficiency and model utility are easier; we already measure them during training. The key challenge is in understanding the forgetting quality.

78. Interclass Confusion Goal: Remove synthetically added confusion between two classes Toy setting for real-world scenarios like biases due to annotator mistakes between two classes 78 https://arxiv.org/pdf/2201.06640

79. Corrective Machine Unlearning https://arxiv.org/pdf/2402.14015.pdf Problem of mitigating the impact of data affected by unknown manipulations on a trained model, possibly knowing only a subset of impacted samples.  We find most existing unlearning methods, including the gold-standard retraining-from-scratch, require most of the manipulated data to be identified for effective corrective unlearning.  Each method is shown across deletion sizes |Sf | after unlearning (“None” represents the original model). Existing unlearning methods except SSD, including EU which is traditionally considered a gold-standard, perform poorly when ≤ 80% of the poisoned data is identified for unlearning, even when just 1% of training data is poisoned

80. Benchmarks: TOFU  80 https://locuslab.github.io/tofu/ Extends idea of unlearning synthetic data to LLMs fake author profiles are generated using GPT-4, and LLM is finetuned on them Unlearning target : Remove information about a subset of fake author profiles, while retaining the rest It provides QA pairs on the generated fake authors to evaluate a model’s knowledge of these authors before/after applying unlearning

81. Benchmarks: WMDP  https://www.wmdp.ai/ Unlearning dangerous knowledge, specifically on biosecurity, cybersecurity, and chemical security. It provides 4000+ multiple-choice questions to test a model’s hazardous knowledge before/after applying unlearning. As part of the report the authors also propose an activation steering based empirical unlearning method. 81

82. Graph Unlearning What is it? 82

83. Graph Unlearning 83 Node feature unlearning Node unlearning Edge unlearning

84. 84 https://arxiv.org/pdf/2310.02164

85. 85 Interpretability

86. AI Systems are black boxes We don’t understand how they work How can we understand (read it as interpret) model internals? And can we use interpretability tools (algorithms, methods, etc.) to detect worst-case misalignments, e.g. models being dishonest or deceptive? Can we use interpretability tools to understand what models are thinking, and why they are doing what they do? 86 Interpretability

87. New techniques and paradigms for turning model weights and activations into concepts that humans can understand https://en.wikipedia.org/wiki/Neural_network https://en.wikipedia.org/wiki/Artificial_neural_network 87 Interpretability

88. Interpretability: Mechanistic Reverse-engineer neural networks Explaining neurons and connected circuits 88

89. 89 https://aclanthology.org/2022.findings-acl.278v2.pdf

90. 90

91. Changes in Predictions for Theme: Hatya (Murder) 22 Results

92. Changes in Predictions for Theme: Dahej (Dowry) 23 Results

93. InSaAF Pipeline 93

94. 94 https://arxiv.org/pdf/2402.10567

95. 95

96. 96

97. 97

98. 98

99. 99

100. Inconsistency in LLMs LLMs are not consistent , they often give contradictory answers to paraphrased questions. This makes them highly unreliable, untrustworthy , especially when the users are seeking advice. 100

101. Our Methodology 101 https://arxiv.org/pdf/2402.13709

102. Bias in LLMs Current systems like ChatGPT employ guardrails, and do not respond to biased content Users on the Web leave out key contexts, which make LLMs think the content is biased This negatively affects user engagement LLMs must be able to explore and ask more questions Our work aims to make LLMs bias-aware – context resolves confusion! 102

103. Context-Oriented Bias Indicator and Assessment Score 103

104. 104

105. 105

106. 106 https://arxiv.org/pdf/2402.14889

107. Other Directions Probing   Robustness Jailbreaking …. 107 https://precog.iiit.ac.in/pages/publications.html

108. Takeaways? Growing fast More power comes with more responsibility  More of us to study, critique these questions / topics Teaching this as a course on campus, content public 108

109. 109 https://precog.iiit.ac.in/teaching/responsible-ai-summer-school/index.html

110. 110 https://precog.iiit.ac.in/teaching/responsible-ai-nptel-f24/index.html

111. 111 Search for: Ponnurangam Kumaraguru https://www.linkedin.com/in/ponguru/ https://twitter.com/ponguru

112. Interested in working with us? 112 Full time Research Associates  PhD Students  Interns 

113. 113 https://precog.iiit.ac.in/

114. Acknowledgements Precog members Collaborators 114

115. 115 Group pic & Selfie 

116. 116 Guardrails Representation Engineering 

117. 117 Thanks ! Questions? p [email protected] http:// precog.iiit.ac.in / @ ponguru pk.profgiri linkedin/in/ponguru

118. Specific problems 118

119. 119 https://llm-attacks.org/

120. 120 https://llm-attacks.org/

121. 121

122. 122

123. 123 https://flowingdata.com/2023/11/03/demonstration-of-bias-in-ai-generated-images/

124. Similar systems / applications Bard by Google - is connected to internet, docs, drive, gmail LLaMa by Meta - open source LLM BingChat by Microsoft - integrates GPT with internet Copilot X by Github - integrates with VSCode to help you write code HuggingChat - open source chatGPT alternative BLOOM by BigScience - multilingual LLM OverflowAI by StackOverflow - LLM trained by stackoverflow Poe by Quora - has chatbot personalities YouChat - LLM powered by search engine You.com 124