[DSC DACH 24] Transparency as a catalyst for trustworthy and sustainable AI - Rania Wazir

One of the most commonly cited characteristics of trustworthy AI systems - whether it's in ethics guidelines, or in interviews with impacted stakeholders - is transparency. But the commonalities end there. What exactly is transparency - is it about documentation, or access? Who is supposed to be transparent towards whom? Is transparency a goal unto itself, or a vehicle towards a greater good? Is transparency different from explainability and traceability? How does transparency relate to other trustworthiness characteristics, such as human agency, fairness, sustainability, privacy, and cybersecurity? And finally - is transparency a luxury, to be afforded only to those who know and those who can pay? Or is it a basic good, that all AI stakeholders are entitled to - and who then, bears the cost? The conversation about transparency, and its role as catalyst for trustworthy and sustainable AI, is just taking off.

1. Transparency As Catalyst for Trustworthy and Sustainable AI DSC DACH , 12.09.2024 Rania Wazir, Ph.D. Co-founder & CTO, leiwand.ai

2. Questions What exactly is transparency - is it about documentation, or access? Who is supposed to be transparent towards whom? Is transparency a goal unto itself, or a vehicle towards a greater good? Is transparency different from explainability and traceability? How does transparency relate to other trustworthiness characteristics, such as human agency, fairness, sustainability, privacy, and cybersecurity? And finally - is transparency a luxury, to be afforded only to those who know and those who can pay? Or is it a basic good, that all AI stakeholders are entitled to - and who then, bears the cost?

3. Outline Rania Wazir, PhD; Dr. Gertraud Leimüller - leiwand.ai 3 An a cknowledgement of l imitations Transparency is the answer? The courage to be transparent

4. Making fair and qualitative AI a reality leiwand.ai We develop tools that make artificial intelligence trustworthy: Trustworthy AI leiwand.ai 2024 We utilise extensive expertise in open innovation, technical AI development, and algorithmic fairness Human Oversight Transparency Robust & Secure Responsibility Data Privacy, Data Governance Societal & Ecological Prosperity Fairness & Trustworthiness Humans at the center of the AI-lifecycle Data Science Social Science

5. Outline Rania Wazir, PhD; Dr. Gertraud Leimüller - leiwand.ai 5 An a cknowledgement of l imitations Transparency is the answer? The courage to be transparent

6. AI: Cool! But does it work ? 6 Rania Wazir, PhD - leiwand.ai Quelle: Raji et al., The Fallacy of AI Functionality, FAccT 2022: https://facctconference.org/static/pdfs_2022/facct22-3533158.pdf . https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2781307 AI in medicine : JAMA, External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients too many false alarms , not enough correctly identified cases

7. AI: Cool! But does it work ? 7 Rania Wazir, PhD - leiwand.ai https ://interaktiv.br.de/ki-bewerbung https://www.technologyreview.com/2021/07/07/1027916/we-tested-ai-interview-tools / https://www.bloomberg.com/graphics/2024-openai-gpt-hiring-racial-discrimination/ AI in recruiting : Depending on the b ackground in the video and accessories, the personality of applicants is judged differently . Generative AI: a Bloomberg study shows bias against candidates based on their names alone

8. AI: Cool! But is it secure? 8 „ Thousands of Avis car rental customers had personal data stolen in cyberattack” TechCrunch, 09.09.2024 cars are becoming fully digitalized, with access to huge amounts of personal data d rivers and passengers are often unaware of the data collection, or how to opt out how to delete data before re-selling? https://techcrunch.com/2024/09/09/thousands-of-avis-car-rental-customers-had-personal-data-stolen-in-cyberattack/ Rania Wazir, PhD - leiwand.ai

9. Stanford AI Index 2023, Chapter 2. Measuring trends in Artificial Intelligence https ://aiindex.stanford.edu/ai-index-report-2023/# individual-chapters

10. The hidden labor fueling AI 10 „the platforms dole out work in a piecemeal, unpredictable fashion; they pit workers against one another to drive up their working hours and drive down their earnings; they suspend accounts - entire countries - without warning. They treat workers as disposable.” K. Hao Several lawsuits for copyright violations: from programmers, authors, musicians ... https://www.wsj.com/articles/chatgpt-openai-content-abusive-sexually-explicit-harassment-kenya-workers-on-human-workers-cf191483 Rania Wazir, PhD - leiwand.ai

11. „All algorithms should be seen as untrustworthy until proven otherwise. Until we as a society acknowledge this, and insist on the transparency required for the public to assess reliability and fairness, we‘re not ready to use them.“ Cathy O‘Neill Rania Wazir, PhD - leiwand.ai C. O’Neill, Mutant Algorithms Are Coming for Your Education , Bloomberg, 08.09.202, https:// www.bloomberg.com/view/articles/2020-09-08/mutant-algorithms-are-coming-for-your-education

12. Outline Rania Wazir, PhD; Dr. Gertraud Leimüller - leiwand.ai 12 An a cknowledgement of l imitations Transparency is the answer? The courage to be transparent

13. Rania Wazir, Ph.D. – leiwand.ai 13 Trustworthy AI: Fundamental requirements according to EU HLEG on AI human agency and oversight technical robustness and safety privacy and data governance transparency diversity , non- discrimination and fairness environmental and societal well- being and accountability

14. Rania Wazir, Ph.D. – leiwand.ai 14 AI Act: Transparency requirements Article 11: Technical documentation – containing elements set out in Annex IV Article 13: Transparency and provision of information to deployers Article 50: Transparency obligations for providers and deployers of certain AI systems Article 53: Obligations for providers of general-purpose AI models

15. y es ... but what is transparency?

16. Rania Wazir, Ph.D. – leiwand.ai 16 AI Act: Open source exemption Article 2 (12) This Regulation does not apply to AI systems released under free and open-source licences , unless they are placed on the market or put into service as high-risk AI systems or as an AI system that falls under Article 5 ( Prohibited practices) or 50 ( Transparency obligations for providers and deployers of certain AI systems). Similar exemptions to documentation requirements apply to providers of open source GPAI models Article 53(2)

17. 33 41 47 49 51 55 56 58 60 62 64 75 75 85 10 20 30 40 50 Score 60 70 80 90 100 Fuyu- 8B Titan Text Express Gemini 1.0 Ultra GPT- 4 Claude 3 Mistral 7B Palmyra- X Stable Video Diffusion Llama 2 Phi- 2 Granite Luminous Jurassic- 2 StarCoder Open Closed Foundation Model Transparency Total Scores of Open vs. Closed Developers, May 2024 Source: May 2024 Foundation Model Transparency Index

18. Some definitions ISO/IEC 22989:2022( en ) Information technology — Artificial intelligence — Artificial intelligence concepts and terminology transparency <system> property of a system that appropriate information about the system is made available to relevant stakeholders Note 1 to entry: Appropriate information for system transparency can include aspects such as features, performance, limitations, components, procedures, measures, design goals, design choices and assumptions, data sources and labelling protocols. Note 2 to entry : Inappropriate disclosure of some aspects of a system can violate security, privacy or confidentiality requirements .

19. Some definitions ISO/IEC 22989:2022( en ) Information technology — Artificial intelligence — Artificial intelligence concepts and terminology transparency <organization> property of an organization that appropriate activities and decisions are communicated to relevant stakeholders in a comprehensive, accessible and understandable manner Note 1 to entry : Inappropriate communication of activities and decisions can violate security, privacy or confidentiality requirements.

20. ISO/IEC 12792 Information technology — Artificial intelligence (AI) — Transparency taxonomy of AI systems

21. Transparency as a Trustworthiness Characteristic TRANSPARENCY EXPLAINABILITY QUALITY SECURITY REPRODUCIBILITY ACCOUNTABILITY VERIFIABILITY PRIVACY CONTROLLABILITY FAIRNESS ROBUSTNESS

22. What are the boundaries of an AI system? AI System D MODELS DATA SOCIETY ENVIRONMENT ECONOMY

23. And breaking it down: AI System D MODELS DATA Transparency Transparency Transparency

24. Transparency through the AI system life cycle stages Inception Design and development Verification and validation Deployment Operation and monitoring … Retirement

25. Transparency From whom Data source AI developer AI provider AI user Towards whom AI developer AI provider AI user AI subject AI auditor Regulator Public

26. How much transparency? From whom Data source AI developer AI provider AI user Towards whom AI developer AI provider AI user AI subject AI auditor Regulator Public

27. transparency ... always good?

28. Transparency 28 Privacy and data protection Cybersecurity Trade secrets and IP Rania Wazir, PhD - leiwand.ai versus

29. Outline Rania Wazir, PhD; Dr. Gertraud Leimüller - leiwand.ai 29 An a cknowledgement of l imitations Transparency is the answer? The courage to be transparent

30. Transparency 30 gives agency to make better decisions about the use of AI systems for the benefit of people and the planet Rania Wazir, PhD - leiwand.ai

31. Rania Wazir, PhD - leiwand.ai 31 AI – we can choose ! Who is in control ? Heikkilä , M., Dutch scandal serves as a warning for Europe over risks of using algorithms , Politico.eu, March 2022. Improving patient treatment through enhanced patient interaction and control of their own data: Quelle: https://www.politico.eu/article/dutch-scandal-serves-as-a-warning-for-europe-over-risks-of-using-algorithms/ https://pubmed.ncbi.nlm.nih.gov/34289996/

32. Rania Wazir, PhD - leiwand.ai 32 What are we optimising for ? Elzayn et al, Measuring and Mitigating Racial Disparities in Tax Audits, SIEPR 2023. A main source of disparity was the choice to detect erroneously claimed refundable credits rather than total under-reporting Pierson, E., et al., An algorithmic approach to reducing unexplained pain disparities in underserved populations, January 2021. Using AI to understand knee pain in underserved populations: AI can detect sources of pain often overlooked by radiologists Quellen : https://siepr.stanford.edu/news/irs-confirms-stanford-study-racial-bias-audits https://dho.stanford.edu/wp-content/uploads/IRS_Disparities.pdf https://www.nature.com/articles/s41591-020-01192-7 AI – we can choose !

33. Rania Wazir, PhD - leiwand.ai 33 AI – we can choose ! What resources are needed? Hao , K., Training a single AI model can emit as much carbon as five cars in their lifetimes, MIT Technology Review, June 6, 2019. Reducing the footprint of recycled steel: Fero Labs uses AI to help steel manufacturers reduce the use of mined ingredients by up to 34%, preventing an estimated 450,000 tons of CO2 emissions per year Quellen ;: https://www.technologyreview.com/s/613630/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/ https://gpai.ai/projects/responsible-ai/environment/climate-change-and-ai.pdf

34. transparency ... what is your choice?

35. Questions? What exactly is transparency - is it about documentation, or access? Who is supposed to be transparent towards whom? Is transparency a goal unto itself, or a vehicle towards a greater good? Is transparency different from explainability and traceability? How does transparency relate to other trustworthiness characteristics, such as human agency, fairness, sustainability, privacy, and cybersecurity? And finally - is transparency a luxury, to be afforded only to those who know and those who can pay? Or is it a basic good, that all AI stakeholders are entitled to - and who then, bears the cost?

36. Dr. Getraud Leimüller Lene Kunze, MSc Mira Reisinger, MA Data Scientist Our Team: Data Science Meets Social Science Rania Wazir, PhD Silvia Wasserbacher-Schwarzer, MA Janine Vallaster, MSc Social Scientist Social Scientist Co-Founder & CTO Co-Founder & CEO Patrick Kosmider, MA Communications Manager Sarah Cepeda, PhD Data Scientist Chief Strategist Mag. Thomas Treml Data Scientist

37. Let’s keep the conversation going! www.leiwand.ai [email protected] Thank you Scan Me!