Investigating Fairness of Decision Making

Presentation given at 5th Int Conference on Social Computing, Guangzhou, August 30, 2024

1. KI – Institute for Artificial Intelligence Investigating Fairness of Decision Making Steffen Staab @ ststaab https:// www.ki.uni-stuttgart.de https:// semanux.com https:// southampton.ac.uk / research / institutes-centres /web-internet- science

2. My background Simulation and ML in Sciences and Engineering Semantics of Data Human-Machine Interaction Stuttgart Research Focus Reflecting on Intelligent Systems Democracy, Diversity , Demography Semantics of Interaction Assistive Technologies Knowledge Graphs Semantics of Text Semantics in Programs Machine Learning Web Science Social Network (De-)Growth Semantics by Imitation Social Group Construction Fairness Circular Factory Architecture, Construction

3. “Code is law” Lawrence Lessig 1999

4. “Code is law” Lawrence Lessig 1999 Distributive justice? Fair political system? Underlying political philosophy? How do algorithmic decisions affect humans?

5. 1.9.2024 5 Where to consider Fairness in Software Development?

6. 1.9.2024 6 Where to consider Fairness in Software Development? Yes, I will talk about this Yes, I have arguments I believe ”yes”

7. 1.9.2024 7 Where to consider fairness in AI Software Development? (Amershi et al 2019)

8. 1.9.2024 8 Where to consider fairness in AI Software Development? (Amershi et al 2019) (Mougan et al 2023) Yes, I have arguments I believe ”yes” Yes, I will talk about this

9. Considering Analysis and Design with Q. Ramadan, M. Konersmann , A. S. Ahmadian , J. Jürjens

10. 1.9.2024 10 Where to consider Fairness in Software Development? (Ramadan et al 2020; Ramadan et al 2024)

11. 11 Individual fairness A decision-making software preserves individual fairness if it produces the same decision for every two individuals whose data are identical except for the protected characteristics or their proxy attributes . Otherwise, their data are identical Bob Alice Decision Making Software OR Rejected , Rejected Granted , Granted

12. 12 Violations from Data Dependencies Data depending on protected characteristics may provoke indirect discrimination Decision-making software 2 Protected characteristics Decision-making software 1 Else Dependent intermediate outcome Join(Protected characteristic, Else) (Dwork & Ilvento 2018)

13. 13 Software model example inspired by real banking terms and conditions income:int healthy:boolean goodCreditHistory / hasLifeInsurance:boolean= false / hasCreditCard:boolean= false / ownPoints:int=0 / hasFreeAccount:boolean= false verifyLifeInsApp() verifyCreditCardApp() exit () CustomerProfile Idle VerifyLifeInsApp [healthy ==true] / hasLifeInsurance=true; ownPoints+=25 VerifyingLifeInsuranceApplication DecisionMaking AcceptedLifeInsApp [else] / hasLifeInsurance=false; verifyLifeInsApp() [hasFreeAccount== false && goodCreditHistory==true && ownPoints>=50] / hasFreeAccount=true; exit () End Start / verifiedLifeInsApp VerifyCredit CardApp VerifyingCreditCardApplication AcceptedCreditCardApp [else ] / hasCreditCard=false; [income>=2500] / hasCreditCard=true; ownPoints+=25 verifyCreditCardApp() / verifiedCreditCardApp Class Attribute Operation State machine State Transition Guard Action We define: Proxy for age: income Decision data attribute: hasFreeAccount

14. 14 The challenge Given a UML model, given a specification of protected characteristics and proxy attributes, the formal problem is defined as: Does the behavior of a specific “DecisionMaking” state machine in the UML model violate individual fairness with respect to protected characteristics? Empirical evaluation indicates that analyzing the UML models manually produces unreliable results with a high chance of 54% that analysts overlook true-positive discrimination.

15. For any initialization of the attributes, the final setting of the decision data attribute in any feasible path of the system is independent from the setting of all protected characteristics and proxy attributes. 15 Verifying Individual Fairness in Labeled Transition Systems Prove or disprove verification objective:

16. MBFair : Verify Labeled Transition Systems with respect to Individual Fairness Verify Preconditions Verify Decision Independence Constraint Generate Initializations For any initialization of the attributes, the final setting of the decision data attribute in any feasible path of the system is independent from the setting of all protected characteristics and proxy attributes.

17. For any initialization of the attributes , the final setting of the decision data attribute in any feasible path of the system is independent from the setting of all protected characteristics and proxy attributes. 17 Verifying Individual Fairness in Labeled Transition Systems Prove or disprove verification objective: income:int healthy:boolean goodCreditHistory / hasLifeInsurance:boolean= false / hasCreditCard:boolean= false / ownPoints:int=0 / hasFreeAccount:boolean= false verifyLifeInsApp() verifyCreditCardApp() exit () CustomerProfile Attributes with guard(s) defined over them Derived data attributes

18. 18 Generate Initializations Derived data attributes Attributes with guard(s) defined over them

19. MBFair : Verify Labeled Transition Systems with respect to Individual Fairness Verify Preconditions Verify Decision Independence Constraint Generate Initializations Check out technicalities at (Ramadan et al 2024)

20. Given a transition system a decision data attribute is independent of all protected characteristics iff for any two initializations and , and any value in domain , the following holds:   20 Decision Independence Constraint             iff Linear temporal logics : at some point always from there on

21. 21 Results: (1) Does not hold Does not hold

22. 2 Explanation Disparity Carlos Mougan et al. : Demographic Parity Inspector: Fairness Audits via the Explanation Space. Draft. Arxiv 2023

23. Limits of Individual Fairness When two people are never the same (similarity) When decisions can only pick one e.g. hiring for a job When randomness is involved (probabilistic verification)

24. 1.9.2024 24 Fairness in AI Software Development – what I will talk about (Amershi et al 2019) Yes, I will talk about this To my knowledge: Statistical notions of fairness have only been applied after the fact

25. 1.9.2024 25 Unsupervised Model Monitoring

26. 1.9.2024 26 Quality Assessment of Unsupervised Model Monitoring Often impossible! You do not know how good the person would have been if you would have hired them, when you did not

27. Example Demographic parity   1.9.2024 27 Statistical notions of fairness wrt protected  

28. Issues with demographic parity Assume artificial data with for good/bad grades for low/high beauty for accepted/reject for protected attribute (gender) but is accepted for beauty and is accepted for good grades   1.9.2024 28

29. Unsupervised criteria Demographic parity Fairness by unawareness do not use in issue: correlations to   Supervised criterion Equal opportunity   1.9.2024 29 Statistical notions of fairness wrt protected  

30. Unsupervised criteria Demographic parity Fairness by unawareness do not use in issue: correlations to   Key idea: common denominator Use Shap values to judge outcome and (un)awareness 1.9.2024 30 Statistical notions of fairness wrt protected  

31. Feature Attributions Explanations: SHAP values 1.9.2024 31 Model        

32. Local Explanations SHAP values for one item Distributions of Explanations Global Explanations averaging SHAP valuesover sets of items Not yet studied in related work! 1.9.2024 32

33. Properties of SHAP values Efficiency:  Sum of all contributions equal to the prediction Allows for studying equal outcome Uninformativeness : If a feature does not contribute to the prediction Allows for studying unawareness from protected attributes Linear + IID: Simplification Easy to understand intuitively 1.9.2024 33   if feature is irrelevant   For  

34. Methodology

35. Methodology

36. Methodology No explanation disparity Explanation disparity

37. 1.9.2024 37 Example results

38. 1.9.2024 38 Example results

39. 3 Conclusion

40. Polity group of people formally/informally organized for governance Politics the set of activities that are associated with making decisions in groups Policy/Rule/Law Structures and institutes of software development stakeholders, analysts, programmers, coders social scientists, lawyers, … Processes of software development co-creation? Code How to craft political structures for a fair society?

41. Technical how to involve fairness in all steps of the software development process? how to support the developer? how to guarantee fairness? Socio-Technical which notion of fairness to use when? how to collect, aggregate and consider societal opinion? Open issues wrt Fairness in Software Development 1.9.2024 41