Ethical Initiatives in AI and accountability.pdf

The slides talk about ethical initiatives around globe in AI. It speaks about ethical harms, AI audit, accountability and governance, Tesla usecase, COMPAS recidivism bias, Facebook Cambridge Analytica Scandal, Deepfake elections threat. It talks about tools as Model Cards, AI Fairness 360, Fairlearn etc.

1. Responsible AI & Ethics
Module 3
Ethical initiatives in AI and
accountability
Dr. ShiwaniGupta

2. Learning Outcomes
Students will be able to:
•Compare and critically evaluate international ethical initiativesin
AI.
•Identify ethical harms & societal concernsin AI applications.
•Perform a basic audit of an AI modelfor bias, fairness, and
transparency.
•Understand mechanisms for accountability and governancein AI
systems.

3. Introduction to AI Ethics & Accountability
Why AI needs ethics
1. Bias in AI
Problem:AI systems trained on biased data may replicate or even amplify societal inequalities.
Examples:
Hiring tools favoring one gender.
Face recognition systems with higher error rates for certain demographics.
Ethical Concern:Unfair treatment of individuals and communities.
2. Black-box Decisions
Problem:Many AI models (deep Neural Networks, Large Language Models) make predictions without clear explanations.
Examples:
Loan approval systems not explaining why an application was rejected.
Healthcare AI recommending treatments without transparent reasoning.
Ethical Concern:Lack of transparency erodes trust and accountability.
3. Responsibility Gaps
Problem:When AI systems cause harm, it’s unclear who is responsible—developers, organizations, or the AI itself.
Examples:
Autonomous car accidents.
Misdiagnoses by medical AI.
Ethical Concern:Legal and moral accountability remains ambiguous.

4. Introduction to AI Ethics & Accountability
AI accountability: who is responsible?
1. Developer (Design & Training Stage)
Responsible for algorithm design, data quality, and bias mitigation.
Example: If a face recognition system is biased due to skewed training data → developer accountability.
Ethical duty: ensure transparency, explainability, and safe model release.
2. Deployer(Organization / Institution Using AI)
Responsible for how the AI is integrated into real-world workflows.
Example: A bank using an AI tool for loan approval → deployermust monitor fairness and legality.
Must implement auditing, monitoring, and safeguards.
3. User (Operator / End-user)
Responsible for appropriate use, supervision, and human-in-the-loop checks.
Example: A doctor using AI in medical diagnosis must not blindly trustAI outputs.
Users must stay trained and accountable for final decisions.
AI can make autonomous decisions → accountability becomes blurred.
Legal systems are still evolving: should liability fall on creators, deployers, or end-users?
Philosophical question: Can AI itself ever be held responsible?

5. Introduction to AI Ethics & Accountability
Case: Tesla self-driving car accident
The Incident: Multiple accidents involving Tesla’s Autopilotand Full Self-Driving (FSD)systems have raised ethical and legal questions.
In some cases, drivers over-relied on automation, leading to fatalities.
The Accountability Debate
Manufacturer (Tesla): Did Tesla overstate the car’s autonomous capabilities?
Were safety warnings and safeguards sufficient?
Driver (Human-in-the-Loop): Was the driver negligent in not paying attention?
Should humans remain responsible even when cars “self-drive”?
Regulators / Governments: Are laws keeping pace with autonomous vehicle technology?
Who sets safety standards and testing requirements?
AI System Itself (Philosophical): Can an AI be held morally responsible?
If not, should liability always fall back to humans and corporations?
Ethical Questions for Discussion
Should AI-driven vehicles prioritize passenger safetyor pedestrian safety?
How much responsibility should rest on the driver vs. the manufacturer?
Should AI decision-making in safety-critical domains be fully transparent?

6. International Ethical Initiatives
Global AI governance Frameworks
1. EU AI Act (2024)
Risk-based classification: Prohibited, high-risk, limited risk, minimal risk
AI.
Transparency obligations: Especially for high-risk systems (e.g.,
biometric ID, medical AI).
First legally binding AI law globally.
2. OECD AI Principles (2019)
Fairness, accountability, transparencyemphasized.
Promote human-centered AIthat respects democratic values and human
rights.
Widely adopted by OECD countries + partners.
3. UNESCO AI Ethics Recommendation (2021)
First global normative frameworkon AI ethics, adopted by 193 countries.
Focus: Human rights, sustainability, diversity, and inclusion.
Calls for banning AI uses that threaten human dignity (e.g., social scoring).
4. US NIST AI Risk Management Framework (2023)
Voluntary guidance for trustworthy AI.
Core pillars: Govern, Map, Measure, Manage.
Strong emphasis on risk identification and mitigationacross AI
lifecycle.
5. India’s NITI AayogResponsible AI Guidelines
Focus on “AI for All”→ inclusive growth, ethics, and innovation.
Principles: Safety, transparency, accountability, privacy
protection.
Advocates for contextual adoptionin sectors like healthcare,
education, agriculture.
6. IEEE Ethically Aligned Design
Global standard-setting initiativefor ethical AI.
Emphasizes values-driven designand alignment with human well-
being.
Promotes technical + ethical convergencein AI systems.
While frameworks differ (law vs. principles vs. guidelines), all converge on fairness, accountability, transparency, and human-
centered AI.

7. Comparative Analysis of Ethical Frameworks
EU AI Act OECD AI
Principles
UNESCO
Ethics
US NIST NITI Aayog
India
IEEE SIMILARITY
Fairness Requires non-
discrimination
in high-risk AI
(e.g.,
recruitment,
credit
scoring).
Calls for
fairness &
inclusive
growth.
Protects human
dignity,
diversity, and
avoids bias.
- “AI for All” –
ensuring
equitable
access.
Embeds values
of inclusiveness
in design.
All stress
avoiding bias,
ensuring
equity, and
preventing
discrimination.
Transparency Mandatory
disclosure for
AI-generated
content &
decision-
making logs.
Transparency &
explainability
of AI systems.
Right to know
when one is
interacting with
AI.
“Map &
Measure” AI
risks through
transparency.
Calls for clear
AI processes &
explainable
models.
Encourages
open design
standards.
All require
explainability
and disclosure
so users
understand AI’s
role.
AccountabilityClear liability:
deployers&
providers must
ensure
compliance.
Human
oversight &
responsibility
remain central.
States &
organizations
are accountable
for AI use.
“Govern” pillar
stresses
organizational
accountability.
Assigns
responsibility to
developers &
institutions.
Engineers must
design with
ethical
accountability
in mind.
Accountability
always lies with
humans/organi
zations, not AI
itself.

8. Comparative Analysis of Ethical Frameworks
1. Legally Binding (Law/Regulation)
These frameworks have enforceable obligations, penalties, and compliance requirements.
EU AI Act
Mechanism:Binding law across EU states.
Tools:Risk-based regulation, mandatory conformity assessments, fines (up to 6% of global turnover).
Focus:High-risk AI (health, education, policing, finance).
National AI Strategies with Legal Backing(e.g., GDPR for automated decision-making).
??????Nature:Hard law→ enforceable through courts, regulators, audits.
Law = must comply (enforceable).
Guidelines = should comply (ethical expectation).

9. 2. Voluntary / Soft Law (Principles & Guidelines)
These rely on self-regulation, best practices, or political commitments, not strict enforcement.
OECD AI Principles
Mechanism:Non-binding, but adopted by 40+ countries as a political commitment.
Focus:Inclusive growth, human rights, transparency.
UNESCO AI Ethics Recommendation
Mechanism:Adopted by 193 Member States, but guidance-oriented. States are encouraged, not forced, to align.
US NIST AI Risk Management Framework
Mechanism:Voluntary adoption for organizations.
Focus:Mapping, measuring, managing risks.
India’s NITI Aayog“Responsible AI for All”
Mechanism:Policy guideline, not law. Institutions may voluntarily align.
IEEE Ethically Aligned Design
Mechanism:Professional standards & design principles, industry-driven adoption.
??????Nature:Soft law→ relies on industry compliance, peer pressure, reputation, and global alignment.
Comparative Analysis of Ethical Frameworks
Key Difference
•Law (EU AI Act, GDPR, national regulations):Binding, penalties for non-compliance, government regulators enforce.
•Voluntary Guidelines (OECD, UNESCO, IEEE, NITI Aayog, NIST):Advisory, encourage ethical culture, rely on
organizations’ willingness.

10. Comparative Analysis of Ethical Frameworks
Missing Global Coordination
1. Fragmented Enforcement
EU AI Actis binding law, but OECD, UNESCO, IEEE, NITI Aayogare voluntary.
This mismatch means companies face strict compliance in the EU but only soft guidelines elsewhere.
2. Divergent Definitions of Key Principles
“Fairness” in EU AI Act (non-discrimination in high-risk AI) ≠ “Fairness” in UNESCO (social justice, inclusiveness).
Lack of standard definitions makes comparative analysis inconsistent.
3. Jurisdictional Silos
Each framework applies within its own region(e.g., EU, US, India).
No universal mechanismensures that principles travel across borders → weak global interoperability.
4. Accountability Gaps
EU AI Act defines liability (provider, deployer).
UNESCO and OECD stop at responsibility principles, without enforcement.
Comparative analysis struggles because accountability shifts depending on jurisdiction.
5. Overlap but No Harmonization
All frameworks stress fairness, transparency, accountability, yet implementation tools differ.
Example: EU uses audits & fines, NIST uses voluntary checklists, UNESCO uses ethical pledges.
6. Lack of Global Arbitration Body
No equivalent of a “World Trade Organization for AI”exists.
Cross-border AI harms (e.g., facial recognition misuse, autonomous vehicle accidents abroad) lack coordinated resolution.
Comparativeanalysisrevealsthatwhileglobal
AIethicssharecommonvalues,their
operationalizationdivergessharplybecause
ofmissingglobalcoordination.Thisleadsto
regulatoryfragmentation,enforcement
asymmetry,andaccountabilityloopholes.

11. Comparative Analysis of Ethical Frameworks
1. Bridging Regulation and Practice
Corporations like Google, Microsoft, OpenAIoperationalize broad ethical principles (e.g., fairness, accountability) into day-to-day design rules, product pipelines, and internal
governance.
They act as the linkbetween high-level frameworks (OECD, UNESCO, EU AI Act)and practical engineering practices.
2. Corporate Frameworks (Examples)
Google AI Principles(2018)
Avoid creating/supporting harmful applications.
Be accountable to people.
Promote fairness and safety.
Microsoft Responsible AI
Six principles: fairness, reliability & safety, privacy & security, inclusiveness, transparency, accountability.
Enforced through tools like Fairlearnand an internal Responsible AI office.
OpenAICharter
Commitment to safe AGI development.
Share benefits globally, cooperate with other institutions, long-term safety focus.
3. Comparative Value
Fairness, Transparency, Accountability→ appear in corporate charters andin global frameworks.
Corporations often adopt broader human-centered AI goals(e.g., Microsoft: inclusiveness, OpenAI: global benefit).
They go further than governments by embedding product-level tools(Fairlearn, interpretability APIs, model cards).

12. 4. Enforcement Mechanisms
•Corporations:Self-enforcement via internal ethics boards, product reviews, employee whistleblowing.
•Governments:Enforced through laws & penalties(e.g., EU AI Act fines).
•International bodies:Recommend but don’t enforce.
??????In comparative analysis, corporations fill the implementation gapwhere voluntary principles lack teeth.
5. Challenges & Criticisms
Conflicts of interest:Profit motives may override ethical commitments.
Ethics washing:Public AI principles not always matched by practice (e.g., criticism of Google Project Maven,
Microsoft ICE contracts).
Non-uniform adoption:Different corporations follow different codes → adds to global inconsistency.
•are de facto regulatorsof AI through their platforms and products.
•Comparative analysis must include them alongside state and intergovernmental frameworks.
•They help translate principles into practice, but also risk fragmentation, inconsistency, and ethics-washing
without external oversight.
Debate: “Should AI ethics be enforced through law or voluntary compliance?”

13. Ethical Harms, Concerns & Auditing
Domain Example Ethical Concern
Hiring
Automated resume screening
(Amazon AI tool)
Gender and racial bias → qualified candidates may be
unfairly rejected
Credit ScoringLoan approval algorithms
Discrimination against minority groups → unequal
financial access
Policing Predictive policing (COMPAS)
Racial bias → overestimation of risk for certain
communities
1. Bias & Discrimination
AI systems can perpetuate or amplify social inequalitieswhen trained on biased data or poorly designed:
Mechanisms:biased datasets, historical discrimination, algorithmic assumptions.

14. Domain Example Ethical Concern
Facial Recognition Airport security, law enforcement
Misidentification, surveillance, lack
of consent
Surveillance Capitalism Facebook–Cambridge Analytica
Exploitation of personal data for
targeted political/advertising
manipulation
2. Privacy Invasion
AI tools can intrude into personal lifeor exploit user datawithout consent:
Mechanisms:mass data collection, opaque data sharing, lack of user awareness.

15. Domain Example Ethical Concern
Recommender Systems
YouTube or TikTok algorithmic
feeds
Creates filter bubbles, influences
preferences without awareness
Deepfakes Political deepfake videos
Misinformation → affects voter
decisions, erodes trust
Political Ads
Micro-targeted campaigns using
psychographics
Covert persuasion → manipulates
behavior and decision-making
3. Autonomy & Manipulation
AI can manipulate choices and influence behavior, reducing human autonomy:
Mechanisms:persuasive AI, opaque profiling, personalization without consent.

16. Domain Example Ethical Concern
Autonomous Weapons Lethal drones
No clear accountability for
wrongful harm or civilian casualties
Healthcare AI Misdiagnosis by AI-assisted tools
Patient harm → difficult to assign
liability between doctor and
algorithm
4. Accountability Gap
High-stakes AI systems may lack clear responsibility, making harms difficult to assign:
Mechanisms:black-box algorithms, unclear governance, diffusion of responsibility.

17. Domain Example Ethical Concern
Large Language Models
GPT, BERT, image synthesis
models
High carbon footprint →
environmental degradation, energy
inequality
High-Frequency ML Systems
Continuous retraining of
recommendation or finance models
Unsustainable resource use →
hidden ecological costs
5. Environmental Cost
Training and deploying large AI models can consume massive energy, contributing to climate impact:
Mechanisms:compute-intensive training, inefficient model design, lack of lifecycle assessment.

18. Ethical Harm AI Example Affected Stakeholders
Auditing / Mitigation
Approaches
Bias & Discrimination
Hiring algorithms, credit scoring,
predictive policing (COMPAS)
Job applicants, loan seekers,
communities of color
Fairness audits (FPR/FNR,
demographic parity), bias
mitigation in datasets, human-in-
loop review, diverse training data
Privacy Invasion
Facial recognition, surveillance
capitalism (Facebook–Cambridge
Analytica)
Users, citizens, employees
Data governance audits, consent
tracking, privacy-preserving
techniques (differential privacy,
federated learning), regulatory
compliance (GDPR, CCPA)
Autonomy & Manipulation
Recommender systems,
deepfakes, micro-targeted political
ads
Voters, consumers, social media
users
Algorithmic transparency,
explainability tools
(SHAP/LIME), media literacy
programs, content provenance
verification, disclosure
requirements
Accountability Gap
Autonomous weapons, AI-assisted
healthcare diagnosis
Patients, civilians, military targets
Traceability of decisions, human
oversight, liability frameworks,
rigorous testing, independent
third-party audits
Environmental Cost
Large language models (GPT,
BERT), continuous retraining in
recommendation systems
Society at large, environment
Energy-efficient model design,
carbon footprint assessment, green
AI strategies, lifecycle impact
audits

19. CASE STUDY: COMPAS recidivism bias
1. Background
COMPAS (Correctional Offender Management Profiling for Alternative Sanctions)is a risk assessment algorithm widely used in the United
States criminal justice system to predict the likelihood of a defendant reoffending. It assigns risk scores that influence decisions such as pretrial
release, sentencing, and parole.
•Developed by Northpointe(now Equivant).
•Uses inputs like criminal history, age, employment, and social factors.
•Designed to help judges make data-driven decisions, but its use raised major ethical concerns.
2. Ethical Harms Identified
The COMPAS case highlights multiple ethical harmsin AI deployment:
a) Discrimination & Bias
Racial bias: A 2016 ProPublicainvestigation found that COMPAS overestimated recidivism risk for Black defendantsand underestimated it
for White defendants.
Black defendants who did not reoffend were twice as likely to be labeled high-riskcompared to White defendants.
White defendants who reoffended were often labeled lower risk.
Disparate impact: The algorithm inadvertently perpetuated systemic racial disparities in sentencing.
b) Transparency & ExplainabilityIssues
COMPAS is a proprietary “black-box” algorithm.
Defendants and even judges often cannot understand how scores are calculated.
Raises questions of due processand accountability.
c) Fairness Trade-offs
COMPAS satisfies predictive parityin aggregate (similar AUC across races), but fails equal false positive/negative rates, leading to ethical
tensions between different fairness definitions.
d) Ethical Harm on Individuals
Mislabeling may lead to longer sentences, harsher parole conditions, or denied release, impacting life outcomes.
Psychological harm: Being labeled high-risk may affect self-perception and reintegration chances.

20. 3. Auditing and Evaluation
Auditing AI tools like COMPAS requires multi-dimensional fairness and reliability assessments:
a) Audit Objectives
•Detect bias against protected groups(race, gender).
•Assess accuracy: false positives vs false negatives.
•Check calibration: do scores match actual recidivism probability?
•Ensure transparency: understand features influencing decisions.
b) Methods Used
1.Statistical Fairness Metrics
1.False Positive Rate (FPR): proportion of non-reoffenders incorrectly classified as high-risk.
2.False Negative Rate (FNR): proportion of reoffenders incorrectly classified as low-risk.
3.Predictive parity: risk score should predict recidivism equally across groups.
4.Demographic parity: similar proportion of high-risk labels across groups.
2.Independent Audits
1.Example: ProPublicavs Northpointedebate (2016):
1.ProPublicaemphasized FPR disparities(highlighting racial bias).
2.Northpointeemphasized predictive parity(argued algorithm is fair in aggregate).
2.Shows that fairness definitions can conflict, requiring context-aware audits.
3.Algorithmic Explainability
1.Tools like SHAP (SHapleyAdditive exPlanations)or LIMEcan estimate feature contributions.
2.Helps judges or auditors understand which factors contribute most to risk scores.
4.Policy and Procedural Audits
1.Evaluate how scores are used in decisions.
2.Assess whether human discretion mitigates or amplifies algorithmic bias.
4. Lessons Learned / Ethical Recommendations
Avoid “black-box” reliancein high-stakes domains: transparency is crucial.
Use multiple fairness metricsand contextualize results.
Regular auditingwith independent third parties is necessary.
Human-in-the-loop: judges should not rely solely on algorithm scores.
Bias mitigation techniques: recalibration, fairness constraints, or post-hoc adjustments.
Stakeholder involvement: communities affected by the system should have a voice in evaluation.

21. 1. Background
The Facebook–Cambridge Analyticascandal(2018) revealed the misuse of personal data from millions of Facebook users for political profiling
and targeted advertising.
•Cambridge Analytica, a political consulting firm, harvested data from ~87 million Facebook profileswithout informed consent.
•Data was used to create psychographic profilesto influence voter behavior in the 2016 U.S. presidential election and Brexitreferendum.
•The data collection exploited a Facebook app (“thisisyourdigitallife”)which collected data from users and their friends.
2. Ethical Harms Identified
The scandal highlighted several ethical harmsin digital and AI-enabled decision-making:
a) Privacy Violation
Users’ data were collected without explicit consent.
Facebook’s privacy policies and enforcement mechanisms were insufficient.
Sensitive personal attributes, including political and psychological profiles, were used for manipulation.
b) Manipulation & Autonomy
Micro-targeted political ads exploited psychological profiling.
Potentially manipulated voter behavior, undermining free and informed choice.
Raises questions of moral autonomy and digital consent.
c) Transparency & Accountability
Neither Facebook users nor regulators were fully aware of how data would be used.
Lack of transparency about algorithmic targeting mechanisms.
Difficult to hold entities accountable due to complex data-sharing chains.
d) Social and Political Harm
Erosion of trustin digital platforms.
Amplification of political polarization and misinformation.
Undermined democratic processes by exploiting data-driven manipulation.
CASE STUDY: Facebook–Cambridge Analyticascandal

22. 3. Auditing and Evaluation
Auditing practices in the Facebook–Cambridge Analyticacase highlight challenges in digital ethics, privacy, and algorithmic accountability:
a) Data Governance Audit
•Investigate data collection practices(consent, purpose limitation).
•Audit data sharing agreementsbetween platforms and third parties.
•Check compliance with regulatory frameworks(e.g., GDPR, FTC guidelines).
b) Algorithmic Transparency
•Audit the ad targeting algorithmsand psychographic models used.
•Assess whether algorithms amplify bias, misinformation, or manipulation.
•Use explainabilitytools to understand which features influence targeting.
c) Risk Assessment
•Evaluate the potential harms to users: privacy loss, psychological manipulation.
•Assess the broader societal impact, including misinformation and political influence.
d) Regulatory & Ethical Audits
•Facebook faced:
•FTC fine of $5 billionfor privacy violations (2020).
•Increased scrutiny under GDPRin Europe.
•Independent audits recommended stronger user consent, third-party access control, and transparency mechanisms.
4. Ethical Lessons and Recommendations
Consent and Transparency
Users must have clear, informed consentfor data collection and processing.
Platforms should disclose how algorithms use personal data.
Data Minimization & Purpose Limitation
Only collect data necessary for the declared purpose.
Avoid indirect harvesting from friends or social networks without consent.
Third-Party Oversight
Any external firm accessing platform data must be audited and accountable.
Algorithmic Accountability
Ensure that ad-targeting and recommendation algorithms are auditable, explainable, and fair.
Regulatory Compliance
Align with frameworks like GDPR, CCPA, and emerging AI ethics guidelines.
User Empowerment
Give users control over their data and visibility into how it influences content or ads.

23. CASE STUDY: Deepfakeelections threat
1. Background
Deepfakesare AI-generated synthetic media—images, audio, or videos—that convincingly impersonate real people. In the context
of elections:
•Deepfakescan be used to manipulate political messages, create fake speeches, or depict candidates in compromising situations.
•Advances in GANs (Generative Adversarial Networks)and AI video synthesis have made realistic deepfakesmore accessible.
•Threats are global: reported incidents include deepfakevideos targeting politicians in the U.S., India, and Europe.
2. Ethical Harms Identified
Deepfakesin elections raise multiple ethical concerns:
a) Misinformation & Voter Manipulation
Fake content can spread false information, influencing voter perception and decisions.
Can undermine democratic processesby promoting false narratives.
b) Trust Erosion
Repeated exposure to deepfakesreduces trust in media, candidates, and institutions.
Leads to “liar’s dividend”, where real statements can be disbelieved because deepfakesexist.
c) Psychological Harm
Candidates and political figures can suffer reputational damage, harassment, or threats.
Voters can experience confusion and anxietydue to indistinguishable real vs fake content.
d) Disproportionate Targeting
Vulnerable populations can be targeted with personalized misinformationusing AI profiling.
Raises ethical fairness concernsin political communication.

24. 3. Auditing and Detection Mechanisms
Auditing deepfakethreats involves technical, procedural, and regulatory measures:
a) Technical Audits
Deepfakedetection algorithmsanalyze inconsistencies in video or audio, such as:
•Facial microexpressions
•Lip-sync mismatches
•Image artifacts
•Audio anomalies
AI-based forensic tools: Microsoft Video Authenticator, DeepwareScanner, SensityAI.
b) Social Media Audits
Platforms must audit content sharing mechanisms, flagging potential deepfakedistribution.
Track propagation patternsto identify coordinated misinformation campaigns.
c) Transparency and Provenance
Use digital watermarks or blockchain-based provenanceto authenticate legitimate media.
Establish source verification policiesfor political ads and candidate communications.
d) Regulatory and Policy Audits
Governments and electoral bodies can enforce:
•Disclosure requirementsfor AI-generated political content.
•Penalties for malicious disseminationof deepfakes.
•Guidelines under Election Commission or data protection frameworks.
4. Ethical Lessons and Recommendations
Early Detection & Monitoring
Continuous monitoring of political content using AI-assisted tools.
Media Literacy Programs
Educate voters to recognize manipulated media.
Transparency in Political Messaging
Candidates and parties should disclose AI-generated content clearly.
Third-Party Audits
Independent agencies should audit platforms for deepfakedetection and response.
Cross-Border Collaboration
Deepfakecampaigns often originate internationally; cooperation between governments and platforms is crucial.
Robust Legal Frameworks
Update election laws to criminalize malicious deepfakedisseminationwhile protecting freedom of expression.

25. Auditing AI Models -What is AI auditing?
Ethical compliancemeans aligning AI development and deployment with values such as fairness, transparency, accountability,
privacy, and safety.
Legal compliancemeans adhering to laws and regulationsgoverning data protection, discrimination, safety, and intellectual
property (e.g., GDPR, EU AI Act, IEEE Ethically Aligned Design, OECD AI Principles). Together, they ensure trustworthy
and responsible AI.
The Compliance Checking Process
Stage Focus Area Example Activities
A. Design & Development Ethics-by-design
Include ethics and risk assessment during model design
Ensure data diversity and fairness
Document intended use and limitations
B. Data Collection & ProcessingLegal data handling
Obtain informed consent
Apply anonymization, differential privacy
Check for bias and representativeness
C. Model Training & ValidationTechnical fairness testing
Use fairness metrics (e.g., demographic parity, equalized odds)
Test explainabilityand robustness
Document model cards and datasheets
D. Deployment & Monitoring Ongoing compliance
Conduct algorithmic audits
Monitor for drift, bias, and misuse
Maintain human oversight and accountability logs
E. Review & Reporting Governance & transparency
Maintain compliance reports
Allow external audits
Disclose system impacts to regulators and public

26. Key Dimensions of Ethical & Legal Compliance
Dimension Objective How to Check / Audit
Fairness & Non-discriminationAvoid unfair bias
Fairness metrics, bias audits, representative
datasets
Transparency & Explainability
Ensure decisions are
interpretable
Use explainable AI (XAI) tools (e.g., LIME,
SHAP), publish model cards
Accountability Assign clear responsibilityDocumentation, traceability logs, audit trails
Privacy & Data ProtectionRespect user rights
GDPR/CCPA compliance, privacy impact
assessment
Safety & Reliability Prevent harm or malfunction
Robustness testing, red teaming, simulation
testing
Environmental SustainabilityReduce carbon impact Energy audits, green AI reporting
Human Oversight Maintain human control
Define escalation processes, human-in-loop
decision review

27. Frameworks and Standards Used for Compliance Checking
Framework / Regulation Scope Key Compliance Elements
EU AI Act (2024) High-risk AI regulation
Risk classification, conformity
assessment, transparency obligations
GDPR Data protection (EU)
Data consent, right to explanation,
lawful processing
OECD AI Principles Global ethical guidanceFairness, transparency, robustness
IEEE 7000 Series Ethical design standards
Value-based engineering, human
rights considerations
NIST AI Risk Management
Framework (2023)
U.S. voluntary
framework
Risk mapping, governance, bias
management

28. Tools and Techniques for Auditing
Tool/Technique Purpose
Model Cards & Datasheets Document dataset/model provenance, limitations, biases
Fairness Toolkits(IBM AI Fairness 360, Google
What-If, Fairlearn)
Quantify and mitigate algorithmic bias
ExplainabilityTools(SHAP, LIME) Interpret model predictions
Privacy Audits Verify compliance with GDPR, CCPA
Carbon Trackers
Estimate model training emissions (e.g., ML CO₂ Impact
Tracker)
Ethical Impact Assessments (EIA)
Holistic check on social, legal, and environmental
consequences
Continuous Compliance & Governance
Compliance isn’t one-time —it’s continuousthroughout the AI lifecycle:
•Periodic re-audits and bias re-evaluation
•Impact assessmentsbefore and after deployment
•Stakeholder engagement(users, regulators, ethicists)
•Incident reporting mechanismsfor harm or misuse

29. Section Description Example (Spam Detection Model)
Model Details
Name, version, developer, date, contact info,
licensing, algorithm type
Spam Classifier v2.1 (BERT-based), Google
Research
Intended Use
Primary use cases, target users, context of
deployment
Email spam filtering for English text, personal
inboxes
Factors / Limitations
Known factors that may influence performance:
demographic, linguistic, geographic
May not perform well on code-switched (English-
Hindi) text
Metrics
Evaluation metrics used (accuracy, F1, AUC,
fairness metrics)
F1 = 0.94; False positive rate = 2%
Evaluation Data
Datasets, preprocessing, splits, demographic
composition
Public Enron email dataset (60K messages)
Training Data Overview of training data sources, labeling processLabeled internal email corpus (anonymized)
Ethical Considerations Possible harms, misuse scenarios, data sensitivity
Risk of classifying newsletters as spam; may impact
outreach emails
Caveats & RecommendationsKnown issues and maintenance notes
Re-evaluate quarterly; monitor drift in slang/spam
patterns
Model Cards

30. Example (Condensed Model Card)
Model:Toxic Comment Classifier (BERT fine-tuned)
Version:1.3
Owner:Google Jigsaw
Date:October 2025
Intended Use:
Identify and flag toxic language in online comments for moderation.
Limitations:
May exhibit lower recall for underrepresented dialects and minority languages.
Performance Metrics:
•AUC: 0.96 (overall)
•FNR (African-American English subset): 8% higher than global average
Ethical Considerations:
Potential risk of over-censorship; continuous monitoring required.
Caveats:
Model not suitable for automated banning decisions —should support human moderation.

31. A simplified workflow using AIF360 might look like:
1.Define the protected attributes / sensitive groups(e.g. gender, race) in your dataset.
2.Load datainto the toolkit’s format, prepare features, labels, and encode protected attributes.
3.Compute fairness metricsto see how biased the data or model is (e.g., is there a large disparity across groups?).
4.Choose mitigationstrategy depending on where you want to intervene (pre-, in-, or post-processing).
5.Train / adjust model or datausing chosen algorithm.
6.Evaluate fairness again + check trade-offs(e.g. fairness vs accuracy).
7.Visualize/show results / produce reportsfor stakeholders.
Strengths
Very comprehensivemetric library and mitigation algorithm set.
Flexibility: multiple stages to apply mitigation, and support for both data and model adjustments.
Strong tooling / documentation and examples which lower barriers.
Open source, permissive license (Apache v2), so usable in academic and industrial settings.
Limitations / Challenges
Choice Overload: With ~70 metrics and many algorithms, picking the right metricand appropriate mitigation methodfor your scenario can be
confusing.
Trade-offs: Mitigating bias often leads to trade-offs (e.g., loss of predictive accuracy). Some methods might degrade performance for somegroups
more than others.
Context Sensitivity: Fairness is domain-and culture-dependent. Metrics that make sense in one context (e.g. finance) might be less meaningful in
another (e.g. healthcare) or require different definitions of fairness.
Protected Attributes / Proxy Variables: Sometimes data doesn’t have explicit protected attributes, or they are imperfect proxies. The toolkit can
help detect proxies, but handling them well is non-trivial.
Scalability / Performance: Some algorithms are computationally intensive, especially on large datasets.
Not a complete solution for governance / regulation: It helps with technical fairness, but organizational, legal, ethical aspects require additional
structures (policy, oversight, external auditing, etc.)
IBM AI Fairness 360

32. FairLearn
FairLearnis an open-source toolkit developed by Microsoftthat helps evaluate and mitigate unfairnessin AI and ML models.
It enables data scientists and organizations to build systems that make equitable decisionsacross different demographic groups,
ensuring ethical transparencyand social responsibilityin AI deployment.
Key Features:
Fairness Assessment (Fairness Dashboard):
Visualizes model performance across sensitive attributes such as gender, age, or ethnicity.
Detects disparities in accuracy, false positives, or other error metrics among subgroups.
Enables stakeholders to see bias patterns before deployment.
Bias Mitigation (Reduction Algorithms):
Implements algorithms like ExponentiatedGradientand Grid Searchto balance accuracy with fairness.
Supports multiple fairness definitions (e.g., demographic parity, equalized odds).
Allows customization based on social or regulatory priorities.
Integration with ML Ecosystems:
Works seamlessly with frameworks such as scikit-learn, TensorFlow, and PyTorch.
Compatible with other responsible AI tools like AI Fairness 360and SHAPfor interpretability.
Community & Transparency:
Backed by an active research and developer community under Microsoft’s Responsible AIinitiative.
Encourages openness and reproducibility in fairness evaluation.
Outcome:
By combining quantitative fairness metricswith ethical decision-making, FairLearnempowers organizations to create AI
systems that are not only accurate but also accountable and inclusive—supporting the broader goal of TechnoSustainthrough
fairness-aware innovation.

33. Typical usage steps are:
1.Have a trained model(TensorFlow, or via AI Platform: XGBoost/ scikit-learn) and a dataset of inputs + ground truth.
2.Load the data into the tool(via TensorBoard, Jupyter/Colab, or AI Platform Notebooks).
3.Setup the What-If Tool config, specifying which feature is the target, and optionally a comparison model.
4.Explore using the tool:
1.Edit datapointsor create counterfactuals to see what changes are needed to flip predictions.
2.Inspect model performance over entire dataset and over slices.
3.Visualize metrics, fairness measures, confusion matrices etc.
4.See how feature changes impact predictions (partial dependence).
5.Use insightsto improve the model: e.g. identify feature engineering issues, fix bias, adjust model thresholds, or even retrain.
Feature What It Enables
Datapointediting & counterfactuals
You can take a single example in your data, tweak its features, and see how the model’s prediction
changes. Also, find “the nearest counterfactual” —a data point similar to your selected one but with a
different predicted outcome.
Partial dependence plots / feature
sensitivity
Shows how predictions vary when you change one feature gradually, helping understand feature effects.
Performance & fairness slicing
You can slice the dataset by different feature values (e.g. gender, age, etc.), compare performance metrics
across those slices, detect bias or subgroup performance issues.
Visualization tools
Histograms, scatterplots, confusion matrices, ROC curves etc. to explore model behavioracross dataset
slices.
Model comparisons
Compare two models (say, different versions or different frameworks) on the same dataset to see which is
better along different dimensions.
Support for multiple model typesWorks with TensorFlow, and also supports Scikit-learn and XGBoostwhen integrated via AI Platform.
Google What If Tool

34. LIME (Local Interpretable Model-Agnostic Explanations)
1.Select an instance (input) to explain.
2.Generate perturbed samples(slight variations) around that
instance.
3.Get predictions from the original model for those samples.
4.Fit a simple interpretable model(usually linear or decision
tree) on these samples and their predictions.
5.The coefficients of this local surrogate model indicate feature
importance.
SHAP (SHapleyAdditive exPlanations)
1.Treat features as playersin a cooperative game where the
“payout” is the prediction.
2.Compute each feature’s average contributionacross all
possible feature combinations.
3.The result is a set of Shapley values, showing how much
each feature adds/subtracts from the base prediction.
4.These values are both consistent(mathematically fair) and
additive(sum to total prediction).
Aspect SHAP (SHapley Additive exPlanations)
LIME (Local Interpretable Model-agnostic
Explanations)
Origin
Based on cooperative game theory
(Shapley values).
Based on local surrogate (linear) models.
Goal
Explain each predictionby fairly
attributing feature contributions.
Approximate model’s local behavior around a
prediction using a simple interpretable model.
Model Type
Model-agnostic (works with any ML
model), with optimized versions for trees,
deep models, etc.
Model-agnostic (works with any ML model).
Primary Output
Consistent contribution values for each
feature (global + local interpretability).
Local explanations showing which features
influenced a specific prediction most.

35. # Example: Explaining a model prediction using SHAP and LIME
import shap
import lime
import lime.lime_tabular
from sklearn.ensembleimport RandomForestClassifier
from sklearn.datasetsimport load_iris
from sklearn.model_selectionimport train_test_split
X, y = load_iris(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test= train_test_split(X, y, random_state=42)
model = RandomForestClassifier().fit(X_train, y_train)
# ---SHAP ---
explainer = shap.TreeExplainer(model)
shap_values= explainer.shap_values(X_test)
shap.summary_plot(shap_values, X_test)
# ---LIME ---
explainer_lime= lime.lime_tabular.LimeTabularExplainer(
training_data=X_train.values,
feature_names=X.columns,
class_names=['setosa','versicolor','virginica'],
mode='classification'
)
exp= explainer_lime.explain_instance(X_test.iloc[0].values, model.predict_proba)
exp.show_in_notebook(show_table=True)

36. Ethical Harms in AI
Ethical harmsarise when AI systems cause unintended negative consequencesthat affect individuals, groups, or society.
These harms often emerge from biased data, opaque algorithms, poor governance, or lack of oversight.
Major Categories of Ethical Harms in AI
Category Description Example Use Case Ethical Concern
Bias & Discrimination
Unequal treatment of
individuals due to biased
data or design
Hiring algorithms,
predictive policing
(COMPAS)
Reinforces gender, racial,
or social inequalities
Privacy Invasion
Unauthorised data
collection, tracking, or
profiling
Facial recognition, social
media analytics
Loss of autonomy, identity
theft, surveillance
Autonomy &
Manipulation
AI influences or
manipulates user decisions
Recommender systems,
deepfakes, targeted ads
Undermines free will,
informed choice
Accountability Gap
Lack of clarity on who is
responsible for harm
Autonomous vehicles,
medical AI misdiagnosis
No clear liability or redress
for harm
Environmental Cost
High energy use and
emissions from large-scale
AI
Training large language
models (GPT, BERT)
Carbonfootprint,
unsustainableAI
infrastructure

37. Why Ethical Auditing Matters
Ethical auditing ensures that AI systems are:
•Fair and unbiased
•Transparent and explainable
•Legally compliant
•Socially and environmentally responsible
It’s the process of systematically evaluating AI systemsto identify, measure, and mitigate ethical risks.
Type of Audit Purpose What It Checks Example Tools / Methods
Fairness Audit Identify bias/discrimination
Demographic parity, false
positive/negative rates
IBM AI Fairness 360,
Fairlearn
Transparency Audit
Check explainability and
decision logic
Model interpretability,
feature influence
LIME, SHAP, Model Cards
Privacy Audit
Ensure compliance with
privacy laws
Data collection, storage,
consent
GDPR audit checklist,
Privacy Impact Assessments
Accountability Audit
Assign responsibility and
traceability
Governance structure,
decision records
AI Incident databases,
accountability reports
Environmental Audit Measure ecological impact
Energy consumption, carbon
footprint
ML CO₂ Impact Tracker,
Green AI metrics

38. Stage Goal Activities
1. Scoping & Risk IdentificationIdentify potential ethical harms
Impact assessment, stakeholder
mapping
2. Data & Model Analysis Examine data and model biasDataset audit, fairness testing
3. Explainability EvaluationCheck interpretability Use XAI tools, transparency reports
4. Governance & Accountability
Review
Ensure clear ownership Policy audit, documentation check
5. Continuous Monitoring Track post-deployment effects
Drift detection, retraining, ethical
re-evaluation
Ethical Auditing Process
Outcomes of Ethical Auditing
✅Improved trustworthinessand accountability
✅Early detection of bias or discrimination
✅Alignment with legal and ethical frameworks
✅Enhanced public confidenceand social acceptanceof AI

39. Bias Detection and Fairness Checks
AI models can unintentionally learn social or demographic biases from training data.
Bias detectioninvolves measuring how outcomes differ across groups (e.g., gender, race,
age), while fairness checksaim to ensure equitable model performance.
Key steps include:
•Data Audit:Examine datasets for imbalanced representation.
•Metric Analysis:Evaluate fairness metrics like demographic parityor equal
opportunity.
•Model Testing:Simulate diverse input scenarios to detect bias.
•Mitigation:Apply techniques such as reweighing, adversarial debiasing, or fairness-
aware learning.
Tools: IBM AI Fairness 360, Fairlearn, What-If Tool (Google)

40. Transparency & Interpretability audits
•Transparency auditevaluates how clearly the AI system’s functioning, data sources, and
decision logic are documented and communicated to stakeholders.
•Interpretability auditassesses whether humans (developers, regulators, or end-users) can
understand, explain, and justifythe model’s outputs or predictions.
•Detect black-box behaviorand opaque decision-making.
•Verify documentation(model cards, datasheets, audit trails).
•Ensure explainabilityof outputs for affected users.
•Support compliancewith ethical and legal frameworks (e.g., EU AI Act, GDPR “right to
explanation”).
•Enhance accountabilityand trust in AI systems.

41. Audit Components
Dimension Transparency Checks Interpretability Checks
Data
Source documentation, collection
consent, preprocessing pipeline, bias
checks
Feature importance mapping, data
influence explanations
Model
Architecture disclosure, training
process, version control
Use of explainable AI (XAI)
methods (e.g., SHAP, LIME,
attention visualization)
Process
Documentation of model lifecycle
(development → deployment)
Human-in-the-loop explanations,
decision pathway visibility
Decision Outputs
Explanation reports for
users/regulators
Counterfactual or example-based
reasoning
Governance
Internal accountability roles, risk
registers
Explainabilitymetrics (fidelity,
consistency, stability)

42. Tools and Frameworks
•Model Cards (Google AI)–summarize model purpose, performance, limitations.
•Datasheets for Datasets (MIT & Google)–document data provenance and ethics.
•Explainable AI libraries–SHAP, LIME, ELI5, Captum(for PyTorch), InterpretML.
•Audit frameworks–AI Fairness 360 (IBM), Fairlearn(Microsoft), Open Ethics Toolkit.
Audit Process
•Scoping:Identify models and stakeholders to audit.
•Documentation Review:Check completeness and transparency artifacts.
•Interpretability Testing:Apply XAI techniques to assess model explainability.
•Bias & Performance Analysis:Ensure decisions are fair and explainable.
•Reporting:Prepare audit report highlighting interpretability gaps, risks, and mitigation plans.
•Continuous Monitoring:Re-audit after updates or retraining.

43. Case Example
Healthcare Predictive Model Audit:
•Transparency issue: undocumented feature engineering.
•Interpretability fix: applied SHAP to show that age and
blood pressure strongly influenced predictions, improving
clinician trust and regulatory acceptance.
Challenges
•Trade-off between accuracy and interpretability
(especially in deep learning).
•Lack of standardized audit protocolsacross domains.
•Difficulty in explaining complex non-linear models.
•Risk of information overloadin transparency reports.
Future Directions
•Integration of automated audit pipelines.
•Regulatory audit standards(ISO/IEC 42001: AI
Management System).
•Use of interpretable-by-design architectures.
•Public AI transparency registriesmandated by law.

44. Outcome audit (impact analysis)
An Outcome Audit—also called Impact Analysis—focuses on evaluating the real-world consequencesof
deploying AI systems. It goes beyond technical accuracy to assess social, ethical, and environmental outcomes.
Key objectives include:
•Measuring real-world impact:How AI decisions affect individuals, groups, or ecosystems.
•Identifying unintended harms:Detecting bias, discrimination, or exclusionary effects post-deployment.
•Assessing sustainability and safety:Evaluating energy use, long-term societal implications, and
environmental footprint.
•Stakeholder engagement:Incorporating feedback from users, affected communities, and domain experts.
•Transparency and accountability:Publishing results through Impact Assessmentsor AI Audit Reportsto
ensure responsible governance.
•Example:
An AI model used in healthcare diagnostics should be audited not just for accuracy but also for equitable
access, patient trust, and reduction in diagnostic disparities.

45. Accountability & Governance Mechanisms
AI Ethics committees in companies
To promote responsible innovation, many leading organizations have established AI Ethics Committeesor Responsible AI
Boardsthat oversee the design, deployment, and impact of AI systems.
Purpose:
AI Ethics Committees ensure that AI systems align with organizational values, legal norms, and societal expectations. They act as
a bridge between technology teams, policy experts, and the public interest.
Key Functions:
•Policy Oversight:Define and update ethical AI guidelines.
•Risk Assessment:Review projects for fairness, transparency, and potential societal harm.
•Accountability:Recommend actions when ethical breaches occur.
•Diversity in Perspectives:Include multidisciplinary members —AI researchers, ethicists, sociologists, legal experts, and
external advisors.
•Training & Awareness:Promote a culture of ethical responsibility among employees and developers.
Examples:
•Google:“Responsible Innovation” review process led by the Responsible AI team.
•Microsoft:“AetherCommittee” (AI, Ethics, and Effects in Engineering and Research).
•IBM:“AI Ethics Board” guiding its principles of transparency and accountability.
•Facebook (Meta):“Responsible AI” team focusing on fairness and inclusivity.
Outcome:
AI Ethics Committees ensure that AI technologies advance human welfare, maintain public trust, and support sustainable
innovationthrough continuous evaluation and accountability.

46. Accountability & Governance Mechanisms
External audits & certification (like financial audits)
As AI systems increasingly influence critical domains such as healthcare, finance, and governance, the need for independent
evaluationhas become essential. External audits and certification processes serve as third-party assessmentsthat verify whether
AI models comply with ethical, legal, and technical standards—much like financial or cybersecurity audits.
Purpose:
To ensure that AI systems operate safely, fairly, and transparently, and that organizations remain accountablefor their AI-driven
decisions.
Key Components:
•Compliance Verification:Assess adherence to frameworks such as the EU AI Act, ISO/IEC 42001 (AI Management System
Standard), or OECD AI Principles.
•Bias and Fairness Auditing:Evaluate datasets and model outputs for discrimination or unfair treatment.
•Robustness & Security Checks:Validate model stability under adversarial or uncertain conditions.
•Transparency Review:Confirm explainability, documentation quality (e.g., model cards, data sheets).
•Certification & Reporting:Provide official recognition or public audit reports for regulatory or reputational assurance.
Examples:
•Algorithmic Auditing Firms:Organizations like ORCAAand Ethical Intelligenceoffer AI audit services.
•Regulatory Standards:ISO/IEC 42001:2023 sets guidelines for responsible AI management.
•EU AI Act (2024):Mandates third-party conformity assessment for high-risk AI systems.
Outcome:
External audits establish trust and accountability, ensuring that AI technologies meet societal and environmental expectations,
just as financial audits sustain economic credibility.

47. Accountability & Governance Mechanisms
Human-in-the-loop & redress mechanisms
As AI systems become more autonomous, human oversightremains essential to ensure ethical accountability, fairness, and
reliability.
Human-in-the-loop (HITL)approaches and redress mechanismsform the backbone of responsible AI governance, ensuring
that technology serves humanity —not replaces it.
1. Human-in-the-Loop (HITL) Systems
A Human-in-the-Loopmodel integrates human judgment at critical stagesof AI decision-making —during design, training, or
deployment.
It ensures that the system’s outputs align with human values, context, and ethics.
Key Features:
Oversight & Control:Humans validate or override AI outputs in sensitive applications (e.g., healthcare diagnosis, credit scoring).
Feedback Loops:Continuous learning from human corrections improves model robustness and trust.
Ethical Safeguard:Prevents automation bias and protects individuals from opaque, unchallengeable decisions.
Examples:
Radiologists verifying AI-assisted medical imaging results.
Loan officers reviewing automated credit approval recommendations.
Content moderators refining AI-driven content classification.

48. 2. Redress Mechanisms
Redress mechanismsprovide individuals with the right to contest or appeal AI decisions, fostering
transparency, accountability, and justice.
They are crucial when AI-driven outcomes affect human rights, employment, access to resources, or healthcare.
Core Principles:
Explainability:Individuals should understand how a decision was made.
Appeal Process:A clear channel to challenge or correct unfair or incorrect AI outputs.
Responsibility Assignment:Defined roles for who must respond and rectify issues.
Restorative Action:Ensuring harm correction —through retraining models, policy changes, or compensatory
measures.
Examples:
The EU GDPR mandates a “right to explanation” for automated decisions.
AI ethics frameworks (OECD, UNESCO) recommend institutional grievance redressalpathways.
Outcome:
Together, Human-in-the-Loopand Redress Mechanismssafeguard the human-centered nature of AI,
ensuring systems remain transparent, accountable, and equitable—especially in high-stakes domains like
biotechnology, healthcare, and environmental sustainability.

49. Principles for Ethical Practices in AI
Core principles(based on IEEE, OECD, EU, NITI Aayog):
Fairness
Accountability
Transparency & Explainability
Privacy & Security
Human-Centricity & Non-Maleficence
Sustainability
Implementation strategies:
Dataset documentation (Datasheets for Datasets)
Model Cards (Google)
Human-in-the-loop systems
Explainable AI (XAI)
Comparative Frameworks
EU AI Act vs OECD Principles vs India’s Responsible AI
Mapping ethical principles to technical practices
Case Reflection:
IBM’s withdrawal of facial recognition tools citing bias & misuse concerns.

50. Case Studies & Applications
Case Study 1: Autonomous Vehicles
Ethical dilemmas:
The Trolley Problem in AI decision-making
Passenger safety vs pedestrian safety
Responsibility in accidents → Manufacturer, Programmer, User, Insurer?
Bias in training data:Road conditions, geography, demographics
Accountability mechanisms:ISO standards, UN regulations, Tesla autopilot
controversies
Activity (Roleplay):
Students act as Judge, Lawyer, Tech Expert, Citizenin a mock trial of a self-
driving car crash.

51. Case Studies & Applications
Case Study 2: Healthcare Robots
Benefits:Surgery assistance, eldercare, rehabilitation, diagnostics
Ethical issues:
Patient autonomy & consent
Bias in medical diagnosis (trained on limited demographics)
Data privacy (HIPAA, GDPR)
Liability: doctor, hospital, or robot manufacturer?

52. Case Example: IBM Watson in Oncology (criticized for unsafe recommendations)
Background
•IBM Watson for Oncology was developed in partnership with Memorial Sloan Kettering Cancer Center (MSKCC).
•Goal: Use Watson’s natural language processing and machine learning to provide personalized cancer treatment
recommendations.
•Marketed globally (India, China, South Korea) as a decision-support tool for oncologists.
Criticisms & Failures
Unsafe or Incorrect Recommendations
Reports surfaced (STAT News, 2018) that Watson often suggested unsafe or erroneous cancer treatments, including
options that would harm patients.
Example: recommending inappropriate chemotherapy for patients with bleeding risks.
Training Data Bias
System trained on synthetic “hypothetical” patient dataand heavily dependent on guidelines from MSKCC doctors.
This created narrow expertise, not generalizable to diverse global populations.
Transparency Gaps
Clinicians could not understand whyWatson suggested certain treatments (black-box issue).
Limited explainabilityreduced doctors’ trust.
Commercial Pressure vs. Clinical Readiness
Rushed to market to showcase IBM’s AI leadership.
Prioritized branding and salesover thorough clinical validation.
Global Deployment Concerns
Sold to hospitals in countries with fewer oncologists (India, South Korea), where clinicians relied more heavily on it →
higher risksin low-resource settings.

53. Ethical Lessons
Fairness:Training on narrow, Western-centric data led to biased outcomes, not suitable for global populations.
Transparency:Lack of explainabilityeroded medical trust.
Accountability:IBM marketed the system as safe, but who is liable when AI makes unsafe recommendations —the developer
(IBM), hospital, or clinician?
Patient Safety vs. Corporate Ambitions:Example of ethics washing, where AI was promoted beyond its tested capacity.
Case Example: IBM Watson in Oncology (criticized for unsafe recommendations)

54. Da Vinci Surgical System
The Da Vinci Surgical Systemis a state-of-the-art robotic platform designed to enable surgeons to perform
minimally invasive surgeries with high precision, dexterity, and control
1. Overview
Developed by Intuitive Surgical, first introduced in 2000.
It translates a surgeon’s hand movements at a console into smaller, precise movements of instruments inside the
patient.
Primarily used for urological, gynecological, cardiothoracic, and general surgeries.
2. Components
Surgeon Console:
Where the surgeon sits and controls the robot.
Provides a high-definition 3D viewof the surgical field.
Hand and foot controls manipulate robotic arms.
Patient-side Cart:
Contains robotic armsthat hold surgical instruments and cameras.
Instruments have EndoWristtechnology, allowing 7 degrees of freedom, mimicking the human wrist but with greater range.
Vision System:
Provides high-definition, magnified 3D imaging.
Allows precise dissection and suturing.
Instruments:
Interchangeable and designed for minimally invasive procedures.
Include scissors, needle drivers, graspers, electrocautery tools, etc.

55. 3. Advantages
Precision: Tremor filtration and fine movements.
Minimally Invasive: Smaller incisions → less pain, reduced blood loss, faster recovery.
Enhanced Visualization: 3D HD imaging improves surgical accuracy.
Ergonomics: Surgeon operates while seated, reducing fatigue.
4. Limitations
High Cost: Both initial investment and per-procedure instrument costs.
Learning Curve: Surgeons need specialized training.
Setup Time: Longer setup than traditional laparoscopic surgery.
Lack of Haptic Feedback: Surgeons rely on visual cues instead of tactile sensation.
5. Common Procedures
Urology: Prostatectomy, kidney surgery.
Gynecology: Hysterectomy, myomectomy.
Cardiothoracic: Mitral valve repair, thoracic surgery.
General Surgery: Colorectal surgery, hernia repair, bariatric procedures.

56. Activity:
Group analysis: Design ethical guidelines for a hospital deploying healthcare robots.
Pick an AI system (education, banking, policing, etc.) → Assess fairness, ethical
principles, and accountability.