[DSC DACH 24] Harnessing AI for Sustainable Results: Strategies for Long-Term Success - Johannes Schauer

In today's world, the urgency to apply AI is more pronounced than ever. Yet, many projects get stuck at the Proof of Concept stage or fail to deliver the desired value, leading to premature discontinuation, and demotivating stakeholders and data teams alike. This talk addresses this challenge by providing practical insights on transitioning AI solutions from concept to sustained, impactful deployment. Join us to explore effective strategies for creating AI solutions that are viable in the long term. We will discuss aligning AI initiatives with business and user goals to achieve meaningful impact and share insights on identifying alarm signals and mitigating risks throughout project execution. Let's transform AI projects into enduring successes and continuously drive value for our organizations.

1. Harnessing AI for
Sustainable Results
Strategies for Long-Term Success
DSC DACH | Vienna | Sept. 12, 2024
Dr. Johannes Schauer

2. The AI Urgency

3. The AI Urgency
20
47
58
50
56
50
55
72
33
65
0
10
20
30
40
50
60
70
80
2017 2018 2019 2020 2021 2022 2023 2024
Organizations that have adopted AI in at least one business function (% of respondents)
Use of generative AI
Adoption of AI
Source: McKinsey, 2024, The State of AI

4. 83-92 % of AI projects fail.
2x the failure rate of corporate IT projects.
The Challenge of Failing AI Projects
Source: Fortune Magazine, 2023, Want your company’s A.I. project to succeed?

5. 83-92 % of AI projects fail.
2x the failure rate of corporate IT projects.
The Challenge of Failing AI Projects
Source: Fortune Magazine, 2023, Want your company’s A.I. project to succeed?
Let‘s focus on root causes & mitigations!

6. Why AI Projects Typically Fail
Leadership-driven
root causes
Organizational
root causes
Data-driven
root causes
Source: RAND, 2024, The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed: Avoiding the Anti-Patterns of AI
From a Practitioner‘s POV

7. Why AI Projects Typically Fail
Leadership-driven
root causes
Organizational
root causes
Data-driven
root causes
•Using AI to solve simple
problems
•Optimizing for the wrong
business problem
•Overconfidence in AI
•Underestimating the time
commitment needed
Source: RAND, 2024, The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed: Avoiding the Anti-Patterns of AI
From a Practitioner‘s POV

8. Why AI Projects Typically Fail
Leadership-driven
root causes
Organizational
root causes
Data-driven
root causes
•Using AI to solve simple
problems
•Optimizing for the wrong
business problem
•Overconfidence in AI
•Underestimating the time
commitment needed
•Too few data engineers
•Too little AI talent
•Lack of domain
understanding
•Underinvestment in
infrastructure
Source: RAND, 2024, The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed: Avoiding the Anti-Patterns of AI
From a Practitioner‘s POV

9. Why AI Projects Typically Fail
Leadership-driven
root causes
Organizational
root causes
Data-driven
root causes
•Using AI to solve simple
problems
•Optimizing for the wrong
business problem
•Overconfidence in AI
•Underestimating the time
commitment needed
•Too few data engineers
•Too little AI talent
•Lack of domain
understanding
•Underinvestment in
infrastructure
•Lack of suitable data
•Too low data quality
•Unbalanced data or
too little data coverage
•Immature technology
Source: RAND, 2024, The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed: Avoiding the Anti-Patterns of AI
From a Practitioner‘s POV

10. Why AI Projects are Actually Stopped

11. Triangle of Stakeholders
Management
Development Domain experts
AI projects require
good communication
among all stakeholders
to achieve best results

12. Choose Problem Worth Solvingwith AI
Things to look for before committing
•Business impact
•Ease in finding / straightforward
•Long term relevance
•Good reason for AI
•Data availability
•Ethical & regulatory compliance

13. Choose Problem Worth Solvingwith AI
Things to look for before committing
•Business impact
•Ease in finding / straightforward
•Long term relevance
•Good reason for AI
•Data availability
•Ethical & regulatory compliance
Red flags
•Tech is the driver –not the problem
•Simply copying solutions from others

14. Ensure the Problem is Real for Users
•The majority of today‘s AI solutions
are built for human users
•But are users also heard enough?

15. Ensure the Problem is Real for Users
•The majority of today‘s AI solutions
are built for human users
•But are users also heard enough?
1.Research the problem
•Interviews, shadowing, competitor analysis, …
•Who? Why? How?
2.Validate solution early
•Visual & technical prototypes
•What’s good/bad/missing?
Stop if the problem is not real!

16. Business Relevance Drives Sustainability of AI Projects
Strategic alignment Business value

17. Business Relevance Drives Sustainability of AI Projects
•Does your project fit into your company‘s
strategic objectives / vision / mission?
•How does your project contribute?
•E.g.enabling CO2 reduction in operations
•E.g.addressing new customers
Strategic alignment Business value

18. Business Relevance Drives Sustainability of AI Projects
•Does your project fit into your company‘s
strategic objectives / vision / mission?
•How does your project contribute?
•E.g.enabling CO2 reduction in operations
•E.g.addressing new customers
•Which core business value does your
project aim to provide?
•Cost reduction
•Sales/revenue growth
•Speed increase
•Risk reduction
•Quality improvement
•Customer satisfaction
Strategic alignment Business value

19. Business Relevance Drives Sustainability of AI Projects
•Does your project fit into your company‘s
strategic objectives / vision / mission?
•How does your project contribute?
•E.g.enabling CO2 reduction in operations
•E.g.addressing new customers
•Which core business value does your
project aim to provide?
•Cost reduction
•Sales/revenue growth
•Speed increase
•Risk reduction
•Quality improvement
•Customer satisfaction
Strategic alignment Business value
→Make sure you evaluate your project in strategic and business relevance!

20. Transparency & Business Metrics for Success
*Sources: Google Cloud, Acacia
Typical types of AI business metrics*:
Efficiency metrics Throughput, resource utilization, amount of human intervention, ..
Performance metrics Availability, failure rates, response times, …
Financial metrics ROI, cost reduction, revenue increase, …
Satisfaction metrics NPS, user ratings, customer surveys, …
Engagement metrics Adoption rate, frequency, retention rates, …
Accuracy metrics Share of correct classifications, correct readings, …

21. Transparency & Business Metrics for Success
•AI projects are volatile in their nature
•Monitoring & sharing status is of utmost importance
•But: Most business metrics are lagging in their nature!
→Make your stakeholders aware of this early on
→Be sure to also report classical project metrics
*Sources: Google Cloud, Acacia
Typical types of AI business metrics*:
Efficiency metrics Throughput, resource utilization, amount of human intervention, ..
Performance metrics Availability, failure rates, response times, …
Financial metrics ROI, cost reduction, revenue increase, …
Satisfaction metrics NPS, user ratings, customer surveys, …
Engagement metrics Adoption rate, frequency, retention rates, …
Accuracy metrics Share of correct classifications, correct readings, …

22. Data Management is Well Worth the Effort
•Data management is still heavily underrated
•Initial investment is required –business value is huge but hard to measure
•Data engineers are key but hard to find

23. Data Management is Well Worth the Effort
•Data management is still heavily underrated
•Initial investment is required –business value is huge but hard to measure
•Data engineers are key but hard to find
5 things you miss out on if you do not invest in data management today:
1.High quality data
2.Reliable AI models
3.Informed decision making
4.Regulatory compliance
5.Scalability options
Summing up: You lose competitive advantage!

24. Data Management is Well Worth the Effort
•Data management is still heavily underrated
•Initial investment is required –business value is huge but hard to measure
•Data engineers are key but hard to find
5 things you miss out on if you do not invest in data management today:
1.High quality data
2.Reliable AI models
3.Informed decision making
4.Regulatory compliance
5.Scalability options
Summing up: You lose competitive advantage!
→Make sure you have a data strategy and performant data platform!

25. Engaging and Motivating Stakeholders
Management
Development Domain experts

26. Engaging and Motivating Stakeholders
Management
Development Domain experts
•Have regular updates, ideally
tied to business metrics
•Share risks proactively
•Provide support in AI literacy
•Include in user testing

27. Engaging and Motivating Stakeholders
Management
Development Domain experts
•Have regular updates, ideally
tied to business metrics
•Share risks proactively
•Provide support in AI literacy
•Include in user testing
•Start with interviews
•Listen to understand
•Ensure human-centric design
•Get feedback regularly
•Take time to train/explain

28. Engaging and Motivating Stakeholders
Management
Development Domain experts
•Have regular updates, ideally
tied to business metrics
•Share risks proactively
•Provide support in AI literacy
•Include in user testing
•Set clear objectives
•Challenging yet realistic work
•Freedom to explore
•Access to required resources
•Collaboration and autonomy
•Start with interviews
•Listen to understand
•Ensure human-centric design
•Get feedback regularly
•Take time to train/explain

29. 1
2
3
4
5
6
Monitoring to Improve Continuously
Validate
Build
Research/
Learn
Measure
Operate
Deploy

30. 1
2
3
4
5
6
Monitoring to Improve Continuously

31. 1
2
3
4
5
6
Monitoring to Improve Continuously
•Make sure you have good feedback
mechanisms in place –close to the actual
feature / solution
•Regularly talk to users for direct feedback
•Monitor use of your solution as proxy for
quality (adoption, intensity, retention, …)
User view

32. 1
2
3
4
5
6
Monitoring to Improve Continuously
•Make sure you have good feedback
mechanisms in place –close to the actual
feature / solution
•Regularly talk to users for direct feedback
•Monitor use of your solution as proxy for
quality (adoption, intensity, retention, …)
User view
•Monitor your model performance
according to set metrics and w.r.t. drifts
•Keep track of technical performance, e.g.
response time and availability
•Perform business performance reviews
(in a meaningful regularity)
Performance view

33. 1
2
3
4
5
6
Recognizing Alarm Signals
Solution
without
problem
Early results are
really bad
Intended users
are not
interested
Unrealistic
timeline
Data team is
decoupled from
domain experts Only access to
data dump
Unrealistic
targets set
Expectation that
MVP is the best

34. 1
2
3
4
5
6
Recognizing Alarm Signals
Solution
without
problem
Early results are
really bad
Intended users
are not
interested
Data team is
decoupled from
domain experts Only access to
data dump
Expectation that
MVP is the best
Unrealistic
timeline
Unrealistic
targets set

35. 01
02
03
01
02
03
Recap of 3+3 Key Strategies
Choose a problem worth solving
Double-check it is a problem for users
Ensure business relevance & set goals
Be proactive in sharing progress & risks
Engage all relevant stakeholders
Get feedback & improve continuously
PROJECT START

36. DSC DACH | Vienna | Sept. 12, 2024 Dr. Johannes Schauer | [email protected]
Let‘s create
persistent AI
solutions!