The road to pragmatic application of AI.pdf

AI, and generative AI in particular, is all the rage now. Everyone is experimenting, and many ambitious projects are underway. However, few projects have created significant business value. While most struggle to translate modern technology into revenue, "born digital" companies are launching AI-powered features with ease. We can find quantitative metrics related to innovation and productivity that differ by 100-1000 times, e.g. lead time from AI innovation idea to launch or operational costs of data flows. We call the difference in innovation and productivity between these companies and the incumbents the "data divide" or the "AI divide.

In this session Lars will explain what the leading companies do differently and why the AI divide persists. The born digital companies have succeeded enabling innovation to grow organically. It is a path that is easy to walk, gives quick return on investment, but is nevertheless rarely chosen. You’ll learn how to go down that path, and how to achieve conditions for success.

1. www.scling.com
The road to pragmatic
application of AI

Lars Albertsson, Founder, Scling
Nordic Software Summit
2024-08-20
1

2. www.scling.com
Hypes disrupt some fields
2
Hype Enabled / boosted Accessible to anyone Disrupted
Personal computer Corporate digital tools Yes Office work, design
Internet Machine communication Finance, telecommunications
Expert systems Rule-based AI, anomaly detection, classification Manufacturing, operations, fraud prevention, games, medicine
Web Broadcast human communication Yes Media, marketing, commerce
Semantic web Machine communication?
Social web Peer human communication Yes Media, marketing, culture, politics
IoT Small machine communication Manufacturing, energy, security
Smartphones Personal digital tools Yes Point-of-sales, field work, travel, eating
Big data Basic AI, personalisation, classification, patterns Media, marketing, commerce, operations, research
Deep learning Object recognition, robot control, data extraction Control systems, vehicles, medicine, translation
Blockchain Cryptocurrencies Yes Criminal finance
Generative AI Text & media generation Yes Media, games, product user interfaces. Fraud, malicious politics.
Impact most
businesses

3. www.scling.com
Big data adoption
3
●2003-2007: Only Google

●2007-2014: Hadoop era. Highly technical companies
succeed and disrupt.

●2015-2019: Enterprise adoption. Big data gone from
Gartner hype cycle. “New normal”

●2019: Many enterprises in production, but big data and
machine learning ROI still confined to high-tech.

4. www.scling.com
Enabling innovation
4
"The actual work that went into
Discover Weekly was very little,
because we're reusing things we
already had."
https://youtu.be/A259Yo8hBRs
https://youtu.be/ZcmJxli8WS8
https://musically.com/2018/08/08/daniel-ek-would-have-killed-discover-weekly-before-launch/
"Discover Weekly wasn't a great
strategic plan and 100 engineers.
It was 3 engineers that decided to
build something."
"I would have killed it. All of a sudden,
they shipped it. It’s one of the most
loved product features that we have."
-Daniel Ek, CEO

5. www.scling.com
Data not acted upon - value left on table
5
The car has sensors for precipitation,
temperature, and window state. I would
have liked to receive a mobile app
warning when all windows were down
during a snowy night.

6. www.scling.com
Data not acted upon - value left on table
6
Changing to winter tyres at Bilia. When I arrived,
the staff could see that mechanics were currently
not fully booked, and offered me a discounted
front wheel adjustment, which I accepted.

Data value taken from table.
Our Volvo incorrectly activates the automatic safety
brake again. This is a known problem since years
back. I have no efficient channel to report time and
circumstances for this occasion, allowing Volvo to
get more data on the problem, and me to feel
confident that the issue is worked on.
After a repair at Bilia of the rear view mirror, the
car does not properly detect cars in front. Camera
near mirror is suspected. Bilia cannot obtain
camera measurements or car detection statistics
from Volvo, so a cycle of trial and error repairs
follows, taking me to the mechanic multiple times.
The car has sensors for precipitation,
temperature, and window state. I would
have liked to receive a mobile app
warning when all windows were down
during a snowy night.
I make the effort to submit a detailed bug report
regarding Volvo's known over-aggressive
automatic safety brake problem, in order to
provide more data for debugging. I receive no
feedback from the process, and lose interest.
I send a design suggestion to Volvo with pics of a
hacked solution for a luggage problem. A simple
hook could solve it. I do not receive "We have
assigned your suggestion id X, and will get back if
it is implemented." I will not send another, and feel
less connected to the brand.
Wife unlocks Volvo. Driver seat switches to her
position. I take the driver's seat, pull it back, and
change to my profile. Car switches seat back to
her position, and I have to pull it back again.
When seat heating was automatically activated,
"Increase seat heating" audio command turns it
off. I make additional commands to get to the
desired state. Measurements could identify
repeated audio or UX commands.
The Volvo phone app warns about many things:
interrupted charging, car parked but not being charged,
etc. But not about the roof and windows left open in the
rain. Rear view mirror electronics now need a repair.
I change the audio balance in the
Volvo with six screen clicks.
Measurements could indicate that
long click sequences at high speed
indicates poor user interface.
Wife has driven our Volvo. Mirrors and seats
automatically move to my position, except for the
right mirror, which I need to correct again, as I
have for years. Volvo could have measured and
detected irregular changes of settings, and found
this bug.
The car keeps informing me
that there is a software
upgrade, but no matter what I
do, nothing gets upgraded.
Map updates fail to auto
install. Support cannot obtain
diagnose information. Manual
installation attempts fail
without error messages.
Navigation suggests a road that
eventually becomes so narrow and close
to the beach that it is dangerous to drive
a 2 ton SUV. With speed statistics from
other vehicles, a better route could have
been suggested.
Neighbour jumps in their Volvo, talks to husband
through window, and drives off. On arrival she
discovers that she has no car key - husband's key
was close enough to start car. An early warning
that key is no longer in proximity would have been
valuable.

7. www.scling.com
1.Log in, search for product X
○Popular items first
2.Find X in product catalog
○Take me to shop
3.Put in cart, delivery?
○I am logged in
4....


An e-shopping tale
7













Full story: “Avoid artificial stupidity” @ mapflat.com/blog
1.Log in, search for product X
○X + 100s of accessories,
random order
2.Find X in product catalog
○No link to web shop
3.Put in cart, delivery?
○Ask for address
○Ask for customer club number
4.…

8. www.scling.com
Data-centric innovation
●Need data from teams
○willing?
○backlog?
○collected?
○useful?
○quality?
○extraction?
○data governance?
○history?



8

9. www.scling.com
Big data - a collaboration paradigm
9
Data platform
Data lake
Data
democratised
Value stream-aligned
teams - few handoffs

10. www.scling.com
From craft to process
10

11. www.scling.com
From craft to process
11
Multiple time windows
Assess ingress data quality
Repair broken data from
complementary source
Forecast based on history,
multiple parameter settings
Assess outcome data quality
Assess forecast success,
adapt parameters

12. www.scling.com
IT craft to factory
12



Security



Waterfall



Application
delivery



Traditional
operations



Traditional
QA



Infrastructure



DevSecOps



Agile



Containers



DevOps



CI/CD



Infrastructure
as code

13. www.scling.com



Security



Waterfall
Data factories
13



Application
delivery



Traditional
operations



DevSecOps



Traditional
QA



Infrastructure




DB-oriented
architecture



Agile



Containers



DevOps



CI/CD



Infrastructure
as code




Data factories,
data pipelines,
DataOps

14. www.scling.com
Naive machine learning
14

15. www.scling.com
Sustainable production machine learning
15
Multiple models,
parameters, features
Assess ingress data quality
Repair broken data from
complementary source
Choose model and parameters based
on performance and input data
Benchmark models
Try multiple models,
measure, A/B test

16. www.scling.com
AI & data priorities
16
Application













AI / data-powered
feature
Probe
Data platform
Probe
Probe
Data sources
Fast data flow development
iterations and feedback loops
Fast integration iterations
Minimal operational overhead
Ways of working & processes
Data platform & tech stack
ImportanceSupporting

17. www.scling.com
Capability KPIs
DORA research / State of DevOps report:

●Lead time for changes
●Deployment frequency
●Change failure rate
●Time to restore service

Small elite
~1000x span
17
Observed differences in data organisations:

●Lead time from idea to production
●Time to mend / change pipeline
●Number of pipelines / developer
●Number of datasets / day / developer

Small elite
100 - 10000x span

18. www.scling.com
Efficiency gap, data cost & value
●Data processing produces datasets
○Each dataset has business value

●Proxy value/cost metric: datasets / day
○S-M traditional: < 10
○Bank, telecom, media: 100-1000
18
2014: 6500 datasets / day
2016: 20000 datasets / day
2018: 100000+ datasets / day,
25% of staff use BigQuery
2021: 500B events collected / day
2016: 1600 000 000
datasets / day
Disruptive value of data, machine learning
Financial, reporting
Insights, data-fed features
effort
value

19. www.scling.com
Myth:
●We are all doing quite ok
●2-10x leader-to-rear span
The great capability divide
19
capability in X
# orgs

20. www.scling.com
Myth:
●We are all doing quite ok
●2-10x leader-to-rear span
The great capability divide
20
capability in X
# orgs
capability in X
# orgs
Reality:
●Few leaders in each area
●100-10000x leader-to-rear span

21. www.scling.com












Size = effort Credits: “Hidden Technical Debt in
Colour = code complexity Machine Learning Systems”,
Google, NIPS 2015

Machine learning product efforts
21
Configuration Data collection
Monitoring
Serving
infrastructure
Feature extraction
Process
management tools
Analysis tools
Machine
resource
management
Data
verification
ML

22. www.scling.com









Boring, valuable data-powered product
Data-driven products vs business value
22
Configuration Data collection
Monitoring
Serving
infra
Feature extraction
Process
management
tools
Analysis tools






Exciting prototype without
business value
Machine
resource
mgmt
Data
verification
ML













< Insert meme with drooling person here >

23. www.scling.com
Generative AI → more complex data engineering
23
Input
LLM
Output
Generative AI is simple? Want multi-modal?
Correct facts too?
Safety mechanisms to
avoid undesirable use?
Add relevance? →
Retrieval augmented
generation (RAG)
Input
LLM
Output
Search engine Document
corpus
Document
snippets
Nvidia: Nemo Guardrails
Langchain: Multi-modal RAG
USTC + UCLA + Google: Corrective RAG

24. www.scling.com
Data mature company product portfolio
24

25. www.scling.com
Future data engineering - "data factory engineering"
25
DW
~10 year capability gap
Enterprise big data failures
"Modern data stack" -
traditional workflows, new technology





4GL / UML phase of data engineering
Data engineering education

26. www.scling.com
10 failure / success factors - flow
26
Friction creating Flow enabling Description
Organisation by functionality

Aligned with the value stream Whether product development that requires multiple skill sets demands
engagement from multiple teams or can be done within a single team.
Human processes

Technical processes Whether there is bias towards changing the way that humans work vs the
way that machines work when facing a challenge.
Push Pull Whether work is initiated and prioritised driven by decisions to implement
capabilities or milestones, or by specific business value needs.
More things

Less things Whether technical systems and processes are seen as assets to grow and
nurture or as liabilities to remove unless they bring sufficient value.
Project Product Whether activities are seen as time-limited efforts or as team-owned
artefacts that evolve in an iterative manner.

27. www.scling.com
10 failure / success factors - governance
27
Friction creating Flow enabling Description
Denied by default Trusted by default

Whether a request needs to be sufficiently motivated before being granted.

Opaque by default Transparent by default

Whether information about systems and products is accessible on a
need-to-know basis or generally available.

Siloed by default Accessible by default

Whether data is locked up in decentralised, heterogeneous systems well
guarded by owners or available in centralised, homogeneous systems.
Risk management by rituals Risk management by golden
paths and guard rails
Whether risks (security, compliance, reliability, etc) are managed by deciding
on a single way to do things and forcing teams to stick to it, or by
decentralising responsibility but providing suggested, paved paths and
warning systems.
Direction & control

Thrust Whether more effort is spent figuring out in detail what to do and how as
opposed to improving speed forward so more things can be tested with trial
and error.

28. www.scling.com
Don't build. Grow.
28
●Every data / AI project has either
○failed
○cost a fortune

●Every leading data company has
○solved product challenges at hand
○improved process / ways of working
○had data / AI success through enabled teams

●"CorpOps" - start where you are
○Data & automation for current challenges
○Data collection grows …
○Data gradually refined and prepared …
○… while value is created
Technology
change
Application
change
Negative ROI
Positive ROI
We must
be AI first!
Solve today's
challenges with
data & automation
Sustainable
data+AI feature
development

29. www.scling.com 29
Taking the CorpOps path
●Let grassroots business value drive automation
○Seek low-hanging fruit
○E.g. things done with spreadsheets
○"What if you had 50 junior assistants?"

●Everything is an engineering challenge
○Software engineers in majority
○Product, data, QA
Remember the priorities:
1.Fast development iterations
2.Fast integration iterations
3.Minimal operational cost
4.Processes that support 1-3
5.Platform that supports 1-4
Avoid the 10 frictions:
1.Org by value stream, not functionality
2.Change technology, not humans
3.Pull, not push
4.Less, not more
5.Product, not project
6.Trust, don't deny
7.Transparent, not opaque
8.Accessible, not siloed
9.Guard rails, not rituals
10.Thrust, not control

30. www.scling.com 30
The AI success paradox
Remember the priorities:
1.Fast development iterations
2.Fast integration iterations
3.Minimal operational cost
4.Processes that support 1-3
5.Platform that supports 1-4
●Let grassroots business value drive automation
○Seek low-hanging fruit
○E.g. things done with spreadsheets
○"What if you had 50 junior assistants?"

●Everything is an engineering challenge
○Software engineers in majority
○Product, data, QA
Avoid the 10 frictions:
1.Org by value stream, not functionality
2.Change technology, not humans
3.Pull, not push
4.Less, not more
5.Product, not project
6.Trust, don't deny
7.Transparent, not opaque
8.Accessible, not siloed
9.Guard rails, not rituals
10.Thrust, not control
By focusing more on your product and less on AI,
you are more likely to succeed with AI.

31. www.scling.com
●Other disciplines?
○Shared innovation
○E.g. Foxconn, Autoliv

●Tech leaders
○Grow, don't build
○Product-driven
31
Data factory adoption
Observations:

●Industrial success requires veterans
●Not enough factory experience around
●Minimal flow of veterans to incumbents /
consultants

●Artisanal tools (Data warehouse / low-code / …)
inadequate for domain-specific AI

●Industrial tools not helpful without veterans
Belief:
We need new ways of
collaboration.










Customer
Data factory
Data platform & lake
data
domain
expertise
Value from data!
Data innovation
as leaders do it

Learning by
doing, in
collaboration

32. www.scling.com
What we learnt
32
Success under good circumstances
●Match Spotify's numbers per developer
●10-1000x client's own capability
●Data + AI innovation on par with leaders
Challenge: Data capability → innovation
●Domain-specific competence
●"Innovation building blocks" needed
○Agile, pull vs push, iterative
○Digital thinking
○Aligned, cross-functional teams
○Product focus
○Value chain alignment
Greater challenge: "How hard can it be?"
●Unawareness of data divide

33. www.scling.com
What we learnt, what we need
33
Humble, but competent clients
●Data with value potential
●Domain experts
●Innovation building blocks
●Humble regarding data divide
Other ways to slice the challenge?
Partners that bridge innovation gap
●Digitalisation in some vertical
●Interested in new business models
Success under good circumstances
●Match Spotify's numbers per developer
●10-1000x client's own capability
●Data + AI innovation on par with leaders
Challenge: Data capability → innovation
●Domain-specific competence
●"Innovation building blocks" needed
○Agile, pull vs push, iterative
○Digital thinking
○Aligned, cross-functional teams
○Product focus
○Value chain alignment
Greater challenge: "How hard can it be?"
●Unawareness of data divide