Gen AI: tech du-jour or the next big thing?

Is generative AI a fundamental improvement in our day-by-day? Or is it just the cool tech trend which shall pass? Are generative AI assistants useful? Can they make us more productive? Can we trust them? Are developers going to be replaced? In this session Massimo will share his perspective on the new opportunities and new challenges that generative AI is bringing to the table.

1. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Gen AI: tech du-jour or the next big thing?
Massimo Re Ferrè
Senior Principal Product Manager
AWS

2. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.2
Disclaimer: Gen AI is much bigger than this content
•This talk covers Gen AI from the (philosophical) angle of my day-by-day (NGDE)
•I can’t speak to why Gen AI is making Hollywood mad

3. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.3
Mainframes
Technology wave #1Technology wave #2
Personal Computers
Functions
Technology wave #3
Technology delivery model
Data center
Cloud
The IT world (through my lenses, over 30 years)
Servers
VMs
Containers

4. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.4
My aha moment for Gen AI

5. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.5
submit
How does Gen AI work (in simple terms)?

6. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.6
How can I use this knowledge and reason about it to create a new asset?
An asset being a piece of code, a whole program, a blog, an architecture, a troubleshooting
workflow, a db query and more outside of the IT realm (a poem, a picture, a receipt …)
Source of unstructured knowledge
Why is Gen AI a big deal (in my opinion)?

7. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
7You
Read and
memorize it all
(LOL – yeah sure) LLM
Train on it
(doable)
(1)
(2)
(3a)
Natural language
conversation
(3b)
Source of unstructured knowledge
asset
asset
Search engines (possibly
not relevant and still hard
- you are the integrator
and generator of a new
asset – text or code)
The big deal
Why is Gen AI a big deal (in my opinion)?

8. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Generative AI applicability in your stack
Builder
User
The builder writes code
The user interacts with the
application using a query language
The code is a bunch of functions

9. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Generative AI applicability in your stack
User
https://stackoverflow.com/questions/16864564/complex-sql-select-statement
The user interacts with the
application in English
Builder
The builder writes code
Gen AI @ Consumption Time
The code is a bunch of functions

10. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Generative AI applicability in your stack
“Create a function that does string comparisons”
Builder
The builder writes in English a
description of the code they need
Gen AI @ Development Time
User
https://stackoverflow.com/questions/16864564/complex-sql-select-statement
The user interacts with the
application in English
Gen AI @ Consumption Time
The code is a bunch of functions

11. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Generative AI applicability in your stack
https://www.haihai.ai/programming-with-llm/
Gen AI @ Run Time
“Create a function that does string comparisons”
Builder
The builder writes in English a
description of the code they need
Gen AI @ Development Time
User
https://stackoverflow.com/questions/16864564/complex-sql-select-statement
The user interacts with the
application in English
Gen AI @ Consumption Time
The code is a set of English statements
Let’s talk about
“Gen AI
assistants”

12. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Where are these assistants going?
How to measure productivity?
Can you trust them?
Are they going to replace us? Gen AI assistant(s) FAQs
12

13. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Where are these assistants going?
How to measure productivity?
Can you trust them?
Are they going to replace us? Gen AI assistant(s) FAQs
13

14. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.14
The progression of assistants - inline coding

15. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.15
The progression of assistants – chat

16. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.16
The progression of assistants – agents
(1) Prompt
(2) Plan(3) Code

17. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.17
From “english-to-code” to “code-to-english”
Rewriting a README file
Documenting functions in the code
https://it20.info/2024/3/using-the-amazon-q-feature-development-capability-to-produce-documentation/

18. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Where are these assistants going?
How to measure productivity?
Can you trust them?
Are they going to replace us? Gen AI assistant(s) FAQs
18

19. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.19
Acceptance rate?This is useful because it doesn’t require explicit feedback (but is it a good proxy?)

20. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.20
Thumbs up / thumbs down?This is useful because it is a good proxy (but requires explicit feedback)

21. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.21
SurveysThis is useful because it is a good proxy (but requires time and it’s more work for eveyone)

22. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.22
Tracking “time to job done”This is one of the best dimension to track (in my opinion), but…

23. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
From an AWS colleague:
“… they [ customer ] see developers wanting to use this tool [ Q ] to complete their existing tasks faster and get off work early”
23
All we know for sure is…
https://twitter.com/astasiamyers/status/1771344440617246774?s=43
Use a Gen AI assitant if you want to go home early!

24. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Where are these assistants going?
How to measure productivity?
Can you trust them?
Are they going to replace us? Gen AI assistant(s) FAQs
24

25. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
2024 (the emergence of Generative AI)
25
“How can you trust a Gen AI assistant? How can you make sure it’s not hallucinating?
You’d need to verify every answer and every artifact!”

26. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Throw back to 2001 (the emergence of VMs)
26
“How can you trust a Gen AI assistant? How can you make sure it’s not hallucinating?
You’d need to verify every answer and every artifact!”
VM the bug is not in the hypervisor
reproduce your bug on a physical server

27. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.27
•Can you trust…
-Consultants?
-Solutions Architects?
-TAMs?
-Me? (hint: you shouldn’t)
•Can you trust a pipeline? A test?
•At the end of the day, it is about controls and risk mitigation
•Can you absorb a certain amount of risk for the value that is being returned?
•What are the controls you have in place to mitigate the risk?
Do you trust everything and everyone?

28. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.28
“Everything fails all the time”
(why Gen AI shouldn’t?)

29. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.29
•Something I hear regularly: “this is not (entirely) correct”
•A better mental model (IMO): ”did it help me saving time?”
A problem of mindset?

30. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Where are these assistants going?
How to measure productivity?
Can you trust them?
Are they going to replace us? Gen AI assistant(s) FAQs
30

31. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The role of Gen AI assistants
31
https://x.com/AuthorJMac/status/1773679197631701238
“I want AI to do my laundry and
dishes so that I can do art and
writing, not for AI to do my art
and writing so that I can do my
laundry and dishes.”

32. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.32
IaC Progress over time
Cloudformation template
What if this is just yet another abstraction?
CDK program
Natural language interface Aspirationally

33. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.33
Developer
Gen AI AssistantBoost zoneLearning zoneDanger zone
The longer this line the riskier it becomes
The longer this line the less useful it becomes
You are out of controlYou are in control
Limited value zone
High returnLow return
A tentative framework for Gen AI assistant adoption (today)
Low task
complexity
High task
complexity
Low level of
knowledge
High level of
knowledge
You are here

34. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.34
•Keep an open mind
•Proving these things wrong is extremely easy
•But extracting value from them can be very useful
•You can rent a car and drive it into a wall or go to the beach with it
•Where do you want to go?
Conclusions

35. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!
Massimo Re Ferrè
Mail: [email protected]
Twitter: @mreferre
35