AWS User Group Torino 2024 #3 - 18/06/2024

Presentation of AWS User Group Torino, n.3 - 2024.

1. UPDATE THIS PRESENTATION HEADER IN SLIDE MASTER
AWS UG Torino
Meetup 2024 #3
18/06/2024

2. UPDATE THIS PRESENTATION HEADER IN SLIDE MASTER
AWS USER GROUP TORINO
ENTRA NEL GRUPPO WHATSAPP!

3. UPDATE THIS PRESENTATION HEADER IN SLIDE MASTER
© 2024, Amazon Web Servic es, Inc. or its affiliates. All rights res erved. Amazon C onfiden tial an d Tradem ark.
Calendario2024:
AWS UG TORINO MEETUP:
•19/02/2024✓
•16/04/2024 ✓
•18/06/2024 oggi
•02/10/2024
•19/11/2024

4. UPDATE THIS PRESENTATION HEADER IN SLIDE MASTER
Rinfresco&
Networking
Agenda di oggi
AWS UG Torino Meetup2024 #3
Carlo Peluso
AI Agentson AWS: Deep-
diveon Amazon Bedrock
Agents
DavideDe Sio
Serverlesse DevOpsper
migliorarela DevXP

5. UPDATE THIS PRESENTATION HEADER IN SLIDE MASTER
© 2024, Amazon Web Servic es, Inc. or its affiliates. All rights res erved. Amazon C onfiden tial an d Tradem ark.
DavideDe Sio
Serverlesse DevOpsper
migliorarela DevXP
AWS UG TORINO MEETUP2024 #3

6. SuperpowerREST API DX
with Serverlessand
DevOpsBest Practices on
AWS
Come ridurre la complessità dell’infrastruttura con
AWS e migliorare la DevExperienceattraverso pratiche
DevOps
El ev a
Sr l

7. Chi Sono
Davide De Sio
Development Tech Lead @Eleva
AWS Solutions Architect (Professional)
Senior Full Stack Developer
“The essenceof the serverlesstrend istheabsence
of the server concept duringsoftware development.”
Tomasz Janczuk, VP atAuth0
ServerlessEnthusiast
AWS UG Lead -Pavia

8. Use Case 1
Mobile Backend

9. Use Case 2
Single Page Application

10. Perché Serverless?
Cosa mi aspetto?
Voglio che l’architettura
scali in automatico (auto-scaling)
Voglio che gli sviluppatori
si concentrino nello scrivere
la business logic
Non voglio pagare
quando non utilizzo risorse (idle)
Voglio che il team di sviluppo si concentri su quello che conta
davvero, potendo far affidamento su una piattaforma di
sviluppo che massimizzi la Developer Experience e la renda
semplice, piacevole, ripetibile e manutenibile.

12. Infrastructureascode
Infrastruttura dichiarata come codice
Documentationascode
Documentazione dichiarata come codice
AutomatedTesting
Test automatici basati sulla documentazione
CI/CD
Pipelines di distribuzione automatizzate
DR
Procedure di DisasterRecovery
Versioning
Versionamentodel singolo micro-servizio
API Architecture
Pratiche DevOps
Security By Design
Best practices di sicurezza
Monitoring
Monitoraggio continuo

13. Provisioning (senza overhead)
Esposizione tramite Amazon API Gateway, Amazon Cloudfront
e Amazon Route53
Esecuzione su funzioni serverlesstramite AWS Lambda
CI/CD
Release tramite strumenti di infrastractureascode
(AWS Cloudformation/SAM/CDK/ServerlessFramework)
Esecuzione del rilascio e dei test su pipeline per CI/CD
(AWS Codepipelinee AWS Codebuild) a partire da commitsu
branchfinalizzati al versionamentoe al rilascio di repo Git
API Architecture
Key points

14. API Architecture
Key points
Doc first / Doc generation
Documentazione come codice secondo standard Open API V3 (via Serverless
Framework)
Generazione della documentazione su pagine statiche (via Redocly) storicizzate
su Amazon S3 e integrazione nella pipeline di distribuzione
Automated Testing
Implementazione di test che validano le API in base alle specifiche Open API V3
generate
Security first e monitoring
Messa in sicurezza con Managed Services
AWS IAM / VPC / WAF / Guard Duty
Monitoraggio tramite AWS X-Ray e Cloudwatch
(dashboard e allarmi)

15. Show me
the (infra) code!
CloudFormation per estensioni (security -
cognito)
Documentazione integrata
(doc-as-code)
Serverless files
Infrastructure as Code
Plugin per monitoraggio
(slic-watch)
Plugin per local environment
(serverless-offline)
CloudFormation per architettura
a supporto e CI/CD come codice
(buildspec) su CodePipeline e
CodeBuild

16. Talk ischeap
Show me the code

17. ServerlessSkeletons
Open Source
Node.js PHP Python

18. Textract: ExampleUse Case
EU-DrivingLicencesOCR

21. Bonus
«Non tutte le ciambelle vengono col buco»
https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-
service-and-reducing-costs-by-90

22. Bonus
«Don't get locked up into avoiding lock-in»
https://martinfowler.com/articles/oss-lockin.html
•Vendor Lock-In
•Product Lock-in
•Version lock-in
•Architecture lock-in
•Platform lock-in
•Skills lock-in
•Legal lock-in
•Mental Lock-in

23. UPDATE THIS PRESENTATION HEADER IN SLIDE MASTER
© 2024, Amazon Web Servic es, Inc. or its affiliates. All rights res erved. Amazon C onfiden tial an d Tradem ark.
Carlo Peluso
AI Agents on AWS: Deep-
dive on Amazon Bedrock
Agents
AWS UG TORINO MEETUP 2024 #3

24. AI AGENTS ON
AMAZON BEDROCK
Carlo Peluso | Solutions Architect @ StormReply

25. INTRODUCTION
ABOUT ME
2018 – Started working as Backend Engineer
2020 – Bachelor’s Degree in Computer Science Engineering
2022 – Master’s Degree in Data Science and Engineering
2022 – Started working as Solutions Architect @ Storm Reply
2022 – Q1 Publication on Transformers for Healthcare

26. INTRODUCTION
TODAY’S TOPIC: AI AGENTS
WHAT
are AI agents?
WHY
AI agents
are useful?
HOW
AI agents
can be
implemented?

27. ARTIFICIAL INTELLIGENCE
FROM SHALLOW ARCHITECTURES TO LARGE LANGUAGE MODELS
Increasing number of hidden layers…
NEURAL NETWORKS & CNNs
AlexNet, ResNet, …
TRANSFORMERS & LLMs
BERT, Claude, Mistral, GPT-3, …
SHALLOW ARCHITECTURES
Support Vector Machines, Decision Trees, …

28. DEEP NEURALNETWORKS
TRAINING COMPLEXITIES
HIGH NUMBER OF
HIDDEN LAYERS
LOTS OF DATA AND
HIGHLY PERFORMANT
COMPUTATIONAL
RESOURCES
BILLIONS OF
TRAINING
PARAMETERS
IMPLY REQUIRE

29. CAN WE
EXPLOIT PRE-TRAINED AI MODELS
FOR DATA SPECIFIC FROM OUR DOMAIN?

30. EXPLOIT PRE-TRAINED AI MODELS
FINETUNING
PRE-TRAINED MODELS CAN BE FINETUNED
(i.e., SMOOTHLY RETRAINED)
ON DATA FROM AN UNKNOWN DOMAIN

31. STILL…
WITH GENERATIVE AI MODELS
WE CAN USE A MORE
STRAIGHTFORWARD APPROACH!

32. EXPLOIT PRE-TRAINED AI MODELS
IN-CONTEXT LEARNING
COLLECT UNKNOWN DOMAIN DATA
WITHIN A KNOWLEDGE BASE
INPUT CONTEXT INFORMATION
FROM THE KNOWLEDGE BASE
TO THE GENERATIVE AI MODEL

33. EXPLOIT PRE-TRAINED AI MODELS
IN-CONTEXT LEARNING
THE GENERATIVE AI MODEL CAN
COMPENSATE THE
MISSING DOMAIN KNOWLEDGE
BY EXPLOITING
IN-CONTEXT INFORMATION

34. Qual è
la capitale
della Grecia?
IN-CONTEXT LEARNING
EXAMPLE
La capitale della Francia è Parigi.
La capitale dell’Italia è Roma.
La capitale della Spagna è Madrid.
La capitale della Grecia è Atene.
La capitale
della Grecia
è Atene.
Semantic search
Provide in-context information to the LLM
1
2
3
4
1
2
3
4
The user asks a question.
The question is used to retrieve from the knowledge base
the documents semantically closer to the question.
The documents retrieved are fed to the LLM, alongside the original
question and a system prompt (that describes how the LLM should
respond).
The LLM, based on the context provided and the system prompt,
answers the question.

35. SEAMLESSLY BUILD AND MAINTAIN
VECTOR DATABASES USING
DOCUMENTS FROM YOUR DOMAIN
EASILY INTEGRATE LEADING LLMs ,
DELEGATING INFRASTRUCTURE
MANAGEMENT
GENERATIVE AI ON AWS
AMAZON BEDROCK
FOUNDATION MODELSKNOWLEDGE BASES

36. AMAZON BEDROCK
KNOWLEDGE BASES
AMAZON BEDROCK
KNOWLEDGE BASES
Textual documents, PDFs,
CSVs, JSON, …
AMAZON S3
AMAZON
RDS
AMAZON
OPENSEARCH
Text to vector representations

37. IN-CONTEXT LEARNING
EXAMPLE USING AMAZON BEDROCK
La capitale della Francia è Parigi.
La capitale dell’Italia è Roma.
La capitale della Spagna è Madrid.
La capitale della Grecia è Atene.
AMAZON BEDROCK
KNOWLEDGE BASES
AMAZON BEDROCK
FOUNDATION MODELSQual è
la capitale
della Grecia?
1
1
The user asks a question.

38. IN-CONTEXT LEARNING
EXAMPLE USING AMAZON BEDROCK
La capitale della Francia è Parigi.
La capitale dell’Italia è Roma.
La capitale della Spagna è Madrid.
La capitale della Grecia è Atene.
AMAZON BEDROCK
KNOWLEDGE BASES
AMAZON BEDROCK
FOUNDATION MODELSQual è
la capitale
della Grecia?
API call: knowledgeBase.retrieve(question)
1
2
1
2
The user asks a question.
The question is used to retrieve from the knowledge base
the documents semantically closer to the question.

39. IN-CONTEXT LEARNING
EXAMPLE USING AMAZON BEDROCK
La capitale della Francia è Parigi.
La capitale dell’Italia è Roma.
La capitale della Spagna è Madrid.
La capitale della Grecia è Atene.
AMAZON BEDROCK
KNOWLEDGE BASES
AMAZON BEDROCK
FOUNDATION MODELSQual è
la capitale
della Grecia?
API call: knowledgeBase.retrieve(question)
API call: bedrock.invoke_model(context)
1
2
3
1
2
3
The user asks a question.
The question is used to retrieve from the knowledge base
the documents semantically closer to the question.
The documents retrieved are fed to the LLM, alongside the original
question and a system prompt (that describes how the LLM should
respond).

40. IN-CONTEXT LEARNING
EXAMPLE USING AMAZON BEDROCK
La capitale della Francia è Parigi.
La capitale dell’Italia è Roma.
La capitale della Spagna è Madrid.
La capitale della Grecia è Atene.
AMAZON BEDROCK
KNOWLEDGE BASES
AMAZON BEDROCK
FOUNDATION MODELSQual è
la capitale
della Grecia?
La capitale
della Grecia
è Atene.
API call: knowledgeBase.retrieve(question)
API call: bedrock.invoke_model(context)
1
2
3
4
1
2
3
4
The user asks a question.
The question is used to retrieve from the knowledge base
the documents semantically closer to the question.
The documents retrieved are fed to the LLM, alongside the original
question and a system prompt (that describes how the LLM should
respond).
The LLM, based on the context provided and the system prompt,
answers the question.

41. STILL,
RETRIEVE AND GENERATE
APPLICATIONS ARE SOMEHOW
“STATIC”

42. FOUNDATION MODELS
ORCHESTRATE TASKS BY
EXPLOTING A KNOWLEDGE BASE
AND
DYNAMICALLY INVOKING APIs
AI AGENTS
AI AGENTS ON AMAZON BEDROCK

43. ACTION GROUPS
DEFINE ACTIONS THAT THE AGENT CAN PERFORM
AI AGENTS ON AMAZON BEDROCK
ACTION GROUPS

44. LAMBDA FUNCTIONS
DEFINE HOW THE AGENT HANDLES
THE PARAMETERS IT RECEIVES
OPENAPI SCHEMAS
DEFINE THE PARAMETERS THE
AGENT MUST EXTRACT FOR THE
ACTION TO BE EXECUTED
AI AGENTS ON AMAZON BEDROCK
ACTION GROUPS

45. AI AGENTS
WORKFLOW
AI AGENT ISINPUTVALID?
PREPROCESSING
PROMPT
1
1The agent verifies that the user input is not malicious.

46. AI AGENTS
WORKFLOW
AI AGENT ISINPUTVALID?
SEARCH WITHIN THE
KNOWLEDGE BASE
CALL AN API
ORCHESTRATION
PROMPT
WHICH ACTION
SHOULD I TAKE?
PREPROCESSING
PROMPT
1 2
1
2
The agent verifies that the user input is not malicious.
The agent creates an orchestration prompt using:
•User’s input
•Conversation history
•Information about knowledge bases and available APIs
•Instructions provided by the system developer

47. AI AGENTS
WORKFLOW
AI AGENT ISINPUTVALID?
PERFORM ACTION
SEARCH WITHIN THE
KNOWLEDGE BASE
CALL AN API
RESPONSE
ORCHESTRATION
PROMPT
WHICH ACTION
SHOULD I TAKE?
PREPROCESSING
PROMPT
1 2
3
1
2
3
The agent verifies that the user input is not malicious.
The agent creates an orchestration prompt using:
•User’s input
•Conversation history
•Information about knowledge bases and available APIs
•Instructions provided by the system developer
The Foundation Model choose which action should be taken.

48. AI AGENTS
WORKFLOW
AI AGENT ISINPUTVALID?
PERFORM ACTION
SEARCH WITHIN THE
KNOWLEDGE BASE
CALL AN API
RESPONSE
ORCHESTRATION
PROMPT
WHICH ACTION
SHOULD I TAKE?
OBSERVATION
PREPROCESSING
PROMPT
1 2
3
4
1
2
3
4
The agent verifies that the user input is not malicious.
The agent creates an orchestration prompt using:
•User’s input
•Conversation history
•Information about knowledge bases and available APIs
•Instructions provided by the system developer
The Foundation Model choose which action should be taken.
The action’s output is an observation that is used to enrich
the orchestration prompt.

49. AI AGENTS
WORKFLOW
AI AGENT ISINPUTVALID?
PERFORM ACTION
SEARCH WITHIN THE
KNOWLEDGE BASE
CALL AN API
RESPONSE
ORCHESTRATION
PROMPT
WHICH ACTION
SHOULD I TAKE?
OBSERVATION
PREPROCESSING
PROMPT
1 2
3
4
5
1
2
3
4
5
The agent verifies that the user input is not malicious.
The agent creates an orchestration prompt using:
•User’s input
•Conversation history
•Information about knowledge bases and available APIs
•Instructions provided by the system developer
The Foundation Model choose which action should be taken.
The action’s output is an observation that is used to enrich
the orchestration prompt.
Looping and refinement until the Agent has all the necessary
informations to answer.

50. TRACING
DETAIL THE STEPS ORCHESTRATED
BY THE AGENT, HELPING TO
FOLLOW THE AGENT’S REASONING
PROCESS
GUARDRAILS
IMPLEMENT SAFEGUARDS FOR
GENERATIVE AI APPLICATIONS
AI AGENTS ON AMAZON BEDROCK
GUARDRAILS & TRACING

51. ACTION GROUP
IMPLEMENT AN API FOR
RETRIEVING THE QUANTITY
OF AVAILABLE PUPPETS
KNOWLEDGE BASE
CONTAINS DESCRIPTIONS
ABOUT POKEMON PUPPETS
AI AGENTS ON AMAZON BEDROCK
USE-CASE
POKEMON PUPPETS INVENTORY ASSISTANT

52. ACTION GROUPS
ARE BASED ON
LAMBDA FUNTIONS
KNOWLEDGE BASE
IS HOSTED ON
AMAZON OPENSEARCH
FOUNDATION
MODEL
IS CLAUDE 2.1
AI AGENTS ON AMAZON BEDROCK
USE-CASE

53. UPDATE THIS PRESENTATION HEADER IN SLIDE MASTER
PROSSIMO APPUNTAMENTO
AWS UG Torino Meetup2024 #4
CALL FOR SPEAKERS
02/10/2024 –18.30
ToolboxCoworking
Via Agostinoda Montefeltro, 2, Torino
Si possonopresentarepropriprogetti, case study,
best practicee altro.