Automatyzacja złożonych chatbotów w Amazon Bedrock

Amazon Bedrock to usługa w AWS, która ułatwia i automatyzuje tworzenie złożonych chatbotów, wykorzystujących duże modele językowe. Oprócz wiedzy, którą model posiada, można w prosty sposób wyposażyć bota w naszą własną, prywatną wiedzę oraz poprosić go o wywołanie zewnętrznych endpointów, w razie gdyby ta wiedza nie wystarczała. W trakcie prelekcji wyjaśnię i pokażę jak łatwe jest przygotowywanie takich rozwiązań w Amazon Bedrock.

1. Automatyzacja złożonych chatbotów w Amazon Bedrock Krzysztof Kąkol Chief of Data Engineering @ Xebia Poland

2. Krzysztof Kąkol Chief of Data Engineering and Solutions Architect in Xebia Poland AWS Community Builder & AWS Ambassador https://www.linkedin.com/in/krzysztofkakol/ Other stuff: Classical and jazz pianist PhD in AI-driven sound processing

3. Access a range of leading FMs through a single API Extend the power of LLMs with your data using retrieval augmented generation (RAG) Privately customize FMs with your own data Amazon Bedrock Build agents that execute complex business tasks by dynamically invoking APIs Get the best price performance without managing infrastructure Enable data security and compliance

4. Layer 1 Content generation

5. Content generation

6. Content generation boto3_bedrock = boto3.client(' bedrock-runtime ') response = boto3_bedrock. invoke_model ( body = '{" inputText ": " Create short email from John Smith to SuperHotels requesting for special offers of hotels in San Francisco. "}', modelId = ' amazon.titan-tg1-large ' ) response_body = json.loads ( response.get ('body'). read ()) print ( response_body.get (' results ')[0]. get (' outputText ')) Dear SuperHotels , I hope this email finds you well . I am reaching out to inquire about any special offers or discounts available at your hotels in San Francisco. As a frequent traveler , I value quality accommodations and great value . I would be delighted to learn about any current promotions or packages that you have to offer . Thank you for your attention to this matter . I look forward to hearing back from you soon . Best regards , John Smith

7. Layer 2 Custom knowledge

8. Retrieval Augmented Generation Hallucination

9. Retrieval Augmented Generation CONTEXT

10. Retrieval Augmented Generation

11. Demo Knowledge Base (RAG) in Bedrock

12. Retrieval Augmented Generation

13. Retrieval Augmented Generation client = boto3.client(' bedrock -agent- runtime ’) response = client. retrieve_and_generate ( input ={ ' text ’:” Who is Andy Dufresne ? ”, }, retrieveAndGenerateConfiguration ={ ' type ': 'KNOWLEDGE_BASE', ' knowledgeBaseConfiguration ': { ' knowledgeBaseId ': ’ABC12345', ' modelArn ': 'arn:aws:bedrock:us-east-1:: foundation -model/anthropic.claude-v2' } }, ) " output ": { " text ": ”...”}, " citations ": [{ " generatedResponsePart ": { " textResponsePart ": { " text ": ”...", " span ": { "start": 0, "end": 198 } } }, " retrievedReferences ": [ { " content ": { " text ": ”..." }, " location ": { " type ": "S3", "s3Location": { " uri ": "s3://test- bedrock - kb-texts / shawshank_redemption.pdf " } } ...

14. Layer 3 Automation ( agents )

15. Agents

16. Demo Agents in Bedrock

17. Travel Agency agent

18. Agents

19. Agents client = boto3.client(' bedrock -agent- runtime ’) response = client.invoke_agent ( agentAliasId ='AJJCWRSIB2', agentId ='3LZHTSX13Q', inputText =' Can you show the offers list?', sessionId ='abc123’ ) completion = "" for event in response.get (" completion "): chunk = event[" chunk "] completion = completion + chunk [" bytes "]. decode () Here are the available destinations / offers : - Egypt , Hurghada - Italy , Venice - Mexico, Cancun

20. Many users -> your costs might explode Prompt injection – validate prompts or validate responses Good system prompt is critical – write guardrails and test them Chunk-level permissions are extremely difficult to handle Chunking is the most important work to do Sometimes custom orchestration is the only choice Be aware !

21. Amazon Bedrock gives access to plenty of different foundation models You can easily orchestrate potentially difficult tasks ( like RAGs or agents ), all the heavy lifting is done by Bedrock Bedrock enables high- level API which simplifies implementing even sophisticated LLM- related scenarios Summary

22. Questions ? https://www.linkedin.com/in/krzysztofkakol/

23. [email protected]