AI Case studies - Experiences and lessons learnt (How would you integrate AI in your business).pdf

This slide was presented at the IPBF FORUM
AI Peer Learning Workshop (06 August 2024).

1. AI CASE STUDY –
EXPERIENCES &
LESSON
LEARNED
How would you integrate with AI in your business?

2. ●Senior Full Stack Enterprise Java
Developer, Engineer and
Consultant, Sindi Technologies
(Pty) Ltd
○18+ years of enterprise software
development experience.
●Tech speaker / Blogger
●Devoxx4Kids South Africa (Non-
Profit Organisation) founder and
event organiser
About Me

3. Where to find me?
https://linktr.ee/theEliteGentleman
[email protected]
@EliteGentleman
www.sindi.co.za
www.devoxx4kids.org.za

4. Agenda
IBFP 2024
Case study02
Solution04
Glossary01
Problems and Concerns03
Finally05 Q&A06

5. GLOSSARY
01

6. •An AI research organization, started in December 2015.
•Mission: To develop “safe and beneficial” artificial
generative intelligence.
•First GPT was introduced in 2018
•Introduced ChatGPT, a chatbot and virtual assistant, in
November 2022
•Today, ChatGPT-4o, released in May 2024
•It powers Microsoft Copilot, Apple Intelligence
•ChatGPT is credited for starting the AI boom, catalyzing
the widespread interest of generative AI.
https://openai.com/

7. ●Use of prompts to generate various data using generative models.
●Chat Prompts
○ChatGPT by OpenAI, Gemini by Google, LLaMAby Meta AI, Mistral, Claude by
Anthropic, Qwenby Alibaba, ERNIE by Baidu
●Text-to-Image models
○DALL-E by OpenAI, Imagen by Google, Firefly by Apple, Midjourney by Midjourney,
Stable Diffusion by Stability AI, RunwayMLby Runway AI
●Text-to-Video models
○Sora by OpenAI, VideoPoetby Google, Runway by Runway AI
GenerativeAI

8. ●Using the combination of Gen AI and Retrieval-Augmented Generation (RAG)
○RAG: a technique for enhancing the accuracy and reliability of generative AI models
with factsfetched from external sources.
●Human-like conversational skills
○Fully understand customer intents, contexts and provide tailor services around their
exact need
●Enterprise knowledge.
○Pre-train your GenAIwith factualenterprise knowledge for superior query resolution,
●Adaptive learning
○Improve your GenAIwith new knowledge acquired from customer interactions to
provide better service skills.
AI Agents for your business

9. CASE STUDY
02

10. ●Problem space:
○Healthcare professionals doing consultations may have too many patient visits:
■Medical records needs to accurately captured and filed.
■Usually captured by an assistant for filing
●Problem: initial training to get up to speed with doctor speak.
●Problem: difficulty reading doctor’s handwriting.
■Time consuming: Doctor may have to go for surgeries.
●Problem: peer review of captured record may happen late.
●Problem: remembering the conversation that is in the past.
●Solution:
○Tool must:
■Transcribe audio recording
●Multi-language support (English, Afrikaans and other supported native
languages)
●Able to identify multiple speakers from audio.
■Text summarization and document generation from transcribed text
●GPTs to the rescue.
MEDICAL CONSULT TRANSCRIBING

11. PROBLEMS AND
CONCERNS
03

12. Varieties of AI companies to choose from

13. ●IBM:
○Watsonx
●Microsoft:
○Various models available on Azure AI services
■OpenAI, Mistral AI, Phi-3, Meta AI, Stability AI, Core42 (Arabic LLM), Nixtla, and
more.
●Google:
○Vertex AI, includes PaLM2
○Generative AI, includes Gemini
●Amazon Bedrock
○AWS
○Fully managed services that offer choices of high-performing foundational models
(FM) from leading AI companies
■AL 21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, Amazon
●Oracle
○Oracle Cloud Infrastructure (OCI) Generative AI
Top tech cloud providers –For enterprise
customers.

14. ●China’s investment into AI
○Alibaba:
■Alibaba
■Qwen(own Gen AI)
○Baidu:
■Baidu AI Cloud
■ERNIE (own Gen AI)
Top tech cloud providers –For enterprise
customers.

15. ●Audio to text services are few:
○Speaker diarizationsupport (identify multiple speakers from audio conversations).
○Multiple languages support (some don’t support local languages).
●Integration to their AI services:
○Closed and proprietary, free and public (REST API), Cloud services
●Accuracy of the responses
○Is text summarization accurate enough and does it understand context accordingly?
○Do we have to refine the chat prompt assistant script to make the response
accurate?
●Data Privacy
○Hide client personal information at all cost.
○What data info can be shared to AI services?
○Governance –US laws vs. SA laws
●Cost.
○Network Bandwidth
○Cost of using AI service (in USD).
○Credit card is required to access AI upfront.
KEY PROBLEMS -CASE STUDY

16. Most have REST APIs
INTEGRATION WITH AI

17. ●Input and Output price, per million tokens
COST

18. ●Measured by tokens per second
OUTPUT SPEED

19. ●Feeding the GenAIwith facts.
○Removing bias
○Data cleanliness
○Information overload
○Context loss after consolidation
●Cost (too expensive)
○LLM API
○Embedding Model API
○Vector database
BUILDING YOUR AI AGENT

20. SOLUTION.
04

21. ●Transcribe with speaker diarization
○Google Cloud Speech.
○Converts speech-to-text.
○Text returned, tagged orderly by speakers identified
●Text summarization:
○Google Gemini
○Prompt designed to understand transcribed text and generate document close to
medical records.
●APIs are open (REST)
○Simple to integrate (written in Java).
●Not too cheap, not too expensive either.
CLOUD

22. ●Not all AI APIs are the same.
●Joined the Eclipse Foundation to standardize AI APIs -> AI Framework for Java
○Eclipse Microprofilefor AI
○One simple framework for developers to access any AI models from various providers
with ease.
Standardization of AI APIs.

23. FINALLY.
05

24. ●A Non-profit Organisation that introduces kids to programming & robotics in
a fun way.
●Started in 2016, hosted various events countrywide.
●Won numerous awards:
○2020 –DzoneCommunity Award
○2023 –JCP Award (Java in Community). Awarded by Oracle US.
●Future plans
○Host future event workshops
○Provide full online courses online that schools, parents can use to teach kids at
home.
●Fundingisalways welcome!
DEVOXX4KIDS SOUTH AFRICA
https://www.devoxx4kids.org.za/

25. Thank you!

26. Questions?

27. Where to find me?
https://linktr.ee/theEliteGentleman
[email protected]
@EliteGentleman
www.sindi.co.za
www.devoxx4kids.org.za
https://jakartaee-ambassadors.io/