Apidays Singapore 2024 - APIs in the world of Generative AI by Claudio Tag, IBM

APIs in the world of Generative AI
Claudio Tag, Chief Architect – IBM Automation – APAC - IBM

Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024)

------

Check out our conferences at https://www.apidays.global/

Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8

Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io

Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/

1. APIs in the world
of Generative AI
IBM @ apidays
The Arts House at the Old Parliament, Singapore
17 & 18 April 2024
Claudio Tag
Chief Architect, IBM Automation
Asia Pacific

3. Artificial Intelligence (AI)
Human intelligence exhibited by machines
Machine Learning (ML)
Systems that learn from historical data
Deep Learning (DL)
ML technique that mimics
human brain function
Generative AI
Large Language Models / Foundation Models
AI can be defined as a technique that enables machines to mimic
cognitive functions associated with human minds – cognitive functions
include all aspects of learning, reasoning, perceiving, and problem solving.
ML-based systems are trained on historical data to uncover patterns.
Users provide inputs to the ML system, which then applies these inputs
to the discovered patterns and generates corresponding outputs.
DL is a subset of ML, using multiple layers of neural networks, which are
interconnected nodes, which work together to process information. DL is
well suited to complex applications, like image and speech recognition.
AI model built using a specific kind of neural network architecture,
called a transformer, which is designed to generate sequences
of related data elements (for example, like a sentence).
1950’s1980’s2010’s2020’s
A brief history of Artificial Intelligence
© 2024 IBM Corporation

4. IBM’s perspective
on AI and APIs
© 2024 IBM Corporation
AI empowers APIs
Applying AI within API products
enables users to achieve powerful
and differentiating results
APIs enable AI
APIs are a critical factor for
the success of organisations
delivering AI projects
AI delivers benefits
across the user spectrum
– business, developers,
and operations
Enhances user
effectiveness as AI
performs manual or
time-consuming work
AI supports and
enhances human-led
innovation, without
replacing the human

5. APIs are critical for
successful AI
projects
© 2024 IBM Corporation
AI models require
access to training data
Successful consumer
interactions require
access to enterprise
systems
APIs are an important
enablement point for AI –
which needs to be
governed
APIs enable AI
As organisations look to deliver their own AI projects, we
see APIs serving a critical role in enabling success:

6. © 2024 IBM Corporation
When will my
package arrive?
It will be delivered
by 2PM on
Wednesday
Successful consumer
interactions require
access to enterprise
systems
Any meaningful consumer interaction requires integration with enterprise
data systems to provide valuable responses, typically through an API

7. © 2024 IBM Corporation
When will my
package arrive?
It will be delivered
by 2PM on
Wednesday
Successful consumer
interactions require
access to enterprise
systems
Understand customer
question using Natural
Language Processing
Retrieve answers or take action
on backend systems using REST
or GraphQL APIs
Convert backend response to
human language using
watsonx.ai
Any meaningful consumer interaction requires integration with enterprise
data systems to provide valuable responses, typically through an API

8. © 2024 IBM Corporation
AI models require
access to training and
tuning data
Sources of business
messages or events
Messages
Event Streams
(Kafka)
Training data
for AI models
Tuning data
for AI models

9. © 2024 IBM Corporation
AI models require
access to training
data
Sources of business
messages or events
Messages
Take a copy of business messages as they
flow through the system, to train the AI model
without affecting normal business processing
Add an additional subscriber to the Kafka
topic to create an independent consumer of
the Kafka events to tune the model in real time
Training data
for AI models
Tuning data
for AI models
Event Streams
(Kafka)

10. APIs are a critically
important enablement
point for AI
© 2024 IBM Corporation
When a new case is raised in Salesforce, expedite the processing to send
an alert via Slack instead of email if the tone of the case is angry, so that
it can be responded to urgently.
Analyze sentiment
watsonx
Route this incoming
helpdesk email request to
the appropriate department
and create a high-level
summary of the text
Analyze a freeform Word doc
CV for a job application to
extract key information such
as Name, Email address,
Phone Number, and
candidate’s specialist skills
Retrieve a WebEx/Teams
meeting transcript and
automatically summarize it
into the key notes and
actions intro Trello

11. APIs are a critically
important enablement
point for AI
© 2024 IBM Corporation
When a new case is raised in Salesforce, expedite the processing to send
an alert via Slack instead of email if the tone of the case is angry, so that
it can be responded to urgently.
Analyze sentiment
watsonx
Route this incoming
helpdesk email request to
the appropriate department
Analyze a freeform Word doc
CV for a job application to
extract key information such
as Name, Email address,
Phone Number, and
candidate’s specialist skills
Retrieve a WebEx/Teams
meeting transcript and
automatically summarize it
into the key notes and
actions intro Trello
Sentiment analysis
Classification
Named entity recognitionSummarization

12. © 2024 IBM Corporation
Enabling controlled
access to AI –
AI API ManagementProviding managed
access to internal and
external AI models
API Management

13. © 2024 IBM Corporation
Providing managed
access to internal and
external AI models
API Management
1. AI models typically
accessed via API, secured
using an authorized API key
to access paid service tiers
2. Large numbers of
individual consumers inside
the enterprise want to
consume those paid AI APIs
3. Use API Management to implement
“reverse API management”, providing
managed access for internal consumers
to access external APIs
a.Single paid plan subscription to the external provider
(embed the external API key in the API flow)
b.API Developer portal to enable discovery of the
available API / models
c.Self-service registration for internal consumers to
generate individual API keys
d.API analytics to monitor internal consumption for
metrics and chargeback
Enabling controlled
access to AI –
AI API Management

14. © 2024 IBM Corporation
Enabling easier
consumption of
multiple AI APIs –
AI API orchestration
Providing a single
interface to query and
orchestrate access to
AI models
API orchestration

15. © 2024 IBM Corporation
Providing a single
interface to query and
orchestrate access to
AI models
API orchestration
1. Multiple AI models
expose different capabilities
through REST interfaces
2. Most use cases require
complex querying and
orchestration logic which
ends up in the application
code
3. Use a GraphQL engine to provide
an easier mechanism for app
developers to consume LLM and
traditional AI models
a.“GraphQLize” REST endpoints
b.Provide a simple query mechanism allowing the logic to
be dynamically provide in the query, not in the app code
c.Orchestrate multiple REST operations in an API flow
invoked via a single command
d.Combine multiple LLMs and AI models (from different
vendors) in fewer endpoints exposed to your users
Enabling easier
consumption of
multiple AI APIs –
AI API orchestration

16. IBM’s perspective
on AI and APIs
© 2024 IBM Corporation
AI empowers APIs
Applying AI within API products
enables users to achieve powerful
and differentiating results
APIs enable AI
APIs are a critical factor for
the success of organisations
delivering AI projects
AI delivers benefits
across the user spectrum
– business, developers,
and operations
Enhances user
effectiveness as AI
performs manual or
time-consuming work
AI supports and
enhances human-led
innovation, without
replacing the human

17. Empowering API
users through
application of AI
© 2024 IBM Corporation
Substantial benefits in productivity and efficiency for users
through application of AI when creating and managing APIs
Capabilities that span
generative, assistive,
proactive and
predictive AI
Product features deliver
advanced outcomes
powered by AI
AI Assistants that support
the user to be more
effective in achieving
their goals
AI empowers APIs

18. AI brings benefits across the whole user spectrum
Business technologist
Need no-code, high
productivity tools to
rapidly assemble
integrations
App / API developer
Need smart tooling
to access and expose
back-end data
Integration expert
Need powerful IDE
including graphical
tools & custom
coding
Operations/SRE
administrator
Simplify
administration and
reduce toil to
improve resilience
and availability
Line of BusinessIT
© 2024 IBM Corporation

19. © 2024 IBM Corporation
IBM AI for APIs | Features available today
Mapping Assist
Intelligent recommendations
to automate the time-
consuming tasks in API dev
AI-generated
flow creation
Simplify productivity by
generating flows using
natural language
Data Assist
Generate complex
transformation functions
with no coding
Simplify developmentAccelerate testingSafeguard production

20. © 2024 IBM Corporation
IBM AI for APIs | Features available today
Mapping Assist
Intelligent recommendations
to automate the time-
consuming tasks in API dev
Data Assist
Generate complex
transformation functions
with no coding
AI-generated
flow creation
Simplify productivity by
generating flows using
natural language
Simplify developmentAccelerate testingSafeguard production
LLM-based back-
end creation
Generate back-end code
(frame and stub) from Open
API specification using LLMs
AsyncAPI spec
creation
Generate Async API specs
from Kafka & asynchronous
endpoints (e.g. MQ)
LLM-based API
spec creation
Generate Open API
specification from legacy
code using LLMs

21. © 2024 IBM Corporation
IBM AI for APIs | Features available today
Mapping Assist
Intelligent recommendations
to automate the time-
consuming tasks in API dev
Data Assist
Generate complex
transformation functions
with no coding
AI-generated
flow creation
Simplify productivity by
generating flows using
natural language
Simplify development
Auto Test Assist
Auto-generate thousands of
API testcases to probe the
behavior of your API
Automated Test
Creation
Auto-generate API test
framework using the API
specification
Accelerate testingSafeguard production
LLM-based back-
end creation
Generate back-end code
(frame and stub) from Open
API specification using LLMs
AsyncAPI spec
creation
Generate Async API specs
from Kafka & asynchronous
endpoints (e.g. MQ)
LLM-based API
spec creation
Generate Open API
specification from legacy
code using LLMs

22. © 2024 IBM Corporation
IBM AI for APIs | Features available today
Mapping Assist
Intelligent recommendations
to automate the time-
consuming tasks in API dev
Data Assist
Generate complex
transformation functions
with no coding
AI-generated
flow creation
Simplify productivity by
generating flows using
natural language
Simplify development
Auto Test Assist
Auto-generate thousands of
API testcases to probe the
behavior of your API
Automated Test
Creation
Auto-generate API test
framework using the API
specification
AI-insight driven test
generation
Identify gaps in synthetic test coverage
compared to real world invocation patterns
Accelerate testingSafeguard production
Noname
Security
Detect vulnerabilities,
misconfigurations and
malicious actors using ML
LLM-based back-
end creation
Generate back-end code
(frame and stub) from Open
API specification using LLMs
AsyncAPI spec
creation
Generate Async API specs
from Kafka & asynchronous
endpoints (e.g. MQ)
LLM-based API
spec creation
Generate Open API
specification from legacy
code using LLMs

23. With IBM you can manage APIs, GraphQL, and Events in one way, one place -
across multiple gateway types – consuming & leveraging AI models and LLMs
IBM
Developer
Portal
IBM API &
Event Endpoint
ManagerAPI
Implementation
IBM API gateway & GraphQL engine
API
Implementation
IBM Event gateway
Producer
application
Producer
application
Producer
application
Event brokerEvent broker
Third-party API gateway
© 2024 IBM Corporation

24. Summary
© 2024 IBM Corporation
AI empowers APIs
Applying AI within API products
enables users to achieve powerful
and differentiating results
APIs enable AI
APIs are a critical factor for
the success of organisations
delivering AI projects
AI delivers benefits
across the user spectrum
– business, developers,
and operations
Enhances user
effectiveness as AI
performs manual or
time-consuming work
AI supports and
enhances human-led
innovation, without
replacing the human

25. Thank you
Claudio Tag
Chief Architect, IBM Automation, Asia Pacific
https://www.linkedin.com/in/claudiotag