Building a Semantic Layer of your Data Platform

Enterprise Knowledge’s Joe Hilger, COO, and Sara Nash, Principal Consultant, presented “Building a Semantic Layer of your Data Platform” at Data Summit Workshop on May 7th, 2024 in Boston, Massachusetts.

This presentation delved into the importance of the semantic layer and detailed four real-world applications. Hilger and Nash explored how a robust semantic layer architecture optimizes user journeys across diverse organizational needs, including data consistency and usability, search and discovery, reporting and insights, and data modernization. Practical use cases explore a variety of industries such as biotechnology, financial services, and global retail.

1. BUILDING A SEMANTIC LAYER OF YOUR DATA PLATFORM

DATA SUMMIT WORKSHOP
MAY 2024

Joe Hilger, COO and Principal ([email protected])
Sara Nash, Principal Consultant ([email protected])

2. What You Will Learn Today
1.The business case for semantic layers
2.Definitions of the semantic layer and
its components
3.Real world applications for semantic
layers
4.How to prototype semantic layer use
cases and models
@EKCONSULTING

3. Introductions
Joe Hilger
COO and Principal Consultant
Enterprise Knowledge
Sara Nash
Principal Consultant
Enterprise Knowledge
@EKCONSULTING

4. Why Semantic
Layers?

5. ENTERPRISE KNOWLEDGE
Modern Data
Platforms…

Provide Answers
and Context

6. ENTERPRISE KNOWLEDGE
Modern Data
Platforms…

Enable Discovery

7. ENTERPRISE KNOWLEDGE
Modern Data
Platforms…

Increase Access
and Trust in Data

8. ENTERPRISE KNOWLEDGE
Modern Data
Platforms…

Break Down Silos

9. ENTERPRISE KNOWLEDGE
How are these experiences enabled?
●Content (knowledge, data, and
information) is managed and
accessible
●Data is connected across
repositories, databases, and
applications
●Context and meaning is
embedded with source data,
making common understanding
of data machine-readable

10. ENTERPRISE KNOWLEDGE
What is a Semantic Layer?
“A semantic layer is a standardized
framework that organizes and
abstracts organizational
knowledge and data (structured,
unstructured, semi-structured) and
serves as a connector for all
organizational knowledge assets.”

11. The Semantic Layer – Your Data’s “Rosetta Stone”
A semantic layer encodes
information based on context and
business logic via a broader
understanding of what those data
values mean to an organization.
The components of the semantic
layer (Knowledge Graph, Taxonomy,
Metadata Service, etc.) work together
to translate data across systems,
enabling comprehensive analytics,
reporting, and search applications.

12. ENTERPRISE KNOWLEDGE
INCONSISTENT
METADATA
NON-INTUITIVE USER
INTERACTIONS
INEFFICIENT
DATA ANALYSIS
PROCESSES
What Problems Does a Semantic Layer Solve?
POOR DATA QUALITY &
GOVERNANCE
VENDOR LOCK

AI HALLUCINATIONS

13. ENTERPRISE KNOWLEDGE
Standardizing Metadata
Unified language used to
talk about information
across the organization
Connecting Information
Structured and
unstructured data and
content are both human
and machine-readable.
Utilizing AI & LLMs
AI-supported applications
like LLMs perform with
higher fidelity and avoid
hallucinations
Reporting and Analyzing
Search results display
information that exists in
multiple locations and
formats – producing a
one-stop data hub
What Does a Semantic Layer Enable?

14. Foundations of
Semantic Layers:
Components

15. Describe
Standardize
Catalog



Metadata
Product Name: iPhone 7
Product Type: Electronics
Product Dataset: prod_data
Date Created: 02-20-2024
What data do we have on our products?
How can we quickly identify and
retrieve product images for our
marketing campaigns?
Which datasets are capturing
profitability from product Sales?

16. Define
Align
Index
Business Glossary
Term TypeDef
Profitability INTThe degree to which
a product sale yields
financial gain
How do we define profitability for a
product area?
What is the meaning of "customer
journey" in our company, and how is it
mapped?
What is the agreed-upon definition of
key performance indicator (KPI) in our
organization?

17. Categorize
Organize
Architect
Taxonomy |
Information Architecture
Electronics
TVs
Cell Phones
Which products fall under this
Product category?
How can we better organize our product
content to enhance customer service
efficiency?
How can we improve the organization of
our online product website for a smoother
customer browsing experience?

18. Relate
Map
Contextualize
Ontology
Product
Region
Customer
Sale
HasCustomer
HasSale
HasLocation
LivesIn
How can we ensure a unified
understanding of product data across
departments?
What are the sequential steps in our
supply chain process, and how are they
interrelated?
How do Online Sales relate to Regions?

19. Connect
Analyze
Infer
Knowledge Graph
iPhone7
NorthEast
Joebot
$$$$
What is the most purchased product
across customers in a given region?
What is our revenue from this product
in Store A in 2023?
How does a disruption in one part of our
supply chain affect other components?

20. Foundations of
Semantic Layers:
Enabling Technology

21. ENTERPRISE KNOWLEDGE
Semantic
Layer
APIs ETL
Application
Search
Research &
Analytics
Recommendations
& Chatbots
Admin &
Governance
Knowledge Graph
Metadata Service
Business Glossary
Taxonomy
RDF Ontology / LPG Schema
Content
Management System
Data Lake / Data
Warehouse
Subscriptions External Sources
Data Sources
Presentation
Layer
APIs
High Level Architecture

22. ENTERPRISE KNOWLEDGE
Semantic
Layer
APIs ETL
Application
Search
Research &
Analytics
Recommendations
& Chatbots
Admin &
Governance
Knowledge Graph
Metadata Service
Business Glossary
Taxonomy
RDF Ontology / LPG Schema
Content
Management System
Data Lake / Data
Warehouse
Subscriptions External Sources
Data Sources
Presentation
Layer
APIs
Semantic Layers and AI
Fact-based natural language query
Relationship-based recommendations
Advanced analytics
Extract entities and relationships
Summarize unstructured data
Recognize patterns and curate facts

23. APPLICATIONS
What products
support the
Western Blot
process?
Natural Language
Search
Categorization and
Classification
Advanced Analytics
Recommender System
Entity Recognition
Clustering and Similarity
Algorithms

Link Detection
Categorization
Semantic Layers & AI
ENTERPRISE KNOWLEDGE

24. ENTERPRISE KNOWLEDGE
What Is NOT a Semantic Layer?
Not Just for
Data
Not A Single
Product
Not Just a
Graph
Not All of Your
Content
Not an Automated/
AI-Generated
Solution

25. Why Our Clients are
Investing in
Semantic Layer

26. ENTERPRISE KNOWLEDGE
Top Enterprise
Use Cases

27. ENTERPRISE KNOWLEDGE
Reporting and Insights
Use Case 1

28. ENTERPRISE KNOWLEDGE
The Challenge
The Solution
●The solution provided a 360° view of data
used for analysis across all systems
●The knowledge graph enabled
automated creation of reports for
regulatory filings and analysis of lots of
different process details in depth.
The Results
An organization’s team of scientists needed
to quickly find and get insights about drug
development processes. However, their
insights were limited to what the scientists
could manually aggregate from siloed legacy
systems with different naming conventions.
●Develop a comprehensive ontology to
model the drug development process
and standardized nomenclature.
●Build a knowledge graph that
aggregated and normalized disparate
data from legacy systems.
Use Case 1: Global Biotechnology Company

29. ENTERPRISE KNOWLEDGE
Use Case 1: Global Biotechnology Company

30. ENTERPRISE KNOWLEDGE
Use Case 1: Global Biotechnology Company
What is the dataset we used for
this Experiment?
How do we define a
Product Run?
What Materials did we use
to create this Product?

What was the Average Cell
Viability for this Product?
What are the Measurements
used in this Experiment?
What are our Products?

Which Business Unit led this
Project?

31. ENTERPRISE KNOWLEDGE
Use Case 2
Enterprise Data Architecture

32. ENTERPRISE KNOWLEDGE
The Challenge
The Solution
●The semantic model allowed consistent
categorization of risk data across 15+
systems.
●Deduplicated risk descriptions by 40%
using AI-based approaches.
●The centralized and standardized
structure for data makes it easier to
connect and handle data on a large scale.
The Results
A multinational financial services firm
needed strategy support in implementing a
data modernization program to solve their
organizations problems with risk
identification and the user experience
around reporting.
●Build central taxonomy network to
provide consistent categorization of risk
data across systems.
●Develop semantic model for data
connectivity to drive knowledge panels,
recommendation engines, and semantic
search.
●Leverage LLMs to standardize risk
category descriptions.
Use Case 2: Financial Services Firm

33. Assessment
Use Case 2: Financial Services Firm
Risks Process
Organization
Business
Units
Legal
Entities
Regions
Products
/Services
Controls
Incidents
Compliance
Rules Policies
Complaints
Regulatory
Matters
Monitoring &
Testing
Priority Risk
Programs
Identify
Mitigate
Governance, Procedures, and Training
Data, Metrics, and Reporting

34. Key Takeaways

●Enterprise Data-warehouse to
drive data aggregation for
reporting and analytics
●Shared Services for
authentication, entitlements,
and logging
●Semantic Services and
Storage to support graph use
cases such as semantic search,
personalization, data quality
and connectivity
●Data Provider Model for
semantic data exchange
between business applications
and semantic layer
Enterprise Architecture
Use Case 2: Financial Services Firm

35. ENTERPRISE KNOWLEDGE
Use Case 2: Financial Services Firm
What is the Rule relevant to this
Risk?
What is a Business Unit vs.
a Product?
What Controls are
mitigating this Risk?
What are the Risk Ratings for
this Assessment?
What are our Products?

How should we describe our
Controls?

36. ENTERPRISE KNOWLEDGE
Use Case 3
Data
Modernization &
User Journeys

37. ENTERPRISE KNOWLEDGE
The Challenge
The Solution
●Enabled creation of a semantic layer to
support organization-wide data
discoverability
●Provided a deep understanding of the
targeted improvements to data
accessibility that would enhance user
satisfaction, engagement, and efficiency
●Identified short, medium, and long term
goals for reaching the defined target state
The Results
A global retail chain needed support in
confronting challenges with finding,
connecting to, and understanding existing
data related to tracking the health of their
stores.
●Create business glossary and taxonomy
to tag business assets for increased
findability and understanding
●Create journey maps describing current
state and target state persona
experiences.
●Design detailed roadmap outlining key
steps for improving organizational
semantic maturity
Use Case 3: Global Retail Chain

38. End State
Independent
understanding the
business meaning
Minutes to see results
Data democratization
and ownership

Current State
Heavy reliance on the IT
data analytics team
Weeks of discussion to
get results
High volume of data
USER STORY: As an Executive…want to understand the business meaning of data …so that I can
make a quick and informed decision about what business strategy my team should use.
Step 1

Executive reads a data
analytics report.
Step 2

Executive doesn’t
understand what the
data is trying to
represent.
Step 3

Executive approaches
data analytics team
asking for explanation.
Step 4

Executive waits for data
analytics team to
consult about the
meaning of data
included in the report.
Step 5

Executive receives an
explanation from the
data analytics team
about data meaning.
Step 6

Executive can make a
strategic decision based
on the meaning of the
data in the report.



Step 1

Executive reads a data
analytics report.
Step 2

Executive looks up the
report in a UI connected
to the Semantic Layer.
Step 3

Executive finds business
definitions of the data
in the report.
Step 4

Executive can make a
decision based on the
meaning of the data in the
report.



Use Case 3: Global Retail Chain
User Journey Map

39. End State
Minutes to find and
understand tables
Self-directed discovery
to vet data
Current State
Weeks to find and
understand tables
Meetings to determine
if the tables can be used
USER STORY: As a Data Analytics Team Member…I want to find and understand relevant data
tables…so that I can easily reuse existing data for a project.
Step 1

DA Team Member is
assigned a project that
requires data analytics.
Step 2

DA Team Member
spends hours to days
looking through familiar
data repositories for
data tables that could
be useful.
Step 3

DA Team Member is
unable to find what
they need, and asks a
SME for instructions on
where to look.
Step 4

DA Team Member
spends weeks looking
through recommended
data repositories, and
finds tables that seem
relevant.
Step 5

DA Team Member does
not understand the
table’s columns, and
asks a SME for
definitions and
calculation rules.
Step 6

DA Team Member is
able to determine if the
existing data table is
suitable for their
project.



Step 1

DA Team Member is
assigned a project that
requires data analytics.
Step 2

DA Team Member
searches for data tables
in a UI connected to the
Semantic Layer.
Step 3

DA Team Member finds
data tables with
metadata and column
definitions.
Step 4

DA Team Member is able to
determine if the existing
data table is suitable for
their project.



Use Case 3: Global Retail Chain
User Journey Map

40. ENTERPRISE KNOWLEDGE
Use Case 3: Global Retail Chain
Where do I find the latest
Performance Reports?
What is the definition of a
Data Product?
Who are the domain
Experts?

What caused a Store
Outage?
Which Metrics does a given
Store need to track?
What Topic do I use to tag my
Data?

41. ENTERPRISE KNOWLEDGE
Use Case 4
Data Consistency
and Usability

42. ENTERPRISE KNOWLEDGE
The Challenge
The Solution
●The contribution model enabled 10+
departments to contribute and lead
semantic model development and
governance.
●Drove the implementation of data
standards through the publication of the
enterprise ontology.
●Increased data awareness, consistent
understanding, and alignment for users
across departments and technologies.
The Results
A large financial corporation had a lack of
alignment around the meaning, format, and
intent of data elements across organizational
divisions, reducing the ability of data
producers and consumers to find, use, and
and trust data.
●Develop an enterprise ontology to
standardize data from multiple systems
and migrate from an existing physical
data model.
●Implement a federated ontology
governance and contribution model.
●Leverage standardized ontology
concepts throughout the data lifecycle.
Use Case 4: Financial Services Company

43. stateCode
CountrySubdivisionCode
Subdivision_Code
state_postal_code
USPS_Code
STATE
USAL
USAK
USAZ
……
countryCode
IsoCountryCode
alpha_code_country
geo.country_code2
iso_3166_country_code_2
CTRY
@EKCONSULTING
Use Case 4: Financial Services CompanyUse Case 4: Financial Services Company

44. ENTERPRISE KNOWLEDGE
Where can I access Compliant
Data Models?
How do we define
Credit Risk across
departments?
What Legacy Systems
were integrated?
How is risk data Classified? How is Data structured for
Regulatory Compliance?
Use Case 4: Financial Services CompanyUse Case 4: Financial Services Company

45. Let’s Roll Up Our
Sleeves

46. ENTERPRISE KNOWLEDGE



Activity 1: Use Case Definition

47. Establishing Your Prototype Use Case:
Sample
Semantic Layer
Use Case
User Story
As an online learner, I want to see course
recommendations when I get an assessment
question incorrect, so that I can upskill in that
area of weakness.
Specifications
Source Data Assets
●Course Library
●Question and
Assessment Library
●Healthcare Topic
Taxonomy
Key Knowledge Concepts
●Course
●Question
●Topic
●Healthcare
Setting
@EKCONSULTING

48. ENTERPRISE KNOWLEDGE
Activity Instructions
1.As a group, complete 3-5 use case
worksheets (15 minutes).
You can select a use case from
your experience or make up
examples.
1.Each group will present their a few
of the use cases we will select
favorites to model.
2.In our next activity, you will be
modeling your selected use case.
@EKCONSULTING

49. ENTERPRISE KNOWLEDGE



Activity 2: Modeling

50. How to Model Your Knowledge Graph
@EKCONSULTING
▪Entity – A unique type of thing that you want to define and relate in your model.
▪Attributes – or properties. Aspects, features, characteristics, descriptors, or
parameters that describe and differentiate instances of an entity.
▪Relationships – The types of connections that can be defined between entities.
Person
Works For
Company
Address
Founding
Date
Annual
Revenue
Sells

51. Activity Instructions
Define
Entities

Write the Entity names on
hexagons.
Define
Relationships

Using string and tape,
define Relationships by
connecting two Entities.
Define
Attributes

Write the Attribute names
on sticky notes and stick to
the relevant Entity.
With your group, build out the semantic layer model for your use case using the materials provided.

Definition of Done: The data concepts required to support your use case are incorporated.
Customer
Custome
r
Product
Customer
Name Age
Buys
Customer
NameAge
Product
SizeColor
Buys
Product
Family
Size
Color
partOf
@EKCONSULTING

52. [email protected]

Joe Hilger
WWW.LINKEDIN.COM/IN/JOSEPH-HILGER/
[email protected]

Sara Nash
WWW.LINKEDIN.COM/IN/SARA-G-NASH

Contact Us
@EKCONSULTING