SAP Data Intelligence Cloud, DIC01 Col10

Explore SAP Data Intelligence Cloud and understand how it enables organizations to manage diverse data sources with confidence. This ebook provides a strategic overview of the platform, details on connection and metadata management, and guidance on building data-driven applications with pipelines and workflows. Readers will discover how intelligent processing and machine learning integration extend the value of their data across the enterprise.

The content emphasizes orchestration of data processes and integration into the Intelligent Enterprise framework. It helps professionals recognize how SAP Data Intelligence Cloud empowers business transformation by providing governance, cataloging, and advanced analytics opportunities in a unified environment.

1. DIC01
SAP Data Intelligence Cloud
.
.
PARTICIPANT HANDBOOK
INSTRUCTOR-LED TRAINING
.
Course Version: 10
Course Duration: 3 Day(s)
Material Number: 50160687
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

2. Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

3. SAP Copyrights, Trademarks and
Disclaimers
©
2022 SAP SE or an SAP affiliate company. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the
express permission of SAP SE or an SAP affiliate company.
SAP and other SAP products and services mentioned herein as well as their respective logos are
trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other
countries. Please see https://www.sap.com/corporate/en/legal/copyright.html for additional
trademark information and notices.
Some software products marketed by SAP SE and its distributors contain proprietary software
components of other software vendors.
National product specifications may vary.
These materials may have been machine translated and may contain grammatical errors or
inaccuracies.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only,
without representation or warranty of any kind, and SAP SE or its affiliated companies shall not be liable
for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate
company products and services are those that are set forth in the express warranty statements
accompanying such products and services, if any. Nothing herein should be construed as constituting an
additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business
outlined in this document or any related presentation, or to develop or release any functionality
mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’
strategy and possible future developments, products, and/or platform directions and functionality are
all subject to change and may be changed by SAP SE or its affiliated companies at any time for any
reason without notice. The information in this document is not a commitment, promise, or legal
obligation to deliver any material, code, or functionality. All forward-looking statements are subject to
various risks and uncertainties that could cause actual results to differ materially from expectations.
Readers are cautioned not to place undue reliance on these forward-looking statements, which speak
only as of their dates, and they should not be relied upon in making purchasing decisions.
© Copyright. All rights reserved. iii
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

4. Typographic Conventions
American English is the standard used in this handbook.
The following typographic conventions are also used.
This information is displayed in the instructor’s presentation
Demonstration
Procedure
Warning or Caution
Hint
Related or Additional Information
Facilitated Discussion
User interface control
Example text
Window title
Example text
iv © Copyright. All rights reserved.
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

5. Contents
vii Course Overview1 Unit 1: SAP Data Intelligence Cloud - Strategy and Overview3 Lesson: Using Data Process Orchestration for Intelligent Enterprise27 Unit 2: Connection Management in SAP Data Intelligence Cloud29 Lesson: Integrating Data Sources and Data Targets in SAP Data
Intelligence
39 Lesson: Integrating SAP Systems51 Lesson: Integrating non-SAP Systems, Applications, and Storages57 Unit 3: Metadata Management and Information Cataloging59 Lesson: Data Discovery and Information Cataloging with the SAP
Data Intelligence Metadata Explorer
71 Lesson: Leveraging Business Glossaries and Managing Data Object
Relationships
77 Lesson: Using Data Quality Rules and Dashboards81 Lesson: Enabling Business Users to Augment Data91 Unit 4: Building Data Driven Applications in Data Intelligence Cloud with
Data Pipelines and Workflows
93 Lesson: Building data-driven applications with the SAP Data
Intelligence Modeler
97 Lesson: Getting an introduction to data pipelines and workflows107 Lesson: Modeling and executing data pipelines and workflows125 Lesson: Monitoring and status tracking of data pipelines127 Lesson: Debugging your data pipelines131 Lesson: ABAP System integration with SAP Data Intelligence141 Unit 5: Intelligent Processing - Machine Learning Integration in SAP
Data Intelligence Cloud
143 Lesson: Operationalizing machine learning / analytical models with
the SAP Data Intelligence ML Scenario Manager
149 Lesson: Realizing intelligent processing with the tooling components
of the ML Scenario Manager
165 Unit 6: Appendix: Positioning167 Lesson: Backup: Positioning
© Copyright. All rights reserved. v
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

6. vi © Copyright. All rights reserved.
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

7. Course Overview
TARGET AUDIENCE
This course is intended for the following audiences:
© Copyright. All rights reserved. vii
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

8. viii © Copyright. All rights reserved.
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

9. UNIT 1
SAP Data Intelligence Cloud -
Strategy and Overview
Lesson 1
Using Data Process Orchestration for Intelligent Enterprise
3
UNIT OBJECTIVES
●Understand the concepts and strategy behind SAP Data Intelligence
© Copyright. All rights reserved. 1
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

10. Unit 1: SAP Data Intelligence Cloud - Strategy and Overview
2 © Copyright. All rights reserved.
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

11. Unit 1
Lesson 1
Using Data Process Orchestration for
Intelligent Enterprise
LESSON OBJECTIVES
After completing this lesson, you will be able to:
●Understand the concepts and strategy behind SAP Data Intelligence
Introduction to SAP Data Intelligence
Overview
The course examines Data Process Orchestration for the Intelligent Enterprise using SAP
Data Intelligence.
This lesson covers the following:
●An introduction to SAP Data Intelligence
●When to use what - the components and user applications in SAP Data Intelligence
●Typical use cases and business scenarios for SAP Data Intelligence
●The Starting Point - how to access the Data Intelligent user frontends with the
(personalized) launchpad application
●Architecture basics - why is SAP Data Intelligence so resilient?
Enterprise Applications and Intelligent Technologies: New Opportunities and New
Challenges
The following figure, Enterprise Applications and Intelligent Technologies: New Opportunities
and New Challenges, outlines the means by which you can bring enterprise applications and
intelligent technologies together.
© Copyright. All rights reserved. 3
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

12. Figure 1: Enterprise Applications and Intelligent Technologies: New Opportunities and New Challenges
A large number of new opportunities emerge as result of the integration and connectivity
options that businesses can leverage today. This is not simply the case within the internal
system environments, but also with external service / data providers, partners, and
customers.
Close to the entirety of the complete value chain can be covered, from research and planning
processes to production, sales and distribution.
However, this goes hand-in-hand with an increased heterogeneity of integrated enterprise
systems and data storages, their (geographical) distribution, the disparity of data, and finally,
a higher complexity of business processes and analyses.
The design and operationalization of end-to-end data management and lifecycle scenarios
becomes a real challenge.
In addition, it is appropriate to consider the embedment of Intelligent Technologies in all
phases of data management processes in order to enrich enterprise applications by holistic
data governance concepts, the end-to-end orchestration and automation of data processing,
and Machine Learning (ML) capacities for data quality improvements and decision support.
Data Management - Difficulties
The following figure, The Difficulties of Data Management, depicts the inherent difficulties of
data management.
Unit 1: SAP Data Intelligence Cloud - Strategy and Overview
4 © Copyright. All rights reserved.
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

13. Figure 2: The Difficulties of Data Management
Data Management for end-to-end business processes can include the following aspects, or a
combination of these aspects:
●Classical Extraction, Transformation, Load (ETL) and data replication
●Data warehousing and data mart proliferation
●Data augmentation (enrichment, cleansing, matching, and so on)
●Event-based data processing / message broking / data streaming (for example, Internet
of Things (IoT) scenarios and real-time replication)
●Management of cloud data stores (data lakes / distributed data)
●Several databases (including not only SQL (NoSQL) and graph databases)
●Third-party data and services
●Information catalogs and metadata management
●Multimedia data processing (image, video, text, in-process metadata extraction)
●Advanced processing (ML, Geo, Streams, and so on)
The diversity of related tasks, the heterogeneity and distribution of system landscapes and
storages, as well as the absence of central data management concepts and platform, has
often led to a realization of point-to-point connections and data processes.
This can make the subsequent maintenance and servicing of those implementations time
consuming and challenging. In particular, the following aspects can lead to a higher degree of
complexity, which necessitates:
●Need for the integration of cloud storage and/or object stores
●Usage of diverse open-source data management tools that are leveraged on a case-by-
case basis
●Lack of internal skills to maintain these setups
Lesson: Using Data Process Orchestration for Intelligent Enterprise
© Copyright. All rights reserved. 5
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

14. ●Scalability concerns
Practically, point-to-point data movements, which are still quite commonly used as a solution
for a concrete business need, will hardly be maintainable with appropriate efforts on the long
run.
Connected Business
Figure 3: Business needs to be more Connected, Confident, and Intelligent
We have already mentioned that challenges related to data management are a reality that
enterprises must tackle.
Current studies by business analysts show that many companies are still struggling to
manage data in silos, the complexity of the system landscape, the connectivity issues, and the
synchronization or orchestration of their data processes.
A basic summary of the findings in these studies reveals the following:
●According to Forrester Consulting, 71% of the enterprises can not really combine their
(classical) enterprise data areas with the big data areas and leverage both for the
optimization of business processes and decision making.
●53% of enterprises are not even treating data as a business asset.
●This may relate to the perception by 84% of the executives at these companies that the
accuracy of data used to make business decisions appears to be insufficient.
Unit 1: SAP Data Intelligence Cloud - Strategy and Overview
6 © Copyright. All rights reserved.
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

15. Why is Data Management so Difficult?
Figure 4: Enterprise IT is Challenged to a Whole New Degree
Other than in the instances of well-structured classical backend / Enterprise Resource
Planning (ERP) systems, enterprise IT is now challenged to a whole new degree.
Changes and developments are not confined to the main dimensions of data demands, such
as integration, processing, discovery, quality, compliance, and operations.
However, they now need to be satisfied in increasingly distributed and complex data
landscapes and, in particular, within seamlessly integrated scenarios - that is, cross systems
and cross storage environments.
SAP Data Intelligence - Solutions
Figure 5: One Solution Supports End-to-end Workflows for Intelligent Enterprise Applications and Business
Processes
Lesson: Using Data Process Orchestration for Intelligent Enterprise
© Copyright. All rights reserved. 7
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

16. SAP addresses these challenges on one data orchestration platform with SAP Data
Intelligence Cloud.
SAP Data Intelligence Cloud combines individual SAP cloud services to deliver a scalable,
integrated solution embedded on SAP Business Technology Platform - to serve various use
cases, such as BW and SQL data warehouse in addition to agile analytical data marts.
In addition, it provides one unified tool set with collaboration options for all tasks related to
building data lakes, data marts, data warehouse, and analytical solutions for different
personas (for example, business users, modelers, operators, and so on). SAP Data
Intelligence Cloud does this by providing the following:
●An integrated information catalog that enables data lineage, data quality, and profiling of
datasets
●A central connection management system to connect to various data sources and targets
(on-premise and Cloud, SAP and non-SAP, structured and unstructured data)
●Tools and repositories for pipeline implementation with graphical user frontends,
versioning enabling, and an aligned lifecycle management process
●ML and analytical model integration and operationalization
●Orchestration and monitoring utilities
SAP Data Intelligence includes business-ready functionality, that is, a content library with
packages of end-to-end business scenarios for specific industries and lines of business to
support the Intelligent Enterprise. The content is delivered by SAP and its business partners.
This application from SAP manages Data Intelligence for customers so that they can innovate
faster and always leverage the latest and greatest capabilities.
SAP Data Intelligence Cloud - Advantages
We now turn to the advantages of SAP Data Intelligence Cloud.
Figure 6: SAP Data Intelligence Cloud - Core Capabilities
There are a number of advantages to using SAP Data Intelligence Cloud, which are as follows:
●Pre-built content in SAP Business Technology Platform (BTP) allows you to accelerate
your Time To Value (TTV).
Unit 1: SAP Data Intelligence Cloud - Strategy and Overview
8 © Copyright. All rights reserved.
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

17. ●SAP ensures that you drive value fast, and maximize continued innovation by offering you
over 1,800 ready-to-use integration packs, cloud extensions tailored for your industry and
line of business, and over 100 pre-built Analytics content packages.
●Some technology platforms take many months of implementation and significant
investment in order to achieve any business value. SAP Business Technology Platform is
unique. The platform comprises multiple application modules, which can be rapidly
combined together for quick TTV. This accelerated payback period fuels subsequent
customer use cases. Our customers achieve success quickly by taking advantage of rapid
value realization.
SAP Data Intelligence - End-to-end Orchestration of Data Processes
SAP Data Intelligence addresses end-to-end data processing for diverse and heterogeneous
landscapes.
It focusses on the management and integration of data assets distributed across the data
landscape in data lakes or enterprise systems.
The building blocks of SAP Data Intelligence Cloud correspond to these data processing
challenges.
Figure 7: SAP Data Intelligence End-to-End Orchestration of Data Processes
Connection Management: Out-of-the-box integration with various SAP and non-SAP systems,
applications, and storages
The Connection Management application in SAP Data Intelligence Cloud is like a "Swiss Army
Knife" for data integration purposes.
It provides multiple, instantly available connection types for data and functional integration
with SAP and non-SAP sources, targets, or runtimes.
SAP systems and application offer options, such as the following:
●SAP S/4HANA, SAP S/4HANA Cloud
●ECC (NetWeaver)
●BW, BW/4 HANA
Lesson: Using Data Process Orchestration for Intelligent Enterprise
© Copyright. All rights reserved. 9
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

18. ●SLT Replication Server
●Data Warehouse Cloud (DWC)
●SAP Analyics Cloud (SAC)
●SAP HANA (on-prem, Cloud, Multi-Cloud)
●SAP Data Services
●SAP PO/PI and CPI
●SAP BTP, including API Business Hub, Open Connector Framework, Enterprise Messaging
Note:
SAP BTP, including API Business Hub, Open Connector Framework, Enterprise
Messaging can be seamlessly integrated, either on a data and/or a functional
level.
Non-SAP system integration is certainly of similar importance. The following list is just an
excerpt of connection capabilities:
●Event-driven integration with message brokers (Kafka, MQTT, NATS, AWS SNS, Google
Pub/Sub, and so on)
●Operations on Cloud Object Stores (like AWS S3, Azure DL and WASB, Google CS, Alibaba
OSS)
●Hadoop (for example, Spark) and the Hadoop Distributed File System (HDFS)
●Cloud services and databases on Hyperscaler platforms (AWS Redshift, Google BigQuery,
Google DataProc, Azure SQL Database, and so on)
●Third-party databases (IBM DB2, Oracle, MS SQL, MySQL, and so on)
●Web services and public cloud applications
●Third-party applications that have any API
Data Governance and Information Cataloging with the Metadata Explorer
The Metadata Explorer of the SAP Data Intelligence Cloud provides you with functionality to
discover, profile, and understand the corporate data assets.
It allows you to publish data objects to the Information catalog, apply hierarchical tagging,
search for catalog items, create business glossaries and relate it to other objects.
The Metadata Explorer also allows to create and apply Business Rules like KPIs or Quality
metrics and to compare and visualize the results of the rules for the chosen data sets.
It also provides Business User Self Services for advanced data preparation and cleansing
activities without requiring technical skills, such as SQL knowledge.
Data Pipelining
The pipeline design and execution application, the DI Modeler, provides a graphical design-
time and runtime environment for the setup of data-driven scenarios.
It supports you and enables you to create data processing pipelines (also named graph, which
follows a flow-based programming paradigm) and can reuse existing coding and libraries to
orchestrate data processing in distributed landscapes.
Unit 1: SAP Data Intelligence Cloud - Strategy and Overview
10 © Copyright. All rights reserved.
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

19. The Modeler also provides a long list of predefined operators and transforms, which you can
use for many productive business use cases out-of-the-box. These operators help you to
define your data pipelines, including non-terminating, non-connected or cyclic graphs.
Intelligent Processing
The SAP Data Intelligence core ML application - that is, the ML Scenario Manager, helps you
to organize your data science artifacts and manage all tasks related to your work in one
central place.
As a multi-faceted data science application, the ML Scenario Manager is built around the key
concept of ML scenarios.
An ML scenario can contain datasets, pipelines, and Jupyter Notebooks. Within the scenario,
you can also manage the model performance metrics and deployment history.
You can version an ML scenario as part of your end-to-end workflow and, if necessary, you
can create a new branch from a previous version.
A typical process within ML Scenario Manager involves the following:
●Managing your datasets and model artifacts
●Creating Jupyter notebooks for your experiments
●Creating and managing data pipelines
●Viewing executions and performance metrics
●Tracking and versioning your model deployments
Data Orchestration and Monitoring
Task workflows allow you to combine your pipelines or the processes that have to get
executed on other runtimes (for example, BW Process Chains, SAP HANA Flowgraphs, Data
Services jobs or Hadoop Spark jobs) within one end-to-end process.
This feature helps to bring all data processes and tasks of your scenario into the correct
sequence or onto the right degree of parallelism. In this way, SAP Data Intelligence supports
the orchestration of data-driven scenarios, even across various environments.
As process steps (operations) can be dependent on each other, the complete workflow can
be monitored, which is particularly importance if you have cross runtime scenarios.
For this purpose, SAP Data Intelligence provides a stand-alone monitoring application to
monitor the status of each pipeline or workflow executed in SAP Data Intelligence Cloud.
The Monitoring application provides capabilities to visualize the summary of pipelines and
workflows executed in the Modeler with relevant charts. In addition, the application allows you
to schedule graph executions. For the instance of each graph, the application provides details,
such as graph execution status, time of execution, graph type, graph source, and more.
The application allows you to open a graph in the SAP Data Intelligence Modeler, view graph
configurations, or to stop process executions.
SAP Data Intelligence - Why it is Different?
SAP Data Intelligence Cloud provide a highly integrated data management platform that
covers the entire value chain for data processing for all relevant corporate roles.
Lesson: Using Data Process Orchestration for Intelligent Enterprise
© Copyright. All rights reserved. 11
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]

20. Figure 8: SAP Data Intelligence - Why it is different?
It is based on an adaptable architecture built on open technologies and is, hence, available as-
a-service, or bring your own license (BYOL) in the cloud on any hyperscaler or on-premise in a
backend environment.
As a result of the tight interfacing with multiple SAP and non-SAP applications, it is easy to
seamlessly integrate other runtimes and save or reuse existing investments.
Components and User Applications - What to Use
Essentially, SAP Data Intelligence is suitable for all companies that want to optimize their
integration and handling of data in diverse and disparate data sources or applications.
This is particularly true if the corporate data is stored in silos and, hence, not available
company-wide, while merging them manually would be complex or too time-consuming.
Alternatively, if existing solutions are reaching their limits in terms of governance,
controllability and automation, SAP Data Intelligence can bring in an integration layer that
enables a seamless end-to-end processing and orchestration of a multitude of cross system
scenarios in distributed landscapes.
Common use cases for SAP Data Intelligence are as follows:
●Business Application Transformation
●IoT Ingestion and data process orchestrations
●Data Warehousing support and data preparation
Typical Use Cases and Business Scenarios
The following figure, SAP Data Intelligence for Intelligent Information Management, outlines
typical uses cases for intelligent information management.
Unit 1: SAP Data Intelligence Cloud - Strategy and Overview
12 © Copyright. All rights reserved.
Librería ERP — Sample | libreriaerp.com/us | [email protected]
Librería ERP — Sample | libreriaerp.com/us | [email protected]