Unleashing the value of metadata with Talend

The proliferation of data and the desire to manage information as an asset is driving the need for better data governance. Metadata Management is gaining traction as a way to improve agility and change management to DevOps, to bring traceabality into data journeys, and foster self-service access to data. This presentation shows how Talend leverages Metadata across use cases from Hadoop to self service, and from visual design to enterprise metadata management

1. Page 1
©2016 Talend
Unleashing the value of Metadata with Talend

2. Page 2
The Mandate for Metadata Management
2
Auditability
Accessibility
Agility
IT needs to move faster and innovate more than ever
More regulations and need for accountability
need a map to track and trace the data journey
Strong governance as a prerequisite to engage a wider
audienceof business users with self service

3. Page 3
e.g. in Banking: Risk and Finance data aggregation and reporting (Basel III BCBS 239)
The need for governance and data lineage
Impact of Risk Data
& Technology Programs
21B
of annual
benefits
Typical budget
For Platforms
125M
Investment
>2%
reduction
Reduced capital
and operations costs
Source: Mc Kinsey & Company: Capturing value of BCBS 239 and beyond

4. Page 4
Why Now ?
“The metadata management tool market will double in size in 2017, driven by the
information of everything, proliferation of data from the Internet of Things (IoT), big data
and data lake forces, and requirements for organizations to know what data they have, as
informationis recognized as an asset with an attached financial value.”

5. Page 5
The five pillars for managing Metadata with Talend
Talend
Studio
Talend
Metadata
Bridge
Talend Big Data
With Cloudera
Navigator
Talend Data
Preparation
Talend
Metadata
Manager
Metadata-driven, with zero coding
Single point of control for Metadata
Dependency checks and propagation
Design once, propagate across any tools
Apply change and enforce standards
Accelerate migrations
Data Lineage with depth on Hadoop
Track, classify, and locate data & data flows
Data Governance on Hadoop
Data inventory for self-service access
Auto-discovery with smart guidance
Facet based search
Risk and compliance with data transparency
Increase agility across the data landscape
Turn data into a business language
(*) Apache Atlas
integration planned
in Talend 6.3

6. Page 6
Metadata-Driven Studio
Transformation path from
source to target
Transformation lineage
from target back to source
Not just files & tables
-Fined-grained metadata to the attribute
-Automatically propagate changes
Export & share
https://help.talend.com/display/TalendDataFabricStudioUser
Guide61EN/V.+Metadata+Management

7. Page 7
Talend Metadata Bridge
•Available from Talend 5.6.2
•Over 100 connectors to
import/export metadata from
virtually any data-driven tools and
platforms
•Capabilities
•Imports/Exports from design and
modeling environments
•Integrates with Databases, Big Data and
development platforms
•Reduces update & conversion costs
and accelerates ETL offloading projects
•Facilitates IT and Lines of Business
collaboration for data flows specification

8. Page 8
Hadoop needs Metadata Management
Lots of data
Lots of Data Types
Lots of Languages
Lots of Frameworks
One data,
many replicas,
many models

9. Page 9
+
Hadoop meets Data Governance
with Cloudera and Talend
The first and only Cloudera Navigator integration that includes Spark Data Lineage

10. Page 10
A visual approach for data flow
design, fully integrated with
Cloudera Navigator
Depth, breadth and reach for Hadoop Data Governance
End-to-end data traceability,
including upstream before the data
reaches Hadoop
Business users empowerment with
metadata through business
glossaries and data catalogs
Deep control over your
Hadoop flows
Extend beyond Hadoop
Reach new audience
with Big Data

11. Page 11
Reclaim scattered files from users’
hard drives
Single point of inventory and
access for self service data
Trace source, ownership, actions
Improve accessibility, Auto-
discover, recommend
Data Inventory (from Talend Data Preparation 1.2)
Govern
Collaborate
Control
Guide

12. Page 12
Talend Metadata Manager
•Available from Talend 6.1 (Dec, 2015)
•Allows to harvest, govern, and share
metadata across data platforms (BI,
Big Data, modelling tools, ETL, etc.)
•Provides :
•Business Glossary
•Shared definition and collaboration around data
definition and policies
•Enterprise Data Governance & Auditing
•Data Flow Lineage and impact analysis, across
systems and tools
•Ensures Data Compliance
•through model versioning, comparisons and
reporting

13. Page 13
Talend Metadata Manager –Integrated Solutions
IT and corporate servicesLines of Business
Metadata Management
Solutions
Data Governance
Solutions
Any Enterprise Architecture
from the Data Lake
to the Data Warehouse
Solutions to both business and technical users
With any vendors/tools

14. Page 14
Talend Metadata Manager –The Big Picture
For today’s evolving
Enterprise Architectures
integrating
the Data Warehouse
Semantic Lineage
Data FlowLineage Impact Analysis
Data
Warehousing
Data
Integration
Business
Intelligence
Data Governance
Data Standardization
Enterprise Architecture
Business Glossary
Semantic Mapping
Conceptual & Logical Models
Design Patterns & Reusable Components
Data Stores
& MDM
Business Applications
Business Reports
and
the Data Lake
both

15. Page 15
Talend Metadata Manager –Metadata Harvesting
http://www.metaintegration.com/Products/MIMB/SupportedTools.html

16. Page 16
Talend Metadata Manager –Enterprise Architecture

17. Page 17
Talend Metadata Manager –Metadata Browsing

18. Page 18
Talend Metadata Manager –Lineage & Impact Analysis

19. Page 19
Talend Metadata Manager –Data Model Diagrams

20. Page 20
Talend Metadata Manager –Data Governance
ISO 11179 standard based Business Glossary
Custom Attributes
Glossary Bootstrapfrom the existing enterprise data model in ERwin, and more
Bulk Change and Edit allowing for quick tabular editing and bulk changes to search results
Semantic Mappings
Governance Workflow provides a very flexible (customizable) workflow

21. Page 21
Talend Metadata Manager –
Business Intelligence Tools
BI Report Documenter
Managed Business Intelligence:
Live integration with BI Applications
Talend MM application as a powerful multi-vendor BI Web Portal
BI Report Documenter allowing users to
get live definitions from any objects on reports,
add them in place if missing,
trace data lineage from any object in reports,
find other related reports (even in other BI tools),
produce live report specific glossaries (e.g. Tableau worksheet)
Report Documenter
Business Glossary
Reuse
Terms
Create
Terms

22. Page 22
Talend Metadata Manager –
Data Modeling Tools
Business Intelligence Tools
Metadata Authoring Apps:
Data Modeler / Documenter
Data Modeler
Business Glossary
Data Stores
Active Data
Governance
Data
Standardization
Automatic Live Updates of
anydata store schema change
Reuse
Terms
Create
Terms
Forward Engineering to:
-to self service BI like Tableau
-or traditional BI Design Tools
SAP BusinessObjects UNV/IDT
IBM Cognos FM

23. Page 23
Metadata
Harvesting
Talend Metadata Manager –
Talend DI StudioTalend MM Data Mapper
Data Mapping
Specifications
Metadata Authoring Apps:
Data Mapper
Data Mapping Requirements Data Flow Implementation
Active Data
Governance
Data Flow Lineage
& Impact Analysis
Metadata
Comparision
Requirement
Compliance
DI/ETL/ELT Tool
Generation
Data Mapping
Design

24. Page 24
©2016 Talend
Unleashing the value of Metadata with Talend