Denodo Data Virtualization - IT Days in Luxembourg with Oktopus

Denodo 200 views 37 slides Nov 28, 2018
Slide 1
Slide 1 of 37
Slide 1
1
Slide 2
2
Slide 3
3
Slide 4
4
Slide 5
5
Slide 6
6
Slide 7
7
Slide 8
8
Slide 9
9
Slide 10
10
Slide 11
11
Slide 12
12
Slide 13
13
Slide 14
14
Slide 15
15
Slide 16
16
Slide 17
17
Slide 18
18
Slide 19
19
Slide 20
20
Slide 21
21
Slide 22
22
Slide 23
23
Slide 24
24
Slide 25
25
Slide 26
26
Slide 27
27
Slide 28
28
Slide 29
29
Slide 30
30
Slide 31
31
Slide 32
32
Slide 33
33
Slide 34
34
Slide 35
35
Slide 36
36
Slide 37
37

About This Presentation

Presentation by Sihem Merah, Sales Engineer at Denodo.


Slide Content

Denodo Data Virtualization Stop collecting, start connecting Sihem Merah – Denodo Senior Sales Engineer Lydie Gwizdz – Oktopus Account Manager

Data Virtualization: An Introduction Data Virtualization Platforms – Key Capabilities through Customer Success stories @ BoW , @Intel and @Asurion Demo c 3 : Connect, Combine and Consume

Denodo The Leader in Data Virtualization DENODO OFFICES, CUSTOMERS, PARTNERS Palo Alto, CA. Global presence throughout North America, EMEA, APAC, and Latin America. LEADERSHIP Longest continuous focus on data virtualization – since 1999 Leader in 2017 Forrester Wave – Enterprise Data Virtualization Winner of numerous awards CUSTOMERS ~500 customers, including many F500 and G2000 companies across every major industry have gained significant business agility and ROI. FINANCIALS Backed by $4B+ private equity firm. 50+% annual growth; Profitable .

Data Virtualization – An Introduction Why Data Virtualization? Challenges, Solution and Benefits

IT & Business data Dilemma IT focuses on Data Collection, Storage & Security Biz focuses on data Consumption, Analysis & strategic decisions Product Data Billing System Cloud/SaaS System Usage Inventory System CRM Product Catalog Customer Voice PLM Web/Social Analytics & Reporting Enterprise Apps Digital Transformation Risks & Compliance Decision making Data Science M & A Enterprise APIs

IT & Business data Dilemma (IT Spaghetti Architecture) IT focuses on Data Collection, Storage & Security Biz focuses on data Consumption, Analysis & strategic decisions Decision making Enterprise APIs Billing System Cloud/SaaS CRM PLM Product Data System Usage Inventory System Product Catalog Customer Voice Web/Social Analytics & Reporting Enterprise Apps Digital Transformation M & A Risks & Compliance Data Science

IT & Business data Dilemma But… Product Data Billing System Cloud/SaaS System Usage Inventory System CRM Product Catalog Customer Voice PLM IT focuses on Data Collection, Management & Security Analytics & Reporting Enterprise Apps Customer Support Digital Transformation M & A Revenue Collection Decision making Risks & Compliance Biz focuses on data Consumption, Analysis & strategic decisions Big Data

IT & Business data Dilemma But… Product Data Billing System Cloud/SaaS System Usage Inventory System CRM Product Catalog Customer Voice PLM IT focuses on Data Collection, Management & Security Analytics & Reporting Enterprise Apps Customer Support Digital Transformation M & A Revenue Collection Decision making Risks & Compliance Biz focuses on data Consumption, Analysis & strategic decisions Big Data Can you retrieve all the data No one data container solution for all Can you consume all the data No unique way for consuming data

IT & Business data Dilemma But… Product Data Billing System Cloud/SaaS System Usage Inventory System CRM Product Catalog Customer Voice PLM IT focuses on Data Collection, Management & Security Analytics & Reporting Enterprise Apps Customer Support Digital Transformation M & A Revenue Collection Decision making Risks & Compliance Biz focuses on data Consumption, Analysis & strategic decisions Big Data So much replication (security, data, license, storage, hardware, €, etc.) So much replication (security, data, license, storage, hardware, €, etc.) So much replication (security, data, license, storage, hardware, €, etc.) So much replication (security, data, license, storage, hardware, €, etc.) 75% data stored not used 90% request need current data 2016

IT & Business data Dilemma But… Product Data Billing System Cloud/SaaS System Usage Inventory System CRM Product Catalog Customer Voice PLM IT focuses on Data Collection, Management & Security Analytics & Reporting Enterprise Apps Customer Support Digital Transformation M & A Revenue Collection Decision making Risks & Compliance Biz focuses on data Consumption, Analysis & strategic decisions Big Data Not Time-to-Market Oriented Between request & consumption, the unit is couple of months, limited agility & devOps readiness  limited value to business

IT & Business data Dilemma But… Product Data Billing System Cloud/SaaS System Usage Inventory System CRM Product Catalog Customer Voice PLM IT focuses on Data Collection, Management & Security Analytics & Reporting Enterprise Apps Customer Support Digital Transformation M & A Revenue Collection Decision making Risks & Compliance Biz focuses on data Consumption, Analysis & strategic decisions Big Data Innovation & digital initiatives (MVPs) (APIs 1 st , independent data, fast & agile products)

IT & Business data Dilemma But… Product Data Billing System Cloud/SaaS System Usage Inventory System CRM Product Catalog Customer Voice PLM IT focuses on Data Collection, Management & Security Analytics & Reporting Enterprise Apps Customer Support Digital Transformation M & A Revenue Collection Decision making Risks & Compliance Biz focuses on data Consumption, Analysis & strategic decisions Big Data Data Governance Who can access what, where, filtering, hiding, encrypting, compliances, lineage, origin, transformation, auditability (IT & Biz)

IT & Business need… IT focuses on Data Collection, Management & Security Analytics & Reporting Enterprise Apps Customer Support Digital Transformation M & A Revenue Collection Decision making Risks & Compliance Biz focuses on data Consumption, Analysis & strategic decisions Product Data Billing System Cloud/SaaS System Usage Inventory System CRM Product Catalog Customer Voice PLM Big Data Governed & Secured Enterprise Data Delivery Platform with best Time-to-Market … …& Cost effective

Data Virtualization – Key Capabilities Customer Success stories @ BoW , @Intel and @Asurion

Essential Capabilities of Data Virtualization Data abstraction – ease of use & standardization Zero replication – on-demand & ease of usage Performance – intelligent algorithms Data Catalog – self-service & search Multi-location – ease of deployment Governance , security & compliances

-Michael Norton, VP Data Architecture, Bank of the West With Denodo, we saw about 30 to 40% increase in our ability to deliver projects to our consumers.“

Challenges Classic EDW Design + ETL + ESB Exponential growth from 2012 to 2018 Source Systems : 7 -> 110+ Nightly jobs: 30 -> 500+ DQ Checks: 120 -> 7000+ Size: 50 -> 560TB+ Emerging Big Data Environment

Solution Classic EDW Design + ETL + ESB Exponential growth from 2012 to 2018 Source Systems : 7 -> 110+ Nightly jobs: 30 -> 500+ DQ Checks: 120 -> 7000+ Size: 50 -> 560TB+ Emerging Big Data Environment Governed & Secured Enterprise Data Delivery Platform with best Time-to-Market … …& Cost effective

Solution Classic EDW Design + ETL + ESB Exponential growth from 2012 to 2018 Source Systems : 7 -> 110+ Nightly jobs: 30 -> 500+ DQ Checks: 120 -> 7000+ Size: 50 -> 560TB+ Emerging Big Data Environment Governed & Secured Enterprise Data Delivery Platform with best Time-to-Market … …& Cost effective Unique Entreprise Data Access Layer

Solution Classic EDW Design + ETL + ESB Exponential growth from 2012 to 2018 Source Systems : 7 -> 110+ Nightly jobs: 30 -> 500+ DQ Checks: 120 -> 7000+ Size: 50 -> 560TB+ Emerging Big Data Environment Governed & Secured Enterprise Data Delivery Platform with best Time-to-Market … …& Cost effective Unique Entreprise Data Access Layer ETL Replacement

Solution Classic EDW Design + ETL + ESB Exponential growth from 2012 to 2018 Source Systems : 7 -> 110+ Nightly jobs: 30 -> 500+ DQ Checks: 120 -> 7000+ Size: 50 -> 560TB+ Emerging Big Data Environment Governed & Secured Enterprise Data Delivery Platform with best Time-to-Market … …& Cost effective Unique Entreprise Data Access Layer ETL Replacement « Lift and Shift »

Solution Classic EDW Design + ETL + ESB Exponential growth from 2012 to 2018 Source Systems : 7 -> 110+ Nightly jobs: 30 -> 500+ DQ Checks: 120 -> 7000+ Size: 50 -> 560TB+ Emerging Big Data Environment Governed & Secured Enterprise Data Delivery Platform with best Time-to-Market … …& Cost effective Unique Entreprise Data Access Layer ETL Replacement « Lift and Shift » Big Data enabler

Solution Classic EDW Design + ETL + ESB Exponential growth from 2012 to 2018 Source Systems : 7 -> 110+ Nightly jobs: 30 -> 500+ DQ Checks: 120 -> 7000+ Size: 50 -> 560TB+ Emerging Big Data Environment Governed & Secured Enterprise Data Delivery Platform with best Time-to-Market … …& Cost effective Unique Entreprise Data Access Layer ETL Replacement « Lift and Shift » Big Data enabler Database Service Bus Exposure

-Stacie Hall, Enterprise Architect, Intel Data Virtualization is a game changer for our data.”

Challenges Intel’s data is globally distributed across heterogeneous tools & technologies New data sources (ex: big data) & consumers (ex: emergence of SaaS) New information exchange channels (ex: mobility) Web Services and API Management M&A BI users want fresh easily accessible data

Results Intel’s data is globally distributed across heterogeneous tools & technologies New data sources (ex: big data) & consumers (ex: emergence of SaaS) New information exchange channels (ex: mobility) Web Services and API Management M&A BI users want fresh easily accessible data

-Larry Dawson, Enterprise Architect, Asurion Our Denodo rollout was one of the easiest and most successful rollouts of critical enterprise software I have seen.”

Challenges Need for Big Data and Predictive Analytics Move to Cloud International Initiative + Personal Identifiable Information = Geographic client based constraints Asurion Premium Support Services

Solution Need for Big Data and Predictive Analytics Move to Cloud International Initiative + Personal Identifiable Information = Geographic client based constraints Governed & Secured Enterprise Data Delivery Platform with best Time-to-Market … …& Cost effective Unique Entreprise Data Access Layer Hybrid environment : Denodo on Premise and Denodo on Cloud Fine grained authorization: row, column, encryption Asurion Premium Support Services

Solution Need for Big Data and Predictive Analytics Move to Cloud International Initiative + Personal Identifiable Information = Geographic client based constraints Governed & Secured Enterprise Data Delivery Platform with best Time-to-Market … …& Cost effective Unique Entreprise Data Access Layer Hybrid environment : Denodo on Premise and Denodo on Cloud Fine grained authorization: row, column, encryption APIs: expose JSON Data Asurion Premium Support Services

Solution Need for Big Data and Predictive Analytics Move to Cloud International Initiative + Personal Identifiable Information = Geographic client based constraints Governed & Secured Enterprise Data Delivery Platform with best Time-to-Market … …& Cost effective Unique Entreprise Data Access Layer Hybrid environment : Denodo on Premise and Denodo on Cloud Fine grained authorization: row, column, encryption APIs: expose JSON Data Performance: Push-down optimization Asurion Premium Support Services

Demo Connect, Combine, Consume

Scenario What’s the impact of a new marketing campaign for each country? Historical sales data offloaded to Hadoop cluster for cheaper storage Marketing campaigns managed in an external cloud app Country is part of the customer details table, stored in the Oracle DW Sources Combine, Transform & Integrate Consume Base View Source Abstractio n join group by state join Historical Sales Campaign Customer

Performance Benchmark : no cache, no MPP Denodo has done extensive testing using queries from the standard benchmarking test TPC-DS * and the following scenario that compares the performance of a federated approach in Denodo with an MPP system where all the data has been replicated via ETL Benchmarks: Federating large data sets Customer Dim. 2 M rows Sales Facts 290 M rows Items Dim. 400 K rows * TPC-DS is the de-facto industry standard benchmark for measuring the performance of decision support solutions including, but not limited to, Big Data systems. vs. Sales Facts 290 M rows Items Dim. 400 K rows Customer Dim. 2 M rows

Performance Benchmark : no cache, no MPP Query Description Returned Rows Netezza Time Denodo Time (Federated Oracle, Netezza & SQL Server) Denodo Optimization Technique (automatically selected) Total sales by customer 1.99 M 20.9 sec. 21.4 sec. Full aggregation push-down Total sales by customer and year between 2000 and 2004 5.51 M 52.3 sec. 59.0 sec. Full aggregation push-down Total sales by item brand 31.35 K 4.7 sec. 5.0 sec. Partial aggregation push-down Total sales by item where sale price less than current list price 17.05 K 3.5 sec. 5.2 sec. On the fly data movement Benchmarks: Federating large data sets Execution times are comparable with single source executions based only on automatic optimizer decisions

Denodo mentioned on “We were very impressed when our DV specialists were able to set up the connections to the data sources, the data hashing, as well as the services exposed as an API for our Blockchain engine, all with tight security and in much less time than we thought would be needed.” http://www.itone.lu/actualites/blockchain-and-data-virtualisation-european-commission

Immediate Access Higher Impact More Agile 3-10x Faster Lower TCO Up to 75% savings “ ” Through 2020, 50% of enterprises will implement some form of data virtualization as one enterprise production option for data integration. Gartner’s Guide to Data Virtualization, Dec. 2017