Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)

Choosing between Microsoft Fabric, Azure Synapse Analytics and Azure Databricks (Presented at Data Left Unattended on December 7th, 2023)

2. Hook 1
You are a
data engineer

3. Hook 2
You are starting
a new project

4. Hook 3
Which tool
do you use?

5. Hook 4
Azure Databricks?
Azure Synapse Analytics?
Microsoft Fabric?

6. Hook 5
It Depends ™

7. Hook 6
What does it
depend on?

8. Hook 7
Let's take
a look!

9. Choosing between
Azure Databricks,
Azure Synapse Analytics
and Microsoft Fabric
MartheMoengen, Emilie Rønning, Cathrine Wilhelmsen

10. Cathrine
Wilhelmsen
Evidi
[email protected]
cathrinew.net
@cathrinew
Marthe
Moengen
Sopra Steria
Emilie
Rønning
Quarks
[email protected]
data-ascend.com
@mmoengen
[email protected]
linkedin.com/in/emilie-ronning
@emilieronning
Solutions Architect Managing Data AnalystSenior Data Engineer
The GOAT in Norway Beyoncé in Tech SnæckQueen

11. Azure
Databricks

12. What is Azure Databricks?
“Azure Databricks is a unified, open analytics platform for
building, deploying, sharing, and maintaining enterprise-
grade data, analytics, and AI solutions at scale”
Data SharingData WarehousingData EngineeringArtificialIntelligence
Data Science MarketplaceReal-time StreamingData Governance

13. What is Azure Databricks?
“Easily ingest and transform batch and streaming data on
the Databricks Lakehouse Platform.”
Use the lakehouse platform to store your
structured, semi-structured and unstructured
data
Write your data transformation using notebooks
as code in four pre-defined languages.
Orchestrate your workflows using low-code/ no-
code workflows
Data Engineering

14. What is Azure Databricks?
Data Warehousing
“Databricks SQL is a SQL editor and dashboarding tools,
allowing team members to collaborate with other Databricks
users directly in the workspace.”
Ad-hoc Querying
Dashboards
Scheduled Querying
Alerts based on Query output

15. Azure
Synapse
Analytics

16. What is Azure Synapse Analytics?
Unified analytics platform:
•Data Integration
•Data Lake
•Data Warehousing
•Big Data Analytics
•Time-Series Analytics
•Data Science

17. What is Azure Synapse Analytics?
Three engines:
•SQL Pools
Dedicated, Serverless
•Spark
•Data Explorer

18. What is Azure Synapse Analytics?
Pick and choose which
components to use:
•Enterprise-scale
withabigbudget?
•Single / simple solution
with a limited budget?

19. DedicatedSQL(Big budget)
Massive Parallel
Processing (MPP)
Data Warehousing:
•Distribution
•Partitioning
•Indexing
•Workloads

20. Serverless SQL(Limited budget)
Query files using SQL and
create virtual databases:
•CSV Files
•Parquet Files
•JSON Files

21. Synapse Overview

22. Microsoft
Fabric

23. What is Microsoft Fabric?
AS IS before
Fabric
Landing Zone Refined Zone Consumption
Zone
Storage Storage Storage
orchestration
compute
VM
AAS
transformations
ingestion

24. What is Microsoft Fabric?
Landing Zone Refined Zone Consumption
Zone
Storage
Storage Storage
orchestration
compute
VM
AAS
transformations
ingestion
One Lake
AS IS before
Fabric

25. What is Microsoft Fabric?
Storage
Storage StorageOne Lake
Data
Engineering
Data
Factory
Data
Ware-
housing
Real
Time
Analytics
Data
Science
Business
Intelligence
Data
Activator
AS IS before
Fabric

26. What is Microsoft Fabric?
Storage
Storage StorageOne Lake
Data
Engineering
Data
Factory
Data
Ware-
housing
Real
Time
Analytics
Data
Science
Business
Intelligence
Data
Activator
One Security
Governance and Administration

27. What is Microsoft Fabric?
Storage
Storage StorageOne Lake
Data
Engineering
Data
Factory
Data
Ware-
housing
Real
Time
Analytics
Data
Science
Business
Intelligence
Data
Activator
One Security
Governance and Administration
AI Assisted AI-Assisted

28. What is Microsoft Fabric?
Storage
Storage StorageOne Lake
Data
Engineering
Data
Factory
Data
Ware-
housing
Real
Time
Analytics
Data
Science
Business
Intelligence
Data
Activator
One Security
Governance and Administration
AI Assisted AI-Assisted
Software as a Service
(SaaS)
All-in-one analytics
solution
•Data movement
•Data storage
•Data science
•Real-Time Analytics
•Business intelligence
•Data Governance
•Data Security
•Data Warehouse
•Data Lakehouse
•Semantic model

29. Let’scompare!

30. Architectures
Where do the services shine?

31. Data Warehouse
“Centralized repository of structured
and often historical data, enabling
businesses to analyze and make
informed decisions.”

32. Data Warehouse
Data Warehouse

33. Data Warehouse

34. Data Lakehouse
“Combines the benefits of data lakes
and data warehouses, providing a
unified platform for storing, processing,
and analyzing data at scale.”

35. Data Lakehouse
Bronze Layer Silver Layer Gold Layer

36. Data Lakehouse

37. Data Mesh
“Architectural pattern for
implementing enterprise data
platforms in large and complex
organizations.”

38. Data Mesh
Self-Serve Data Platform
Federated Governance
Domain A
Domain B
Domain C
Domain E
Product
1
Product
2
Product
3
Product
4
Product
6
Product
5
Product
7

39. Data Mesh

40. ????????????????????????

41. Decisionfactors
How do youchoose?

42. Whatkindofarchitecture?
Data Warehouse
Data Lakehouse
Data Mesh

43. Who aretheusers?
Business Users
Power Users
IT Professionals

44. Whataretheirexistingskills?
No-code/ Low-code
SQL
Python

45. Pricing
How muchwillit cost?

46. Azure Databricks
Pay-as-you-go:
Engineering Compute* (Standard/Premium)
•All-Purpose ??????????????????
•Job ????????????
•Job Light ??????
Warehousing Compute* (Premium)
•SQL ??????
•SQL Pro ????????????
•Serverless ??????????????????
❗In addition to VMs, Azure Databricks will also bill
for managed, disk, blob storage, Public IP Address.
*1-year and 3-year pre-purchaseplans available!

47. Azure Synapse Analytics
Pay-as-you-go:
•Data Integration
•Spark Pools
•Data Explorer Pools
•Dedicated SQL Pools
•Serverless SQL Pools
Pre-purchaseplans available!

48. Microsoft Fabric
Azure SKUs
•Pay-per-second
•Scale / Pause / Resume
Microsoft 365 SKUs
•Pay monthly or yearly
(monthly commitment)

49. Whichonedo youchoose?
Data Intelligence
Platform
Analytics Platform
From a Technical Perspective
All-in-one Service
From a Business Perspective

50. Whichonedo youchoose?

51. Want to learn more about Fabric?
www.fabricfebruary.com
JoinusFebruary8th, 2024 • Oslo, Norway
Fabric February 2024

52. Thank you!
MartheMoengen, Emilie Rønning, Cathrine Wilhelmsen