The Zero-ETL Approach: Enhancing Data Agility and Insight

In the ever-evolving landscape of data management, Zero-ETL is an approach that is reshaping how businesses handle and integrate their data. This webinar explores Zero-ETL, a paradigm shift from the traditional Extract, Transform, Load (ETL) process, offering a more streamlined, efficient, and real-time data integration method.

We will begin with an introduction to the concept of Zero-ETL, including how it allows direct access to data in its native environment and real-time data transformation, providing up-to-date information with significantly reduced data redundancy.

Next, we'll take you through several demonstrations showing how Zero-ETL can deliver real-time data and enable the free movement of data between systems. We will also discuss the various tools that support all aspects of Zero-ETL, providing attendees with an understanding of how they can adopt this innovative approach in their organizations.

Lastly, the session will conclude with an interactive Q&A segment, allowing participants to gain deeper insights into how Zero-ETL can be tailored to their specific business needs and how they can get started today.

Join us to discover how Zero-ETL can elevate your organization's data strategy.

1. The Zero-ETL Approach:
Enhancing Data Agility
and Insights

2. Christian
Berger

Solutions Specialist


Stewart
Harper

VP of Product
Development

Don
Murray

Co-Founder

Meet our Team

3. Welcome to Livestorm.
A few ways to engage with us during the webinar:


Audio issues? Click this for 4 simple
troubleshooting steps.

4. How to download slides
1.Hover over the
slide deck in the
webinar room
2.Click this button

5. Agenda
1What is ETL, and Reverse ETL?
2What about Spatial Data and ETL?
3What is Zero-ETL?
4Real Time Data Services and the last data mile
5Connecting Applications in Real-time
6High Velocity Data Streams
7 Coming Soon: Data Virtualization
8Conclusion, Resources, Next Steps, QA
Agenda

6. Enterprise Integration.
Getting data to where it adds the most
value, when it is needed.
Extract, Transform, and Load (ETL) is one way to
describe the technology to do this.

7. 1
What is ETL, and
Reverse ETL?

8. On-Premises
ETL = Extract data from
systems, Transform, and
Load into Data stores.

9. ETL is critical for businesses, combining data
from multiple sources, to provide a consistent
and unified view for reporting and analysis.

10. On-Premises
ETL = Extract data from
systems, Transform, and
Load into Data stores.
Reverse ETL = Extract data
from data stores, Transform, and
Load into systems.

11. Reverse ETL operationalizes insights, making
the data actionable in workflows and tools that
support customers, manage operations, and drive
business growth.

12. Any type of ETL is
●getting and combining data,
●from where it is,
●to where you need it,
●when you need it,
●how you need it!

This data movement is what the
FME Platform is all about.

13. 2
What about
Spatial Data
and ETL?

14. The world we live in is
Spatial.


How many ways have you used spatial today without even
thinking about it?
Yet most businesses don’t use it to gain new insights!

15. Spatial Data is critical to get from
digital optimization to digital transformation.
●Spatial data reveals new relationships and insights to make
better decisions.
●Without spatial data you miss opportunities, allocate
resources poorly, and fall behind your competitors.
●Spatial Data is critical for all types of ETL and data
movement.

16. Unrivalled Data Type support
Spatial Data is
Everywhere

17. Support for
Spatial Data
Competitor

18. 3
What is
Zero-ETL?

19. On-Premises
Zero-ETL
●Direct Access to
current data and
services.
●Real-time
Synchronization of
data across systems.
●Flexibility and Agility:
Data flows across all
systems and data
stores in Real-Time.
Real-Time Data
Movement

20. Zero-ETL delivers real-time data management
and analysis so organizations can respond
quickly to needs and opportunities.

21. One platform, two technologies
FME Form FME Flow
Data Movement and transformations
(“ETL”) workflows are built here.
Brings life to FME Form workflows
FME Flow Hosted
Safe Software managed FME Flow
fme.safe.com/platform
FME Enterprise Integration Platform
Safe & FME

22. FME Flow delivers Zero-ETL

23. 4
Real-Time data
services and the last
data mile

24. 1 2 3
Unlimited, Managed
Users
-User/Role permissions
-Authentication Services
(AD, SAML)
Flexible Engine
Deployment
-Deploy Engines Anywhere
-CPU-Based Pricing
-Remote Engines
Advanced Engine
Management
-Queues & Job Routing
-Active Periods
-Engine Assignment

25. Data Streaming Workspace Flow Apps
User uploads file
to FME Flow
Workspace App

26. Data Streaming Workspace Flow Apps

User uploads file
to FME Flow
Workspace App
FME Flow Engines
DataValidation.fmw
executes on next
available engine
Invalid Feature
HTML Report
Submission ID
and Timestamp

27. Data Streaming Workspace Flow Apps
User uploads file
to FME Flow
Workspace App
Job Queue
DataValidation.fmw
routed to a specified
Queue
Invalid Feature
HTML Report
Submission ID
and Timestamp

28. Data Streaming Workspace Flow Apps

Invalid Feature
HTML Report
Submission ID
and Timestamp

30. Data Download Workspace Flow Apps

Job Queue
DataDownload.fmw
routed to a specified
Queue
User Selects
Area of Interest
to Retrieve
Feature(s) from
Job output
provided to client
as downloadable
.zip file

32. Customer Story
City of Henderson
Project
Digital Plan Submission

Solution
FME provides self-serve CAD data upload, validation, and
integration into GIS.

Results
●Reduced data integration time by 75%.
●Eliminated the risk of manual error.
●Removed duplicate work.
●Faster processing of submissions.
●Ensured CAD data validation.
“We chose FME for this project because the interface is easy to use and
allows us to provide an adaptable solution to enhance interaction with the
City’s partners.”
- Erika Provost, City of Henderson

33. 5
Connecting
Applications in
Real-Time

34. FME Flow Automations
Connect different systems so actions in one system update another
●Powerful trigger mechanism via webhooks, data arrival, database
triggers, etc.
●Build multi-step complex workflows
●Take advantage of true parallelism.
●Full orchestration control via split merge blocks.
●Reduce errors and save time by replacing manual tasks with
automation.

35. Uni-Directional
Integration Architecture
Linear data flow used to repair, validate, and replicate data
from system A to system B
●Triggers can vary from files being received to schedules
and webhooks.


Dynamic data flow(s) used to integrate multiple systems by
synchronizing changes between 2 (or more) systems.
●Typically driven by an action being performed in either
system for near real time processing via webhooks or
topics/notifications.

Bi | Multi-Directional

36. SFTP
Dropbox
System Event
Azure Event
Grid
JMS
UDP
HTTP Request WebSocket
FTP
Logger
SQS/SNS SMTPSQS/SNS
Filter
Merge
S3 Bucket
Network
Triggers and
Notifications

37. FME Flow
Automation Triggered
(Webhook Received
from Survey123)
User submits Survey
Integrate AGOL and Cityworks via REST API
Published FME Workspace (.fmw)

38. FME Flow
Automation Triggered
Record inserted, updated, or deletedRecord inserted, updated, or deleted
Published FME Workspace executes and synchronizes
changes to between ArcGIS Online Features & Cityworks Work Orders

39. FME Flow
Automation Triggered
Record inserted, updated, or deleted Record inserted, updated, or deleted
Published FME Workspace executes and synchronizes
changes to features from Cityworks, Salesforce, and
ArcGIS Online

40. Demo

42. Customer Story
City of Naperville
Project
Automating time-consuming processes for their Electric,
Public Works, Finance, and Water departments.

Solution
Integrating FME with Cityworks to automate service requests
and work orders.

Results
●3300+ service requests and work orders automated,
saving hours of manual effort.
●Three staff with specialized coding skills free to focus
on high-value tasks rather than manual work
●Increased collaboration and stronger relationships
between departments
“With FME, we were suddenly saying “yes!” to other departments and
implementing requests quickly. Now, we’re viewed differently in our
interactions. FME makes it easy for us to improve our relationships with
other departments by helping improve their business processes.”
- Danielle DuBose, Database Developer, City of Naperville

43. 6
High Velocity Data
Streams

44. Let’s start with the data
●Bounded data is finite and has a discrete beginning and end. It is associated with batch processing.
●Unbounded data is infinite, having no discrete beginning or end. It is associated with stream processing.
●FME handles both!

45. Use Cases
1.Business Applications - Essential for driving real-time business intelligence and process automation.
e.g. Customer orders, airline reservations, insurance claims, bank transactions, telco call detail records (CDR).
2.Digital Information - Captures the dynamics of online engagement and system health.
e.g. clickstreams, logs, market data, email.
3.Internet of Things (IoT) - Capture the vast and growing network of connected devices, translating real-time data from
the physical world into actionable intelligence.
e.g RFID, telemetry SCADA, GPS sensors, smart meters, smart city infrastructure.

46. Supported IoT &
Messaging Protocols

47. Simple Streaming Pattern: Filter by Attribute
Reduce data volumes by saving only what you are interested in.

48. FME Flow Streams
Process data continuously with low latency.

49. 7
Coming Soon:
Data
Virtualization

50. Data Virtualization
Create Open API compliant APIs on
top of all FME Data Types.

Real-time data (Zero-ETL) to deliver
direct data access via web API.

Effectively making any data cloud
native.

51. Data Virtualization Value
Agility
●Unified view isolates applications from underlying systems.
Cost Savings
●Simplify infrastructure. Save time and resources on data integration.
Improved Decision Making
●Real-time access to enterprise applications used by decision makers.
Enhanced Data Security and Compliance
●Data access via a secure API improves data security and governance.
Increased Productivity
●Frees IT to focus on strategic projects; business applications access data directly.

52. Data Virtualization
ConnectsAnything
Caching.
We don’t want to run any job unless
needed.
Security.
controlling who and type of access
they have.
Monitoring
Who is using what how much. Rate
limiting, etc.
FME Flow

53. FME Flow 2024.x

54. FME Form Workbench - Data Virtualization

55. 8
Conclusion

56. 29+
27K+
128
190
20K+
years of solving data
challenges
FME Community
members
countries with
FME customers
organizations worldwide
global partners with
FME services
30+
29K+
128
140+
25K+
years of solving data
challenges
FME Community
members
countries with
FME customers
organizations worldwide
global partners with
FME services
200K+
users worldwide
Safe & FME

57. Resources

58. Get our Ebook
Spatial Data for the
Enterprise

fme.ly/gzc


Guided learning
experiences at your
fingertips
academy.safe.com


FME Academy
Resources
Check out how-to’s &
demos in the knowledge
base
community.safe.com
/s/knowledge-base
Knowledge Base Webinars
Upcoming &
on-demand webinars

safe.com/webinars

59. Next Steps

60. We’d love to help you get
started.
Get in touch with us at
[email protected]
Experience the
FME Accelerator
Contact Us
A world where data is not just a
commodity but a catalyst for
real change.
fme.safe.com/accelerator



Next Steps

61. Claim Your Community Badge &
Dive into the new Community!
●Get community badges for watching
webinars
●community.safe.com
●Today’s code: C35A62

Join the Community today!

62. Q&A

63. Thank You
Recap of Next Steps

1Join the FME Community
2Contact us
3Experience the FME Accelerator

Please fill out our
webinar survey