introduction and use of apache_kylo.pptx

Kylo is an open source enterprise-ready data lake management software platform for self-service data ingest and data preparation with integrated metadata

1. Apache Kylo 2022-06-24  15:00 (UTC+5) Recorded by                                              Organized by Ibrar Ahmed                                  Rida Zahid

2. Apache Kylo  Ibrar Ahmed   Staff Software Engineer

3. Agenda

4. Apache Kylo Kylo is an opensource enterprise-ready data lake management software platform for self-service data ingest and data preparation with integrated governance, security management and best practices inspired by  Think Big's  150+ big data implementation projects. Kylo Service components :

5. Apache Kylo Key Features: Ingest:           Self-service data ingest with data cleansing, validation, and automatic profiling Prepare:                Wrangle data with visual sql and an interactive transform through a simple user interface. Discover:                   Search and explore data and metadata, view lineage, and profile statistics. Monitor:                Monitor health of feeds and services in the data lake. Track SLAs and troubleshoot performance. Design:                  Design batch or streaming pipeline templates in Apache NiFi and register with Kylo to enable user self-service.

6. Apache Kylo

7. Challenges faced by a Platform Team - Minimize data engineer's efforts - Provided solution must be simple/easy to use -  Shortage of experienced software engineers and administrators -   Solution must be self service

8. PowerSchool  Analytics Platform Architecture

9. Apache Kylin Demo/Training