Big data Scratch The Surface

Mladen Antunovic - Senior Manager Big Data @ Mapp Digital GmbH

1. Big data
Scratch the surface

2. 2
About me (https://www.linkedin.com/in/antunovic/)
Mladen Antunović
Sr. Manager, Big Data Engineering –Mapp Digital GmbH München
Work
dipl.el.ing., Faculty of Electrical Engineering. University of Sarajevo
Education
AtlantBHSarajevo (2008 -2013) –Lead Engineer
Mapp Digital (2013 -Current) –Sr. Manager, Big Data Engineering
Experience

3. 3
What am I talking about?

4. 4
What is Big data?
“Today, every two days we create as much
data as we did from the beginning of time until
2000.”
That’s right, every two days

5. 5
Usage
Predictions
Simulations
Analytics
Artificial Intelligence
Machine Learning
Statistics

6. 6

7. 7
Basic App
Architecture
7

8. 8
Distributed App
Architecture
8
831
065
23

9. 9
Distributed platform architecture
Masters
Workers
Lookup (ZK)

10. 10
Big Data = Big Problems
10
Costs
Performance
Scalability
High Availability (HA)
Maintenance (Operation)
Migration (Deployments, Upgrades, UPTIME)

11. @Mapp

12. 12
Big Data Infrastructure
12
Self managed (2 Datacenters in Munich)
Open Source
No paid support
~600 Servers (Dell R4xx, R6xx, R8xx)
1 -10 Gbps Network
Private cloud + Docker
3 people (Automated)

13. 13

14. 14
Planning
14
Software (OS, Big Data software)
# of servers / # of racks
CPUs
Memory
Storage (SATA, SSD)
Network
Application architecture

15. 15
Maintenance
15
Uptime SLAs
Hardware replacements
Software upgrades (before | now)
Migration
Network upgrade
Application architecture

16. 16
Improvements
16
Monitoring
Log analysis
Metrics collection
Logging
Alerting
Machine Learning
Artificial Inteligence

17. 17
Public vs Private Cloud
17
Costs
Scalability
Maintenance (failures, backups, replication, recovery)
Migration
Project (Phase, External/Internal)

18. 18
Life @Mapp

19. 1919
Q&A
Should we use Big Data technologies for every project?
Why?
Why not?