Data Streaming and Stream management system

This presentation include the point about Data Stream. its proper discerption and finest way of understanding the topics with real world business use case.
Also content includes Management System for that part to enhanced the scalability of process.

1. Introduction to Data Streams and
Stream Management Systems
SHAIKH RIZWAN ASRAR
190105231018
B.TECH AIML B.E
ADVANCE DATABASES
GUIDED BY: DR. PRASANNA KAPSE

2. Definition: Data streams are continuous and high-volume flows of data that arrive
sequentially or in real-time.
Examples: Social media feeds, sensor data from IoT devices, stock market data, web server
logs.
What are Data Streams?

3. Continuity: Data streams are continuous and never-ending.
High Volume and Velocity: Streams can generate large amounts of data at high speeds.
Sequentiality: Data arrives in an ordered sequence and must be processed in real-time.
Time-Sensitive: Analysis and decision-making must be performed in near-real-time.
1.
2.
3.
4.
Characteristics of Data
Streams

4. Data Volume and Velocity: Handling and processing large volumes of data at high speeds.
Real-time Processing: Analyzing and extracting insights from data as it arrives.
Limited Resources: Managing and allocating system resources efficiently.
Data Quality: Dealing with noisy or incomplete data in real-time.
Scalability: Ensuring the system can handle increasing data volume and stream complexity.
1.
2.
3.
4.
5.
Challenges in Managing Data Streams

5. Definition: A Stream Management System is a software framework or platform that
handles the challenges of processing and managing data streams.
Purpose: Collect, process, analyze, and store data streams efficiently in real-time.
What is a Stream Management
System (SMS)?

6. Stream Ingestion: Ability to receive and collect data streams from various sources.
Stream Processing: Real-time analysis and computation on the incoming data.
Stream Querying: Capability to query and retrieve specific information from the stream.
Stream Storage: Efficient storage and retrieval of data streams.
Stream Integration: Integration with external systems and databases.
Fault Tolerance: Resilience to failures and ability to recover from errors.
1.
2.
3.
4.
5.
6.
Key Features of Stream Management
Systems

7. Stream processing can be done using various architectures, including:
Event-driven architectures (EDA)
Message queueing systems (MQS)
Complex event processing (CEP)
Lambda architectures
Microservices-based architectures
1.
2.
3.
4.
5.
Stream Processing Architectures

8. Fraud Detection: Real-time monitoring of transactions for suspicious activities.
Predictive Analytics: Analyzing streaming data to make predictions and recommendations.
IoT Data Processing: Handling and processing sensor data from IoT devices.
Social Media Monitoring: Analyzing social media feeds for sentiment analysis and trending
topics.
Network Monitoring: Real-time analysis of network traffic for security and performance
monitoring.
1.
2.
3.
4.
5.
Use Cases of Stream Management
Systems

9. Mention some popular SMS platforms:
Apache Kafka
Apache Flink
Apache Samza
Amazon Kinesis
Google Cloud Pub/Sub
Microsoft Azure Stream Analytics
1.
2.
3.
4.
5.
6.
Popular Stream Management Systems

10. Data streams are continuous flows of data that require specialized management systems.
Stream management systems provide capabilities for real-time processing, analysis, and
storage of data streams.
SMS platforms play a crucial role in various domains like finance, IoT, social media, and
more.
Conclusion

11. Thank you!