HTTP Adaptive Streaming – Quo Vadis (2024)
christian.timmerer
1,197 views
46 slides
Jul 01, 2024
Slide 1 of 46
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
About This Presentation
Video traffic on the Internet is constantly growing; networked multimedia applications consume a predominant share of the available Internet bandwidth. A major technical breakthrough and enabler in multimedia systems research and of industrial networked multimedia services certainly was the HTTP Ada...
Slide Content
HTTP Adaptive Streaming – Quo Vadis?
Christian Timmerer, Professor at AAU, Director at CD Lab ATHENA
Hermann Hellwagner, Professor at AAU
Klagenfurt, Austria
June 27, 2024
1
Change log:
● DSRC Telecom Seminar Series’24
● FEEC/UNICAMP Seminar in Comp.
Eng.’23
● IEEE MMTC DL Series’23
● PCS’21
● DDRC’21
● ISM’20
● WebMedia’20
https://www.slideshare.net/christian.timmerer
Christian Timmerer, Professor at AAU, Director at CD Lab ATHENA
Hermann Hellwagner, Professor at AAU
Klagenfurt, Austria
June 27, 2024
1
Change log:
● DSRC Telecom Seminar Series’24
● FEEC/UNICAMP Seminar in Comp.
Eng.’23
● IEEE MMTC DL Series’23
● PCS’21
● DDRC’21
● ISM’20
● WebMedia’20
https://www.slideshare.net/christian.timmerer
The number of transistors in an integrated circuit
doubles about every two years – Moore's Law
(https://en.wikipedia.org/wiki/Moore%27s_law)
Users' bandwidth grows by 50% per year (10% less
than Moore's Law for computer speed) – Nielsen's
Law of Internet Bandwidth
(https://www.nngroup.com/articles/law-of-bandwidth/)
Video accounts for more than 65% of the global
Internet traffic – Sandvine Global Internet
Phenomena (January 2023, https://www.sandvine.com/phenomena)
2
doubles about every two years – Moore's Law
(https://en.wikipedia.org/wiki/Moore%27s_law)
Users' bandwidth grows by 50% per year (10% less
than Moore's Law for computer speed) – Nielsen's
Law of Internet Bandwidth
(https://www.nngroup.com/articles/law-of-bandwidth/)
Video accounts for more than 65% of the global
Internet traffic – Sandvine Global Internet
Phenomena (January 2023, https://www.sandvine.com/phenomena)
2
Presenter
Christian Timmerer
Univ.-Prof. at Alpen-Adria-Universität Klagenfurt
Director CD Lab ATHENA
CIO | Head of Research and Standardization at Bitmovin
3
3
2003: MSc CS (Dipl.-Ing.)
2006: PhD CS (Dr.-techn.)
2012: Co-founded Bitmovin
2014: Habilitation (Priv.-Doz.) & Assoc. Prof.
2016: Dep. Director @ ITEC/AAU
2019: Director @ ATHENA
2022: Univ.-Prof. for Multimedia Systems
Web: http://timmerer.com/
Bitmovin MPEG-DASH
Christian Timmerer
Univ.-Prof. at Alpen-Adria-Universität Klagenfurt
Director CD Lab ATHENA
CIO | Head of Research and Standardization at Bitmovin
3
3
2003: MSc CS (Dipl.-Ing.)
2006: PhD CS (Dr.-techn.)
2012: Co-founded Bitmovin
2014: Habilitation (Priv.-Doz.) & Assoc. Prof.
2016: Dep. Director @ ITEC/AAU
2019: Director @ ATHENA
2022: Univ.-Prof. for Multimedia Systems
Web: http://timmerer.com/
Bitmovin MPEG-DASH
4
4
My offices are here
Copyright: AAU/Steinthaler
4
My offices are here
Copyright: AAU/Steinthaler
●Introduction
●ATHENA
○Content Provisioning
○Content Delivery
○Content Consumption
○End-to-End Aspects
○Quality of Experience
●Conclusions: HAS – Quo Vadis?
Agenda
5
●ATHENA
○Content Provisioning
○Content Delivery
○Content Consumption
○End-to-End Aspects
○Quality of Experience
●Conclusions: HAS – Quo Vadis?
Agenda
5
Introduction / Motivation
Sources:
* Sandvine Global Internet Phenomena (January 2024).
** Cisco Annual Internet Report (2018–2023) White Paper (March 2020)
6
6
Video streaming is dominating today’s
Internet traffic
●2024*: 68% (fixed) and 64% (mobile)
●Video on-demand: 54% / 57% – Live: 14% / 7%
●Main applications: YouTube, Netflix (>10%),
Tik Tok, Amazon Prime, Disney+ (<10%)
●Video and other applications continue to be of
enormous demand in today’s home, but there will
be significant bandwidth demands with the
application requirements of the future**
Sources:
* Sandvine Global Internet Phenomena (January 2024).
** Cisco Annual Internet Report (2018–2023) White Paper (March 2020)
6
6
Video streaming is dominating today’s
Internet traffic
●2024*: 68% (fixed) and 64% (mobile)
●Video on-demand: 54% / 57% – Live: 14% / 7%
●Main applications: YouTube, Netflix (>10%),
Tik Tok, Amazon Prime, Disney+ (<10%)
●Video and other applications continue to be of
enormous demand in today’s home, but there will
be significant bandwidth demands with the
application requirements of the future**
HTTP Adaptive Streaming 101
Adaptation logic is within the
client, not normatively specified
by a standard, subject to
research and development
7
7
Client
Adaptation logic is within the
client, not normatively specified
by a standard, subject to
research and development
7
7
Client
Multimedia Systems Challenges and Tradeoffs
8
8
Basic figure by Klara Nahrstedt, University of Illinois at Urbana–Champaign, IEEE MIPR 2018
8
8
Basic figure by Klara Nahrstedt, University of Illinois at Urbana–Champaign, IEEE MIPR 2018
“Application-oriented basic research” to address current and future research
and deployment challenges of HAS and emerging streaming methods
ATHENA – Adaptive Streaming over HTTP and
Emerging Networked Multimedia Services
Content Provisioning Content Delivery Content Consumption
End-to-End Aspects
●Video encoding for HAS
●Quality-aware encoding
●Learning-based encoding
●Multi-codec HAS
●Edge computing
●Information CDN/SDN ⇿clients
●Netw. assistance for/by clients
●Utility evaluation
●Bitrate adaptation schemes
●Playback improvements
●Context and user awareness
●Quality of Experience (QoE) studies
●Application/transport layer enhancements
●Quality of Experience (QoE) models
●Low-latency HAS
●Learning-based HAS
https://athena.itec.aau.at/
9
9
Funding:
and deployment challenges of HAS and emerging streaming methods
ATHENA – Adaptive Streaming over HTTP and
Emerging Networked Multimedia Services
Content Provisioning Content Delivery Content Consumption
End-to-End Aspects
●Video encoding for HAS
●Quality-aware encoding
●Learning-based encoding
●Multi-codec HAS
●Edge computing
●Information CDN/SDN ⇿clients
●Netw. assistance for/by clients
●Utility evaluation
●Bitrate adaptation schemes
●Playback improvements
●Context and user awareness
●Quality of Experience (QoE) studies
●Application/transport layer enhancements
●Quality of Experience (QoE) models
●Low-latency HAS
●Learning-based HAS
https://athena.itec.aau.at/
9
9
Funding:
Video coding for HTTP Adaptive Streaming
●Quality improvement
Per-Title Encoding (PTE) et al. for live use cases: video complexity analysis,
per-title/-scene/-shot/-segment, content-/context-aware, content-adaptive,
quality-aware encoding
●Runtime improvement
Hardware-/software-based (cloud), parallel/distributed, information reuse from
reference encodings (multi-rate/-resolution) ⇨ cf. earlier versions of this talk
●Application scenarios
Video on Demand (VoD incl. diff. flavors AVoD, SVoD), live (incl. diff. flavors),
interactive, games, video conferencing
Content Provisioning
10
10
●Quality improvement
Per-Title Encoding (PTE) et al. for live use cases: video complexity analysis,
per-title/-scene/-shot/-segment, content-/context-aware, content-adaptive,
quality-aware encoding
●Runtime improvement
Hardware-/software-based (cloud), parallel/distributed, information reuse from
reference encodings (multi-rate/-resolution) ⇨ cf. earlier versions of this talk
●Application scenarios
Video on Demand (VoD incl. diff. flavors AVoD, SVoD), live (incl. diff. flavors),
interactive, games, video conferencing
Content Provisioning
10
10
Problem Statement / Basic Idea for Live Streaming
●Static bitrate ladder
Pre-defined bitrate/resolution pairs for all contents
●Per-title encoding
Optimize bitrate ladder based on the content
●UHD / HFR content
Increase of spatial and temporal resolutions
●Live (is Life)[* Opus, 1985]
Perceptual-based bitrate/resolution/framerate
prediction algorithm based on content complexity
Video Complexity Analyzer (VCA)
11
11
VoD
Live
* … https://www.youtube.com/watch?v=pATX-lV0VFk
Vignesh V Menon, Christian Feldmann, Hadi Amirpour, Mohammad Ghanbari, and Christian Timmerer. 2022.
VCA: video complexity analyzer. In Proceedings of the 13th ACM Multimedia Systems Conference (MMSys '22). Association
for Computing Machinery, New York, NY, USA, 259–264. https://athena.itec.aau.at/2022/04/vca-video-complexity-analyzer/
●Static bitrate ladder
Pre-defined bitrate/resolution pairs for all contents
●Per-title encoding
Optimize bitrate ladder based on the content
●UHD / HFR content
Increase of spatial and temporal resolutions
●Live (is Life)[* Opus, 1985]
Perceptual-based bitrate/resolution/framerate
prediction algorithm based on content complexity
Video Complexity Analyzer (VCA)
11
11
VoD
Live
* … https://www.youtube.com/watch?v=pATX-lV0VFk
Vignesh V Menon, Christian Feldmann, Hadi Amirpour, Mohammad Ghanbari, and Christian Timmerer. 2022.
VCA: video complexity analyzer. In Proceedings of the 13th ACM Multimedia Systems Conference (MMSys '22). Association
for Computing Machinery, New York, NY, USA, 259–264. https://athena.itec.aau.at/2022/04/vca-video-complexity-analyzer/
Spatial (E) / Temporal (h) Content Characteristics
Video Complexity Analyzer (VCA)
12
12 *) Michael King, Zinovi Tauber, and Ze-Nian Li. 2007. A New Energy Function for Segmentation and Compression. In 2007
IEEE International Conference on Multimedia and Expo. 1647–1650. https://doi.org/10.1109/ICME.2007.4284983
*)
VCA 2.0 (seven features): the average
●luma texture energy E
Y
●gradient of the luma texture energy h
Y
●luma brightness L
Y
●chroma brightness (L
U
and L
V
)
●chroma texture energy (E
U
and E
V
)
https://athena.itec.aau.at/2023/02/vca-v2-0-released/
Video Complexity Analyzer (VCA)
12
12 *) Michael King, Zinovi Tauber, and Ze-Nian Li. 2007. A New Energy Function for Segmentation and Compression. In 2007
IEEE International Conference on Multimedia and Expo. 1647–1650. https://doi.org/10.1109/ICME.2007.4284983
*)
VCA 2.0 (seven features): the average
●luma texture energy E
Y
●gradient of the luma texture energy h
Y
●luma brightness L
Y
●chroma brightness (L
U
and L
V
)
●chroma texture energy (E
U
and E
V
)
https://athena.itec.aau.at/2023/02/vca-v2-0-released/
VCA 2.0 Latest Results
13
13
Vignesh V Menon, Christian Feldmann, Klaus Schoeffmann, Mohammad Ghanbari, Christian Timmerer,
"Green video complexity analysis for efficient encoding in Adaptive Video Streaming, " ACM Green
Multimedia Systems 2023, June 2023.
https://athena.itec.aau.at/2023/04/green-video-complexity-analysis-for-efficient-encoding-in-adaptive-video-streaming/
13
13
Vignesh V Menon, Christian Feldmann, Klaus Schoeffmann, Mohammad Ghanbari, Christian Timmerer,
"Green video complexity analysis for efficient encoding in Adaptive Video Streaming, " ACM Green
Multimedia Systems 2023, June 2023.
https://athena.itec.aau.at/2023/04/green-video-complexity-analysis-for-efficient-encoding-in-adaptive-video-streaming/
Live Per-Title Encoding Scheme
14
14
VCA: Video Complexity Analyzer (to extract/calculate E/h metrics)
Github: https://github.com/cd-athena/VCA
Documentation: https://cd-athena.github.io/VCA/
14
14
VCA: Video Complexity Analyzer (to extract/calculate E/h metrics)
Github: https://github.com/cd-athena/VCA
Documentation: https://cd-athena.github.io/VCA/
Live Per-Title Encoding: Example Bitrate Ladder
15
15
VCA: Video Complexity Analyzer (to extract/calculate E/h metrics)
Github: https://github.com/cd-athena/VCA
Documentation: https://cd-athena.github.io/VCA/
15
15
VCA: Video Complexity Analyzer (to extract/calculate E/h metrics)
Github: https://github.com/cd-athena/VCA
Documentation: https://cd-athena.github.io/VCA/
Live Per-Title Encoding: Example Frames (1)
16
16
16
16
Live Per-Title Encoding: Example Frames (2)
17
17
17
17
Results: Online Per-Title Encoding (OPTE)
18
18
V. V. Menon, H. Amirpour, M. Ghanbari and C. Timmerer, "OPTE: Online Per-Title Encoding for Live
Video Streaming," ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP), Singapore, Singapore, 2022, pp. 1865-1869, doi: 10.1109/ICASSP43922.2022.9746745.
https://athena.itec.aau.at/2022/01/opte-online-per-title-encoding-for-live-video-streaming/
18
18
V. V. Menon, H. Amirpour, M. Ghanbari and C. Timmerer, "OPTE: Online Per-Title Encoding for Live
Video Streaming," ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP), Singapore, Singapore, 2022, pp. 1865-1869, doi: 10.1109/ICASSP43922.2022.9746745.
https://athena.itec.aau.at/2022/01/opte-online-per-title-encoding-for-live-video-streaming/
Perceptually-aware Per-title Encoding (PPTE)
19
19
The minimum visual difference that
can be perceived by HVS, i.e., the
difference between two adjacent
perceptual distortion levels, refers as to
one Just Noticeable Difference (JND)
V. V. Menon, H. Amirpour, M. Ghanbari and C. Timmerer, "Perceptually-Aware Per-Title Encoding for
Adaptive Video Streaming," 2022 IEEE International Conference on Multimedia and Expo (ICME), Taipei,
Taiwan, 2022, pp. 1-6, doi: 10.1109/ICME52920.2022.9859744.
https://athena.itec.aau.at/2022/05/perceptually-aware-per-title-encoding-for-adaptive-video-streaming/
19
19
The minimum visual difference that
can be perceived by HVS, i.e., the
difference between two adjacent
perceptual distortion levels, refers as to
one Just Noticeable Difference (JND)
V. V. Menon, H. Amirpour, M. Ghanbari and C. Timmerer, "Perceptually-Aware Per-Title Encoding for
Adaptive Video Streaming," 2022 IEEE International Conference on Multimedia and Expo (ICME), Taipei,
Taiwan, 2022, pp. 1-6, doi: 10.1109/ICME52920.2022.9859744.
https://athena.itec.aau.at/2022/05/perceptually-aware-per-title-encoding-for-adaptive-video-streaming/
Results: Perceptually-aware Per-title Encoding (PPTE)
20
20
V. V. Menon, H. Amirpour, M. Ghanbari and C. Timmerer, "Perceptually-Aware Per-Title Encoding for
Adaptive Video Streaming," 2022 IEEE International Conference on Multimedia and Expo (ICME), Taipei,
Taiwan, 2022, pp. 1-6, doi: 10.1109/ICME52920.2022.9859744.
https://athena.itec.aau.at/2022/05/perceptually-aware-per-title-encoding-for-adaptive-video-streaming/
20
20
V. V. Menon, H. Amirpour, M. Ghanbari and C. Timmerer, "Perceptually-Aware Per-Title Encoding for
Adaptive Video Streaming," 2022 IEEE International Conference on Multimedia and Expo (ICME), Taipei,
Taiwan, 2022, pp. 1-6, doi: 10.1109/ICME52920.2022.9859744.
https://athena.itec.aau.at/2022/05/perceptually-aware-per-title-encoding-for-adaptive-video-streaming/
JND-Aware Two-Pass Per-Title Encoding Scheme for Adaptive Live Streaming (JTPS)
21
21
V. V. Menon, P. T. Rajendran, C. Feldmann, K. Schoeffmann, M. Ghanbari and C. Timmerer, "JND-Aware
Two-Pass Per-Title Encoding Scheme for Adaptive Live Streaming," in IEEE Transactions on Circuits and
Systems for Video Technology, vol. 34, no. 2, pp. 1281-1294, Feb. 2024, doi: 10.1109/TCSVT.2023.3290725.
https://athena.itec.aau.at/2023/06/jnd-aware-two-pass-per-title-encoding-scheme-for-adaptive-live-streaming/
21
21
V. V. Menon, P. T. Rajendran, C. Feldmann, K. Schoeffmann, M. Ghanbari and C. Timmerer, "JND-Aware
Two-Pass Per-Title Encoding Scheme for Adaptive Live Streaming," in IEEE Transactions on Circuits and
Systems for Video Technology, vol. 34, no. 2, pp. 1281-1294, Feb. 2024, doi: 10.1109/TCSVT.2023.3290725.
https://athena.itec.aau.at/2023/06/jnd-aware-two-pass-per-title-encoding-scheme-for-adaptive-live-streaming/
●Efficient Content-Adaptive Feature-based Shot Detection for HTTP Adaptive Streaming
(IEEE ICIP 2021)
●INCEPT: INTRA CU Depth Prediction for HEVC (IEEE MMSP 2021)
●CODA: Content-aware Frame Dropping Algorithm for High Frame-rate Video Streaming
(DCC 2022)
●OPTE: Online Per-title Encoding for Live Video Streaming (IEEE ICASSP 2022)
●Live-PSTR: Live Per-title Encoding for Ultra HD Adaptive Streaming (NAB BEITC 2022)
●Perceptually-aware Per-title Encoding for Adaptive Video Streaming (IEEE ICME 2023)
●Light-weight Video Encoding Complexity Prediction using Spatio Temporal Features (IEEE MMSP
2023)
●ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming (IEEE ICIP 2023)
●Content-adaptive Encoder Preset Prediction for Adaptive Live Streaming (PCS 2023)
●Transcoding Quality Prediction for Adaptive Video Streaming (ACM MHV 2023)
●Green video complexity analysis for efficient encoding in Adaptive Video Streaming
(ACM GMSys 2023)
●JND-aware Two-pass Per-title Encoding Scheme for Adaptive Live Streaming (IEEE TCVST 2023)
●Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Streaming (VCIP 2023)
VCA-based Applications (selection)
22
22
(IEEE ICIP 2021)
●INCEPT: INTRA CU Depth Prediction for HEVC (IEEE MMSP 2021)
●CODA: Content-aware Frame Dropping Algorithm for High Frame-rate Video Streaming
(DCC 2022)
●OPTE: Online Per-title Encoding for Live Video Streaming (IEEE ICASSP 2022)
●Live-PSTR: Live Per-title Encoding for Ultra HD Adaptive Streaming (NAB BEITC 2022)
●Perceptually-aware Per-title Encoding for Adaptive Video Streaming (IEEE ICME 2023)
●Light-weight Video Encoding Complexity Prediction using Spatio Temporal Features (IEEE MMSP
2023)
●ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming (IEEE ICIP 2023)
●Content-adaptive Encoder Preset Prediction for Adaptive Live Streaming (PCS 2023)
●Transcoding Quality Prediction for Adaptive Video Streaming (ACM MHV 2023)
●Green video complexity analysis for efficient encoding in Adaptive Video Streaming
(ACM GMSys 2023)
●JND-aware Two-pass Per-title Encoding Scheme for Adaptive Live Streaming (IEEE TCVST 2023)
●Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Streaming (VCIP 2023)
VCA-based Applications (selection)
22
22
Network assistance for HTTP Adaptive Streaming
●Edge computing support (at CDN / cellular network edge)
Functions at (or, assisted by) the edge: adaptation, analytics, (pre-)fetching,
caching, transcoding, repackaging of content, request aggregation
●Server/network/CDN ↔ HAS client information exchange and collaboration
IETF ALTO, MPEG SAND, MPEG NBMP, …; SDN-DASH, SDN-HAS, SABR, ...
●Use of modern network architecture features
SW Defined Networking (SDN); Network Function Virtual. (NFV); MC-ABR
Content Delivery
23
23
●Edge computing support (at CDN / cellular network edge)
Functions at (or, assisted by) the edge: adaptation, analytics, (pre-)fetching,
caching, transcoding, repackaging of content, request aggregation
●Server/network/CDN ↔ HAS client information exchange and collaboration
IETF ALTO, MPEG SAND, MPEG NBMP, …; SDN-DASH, SDN-HAS, SABR, ...
●Use of modern network architecture features
SW Defined Networking (SDN); Network Function Virtual. (NFV); MC-ABR
Content Delivery
23
23
Different policies/metrics/resource utilization
●Prefetching based on last segment quality
○Last segment quality (LSQ)
○Last segment quality plus (LSQ+)
○All segment qualities (ASQ)
●Prefetching based on a Markov Model
●Prefetching based on transrating
●Prefetching Based on machine Learning (buffer
size, link bitrate, prev. quality, prev. link bitrate;
Random Forest, Gradient Boost, AdaBoost,
Decision Trees and Extremely Randomized Trees)
●Prefetching based on super resolution
Segment Prefetching and Caching at the Edge (SPACE)
24
24
J. Aguilar-Armijo, C. Timmerer and H. Hellwagner, "SPACE: Segment Prefetching and Caching at
the Edge for Adaptive Video Streaming," in IEEE Access, vol. 11, pp. 21783-21798, 2023
https://athena.itec.aau.at/2023/02/space-segment-prefetching-and-caching-at-the-edge-for-adaptive-video-streaming/
●Prefetching based on last segment quality
○Last segment quality (LSQ)
○Last segment quality plus (LSQ+)
○All segment qualities (ASQ)
●Prefetching based on a Markov Model
●Prefetching based on transrating
●Prefetching Based on machine Learning (buffer
size, link bitrate, prev. quality, prev. link bitrate;
Random Forest, Gradient Boost, AdaBoost,
Decision Trees and Extremely Randomized Trees)
●Prefetching based on super resolution
Segment Prefetching and Caching at the Edge (SPACE)
24
24
J. Aguilar-Armijo, C. Timmerer and H. Hellwagner, "SPACE: Segment Prefetching and Caching at
the Edge for Adaptive Video Streaming," in IEEE Access, vol. 11, pp. 21783-21798, 2023
https://athena.itec.aau.at/2023/02/space-segment-prefetching-and-caching-at-the-edge-for-adaptive-video-streaming/
The best segment prefetching policy depends on the service provider’s
preferences and resources
●Straightforward implementation with
low resource utilization
⇨ LSQ and Markov-based
●Prioritize QoE at the expense of costs
⇨ ASQ, LSQ+ and Transrating-based
●Balance between performance and costs
⇨ ML-based
●Possible next steps: dynamically adapt;
premium clients
Segment Prefetching and Caching at the Edge (SPACE)
25
25
preferences and resources
●Straightforward implementation with
low resource utilization
⇨ LSQ and Markov-based
●Prioritize QoE at the expense of costs
⇨ ASQ, LSQ+ and Transrating-based
●Balance between performance and costs
⇨ ML-based
●Possible next steps: dynamically adapt;
premium clients
Segment Prefetching and Caching at the Edge (SPACE)
25
25
Approach:
●Mechanism: Introduce a new server/segment selection approach at the edge of the network
●Main goal: Improve the users' QoE and network utilization
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP
Adaptive Video Streaming
26
26
R. Farahani, F. Tashtarian, A. Erfanian, C. Timmerer, M. Ghanbari, and H. Hellwagner. ”ES-HAS: An
Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming”.
The 31st edition of the Workshop on Network and Operating System Support for Digital Audio and
Video (NOSSDAV’21), Sept. 28-Oct. 1, 2021, Istanbul, Turkey.
https://athena.itec.aau.at/2021/04/es-has-an-edge-and-sdn-assisted-framework-for-http-adaptive-video-streaming/
●Mechanism: Introduce a new server/segment selection approach at the edge of the network
●Main goal: Improve the users' QoE and network utilization
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP
Adaptive Video Streaming
26
26
R. Farahani, F. Tashtarian, A. Erfanian, C. Timmerer, M. Ghanbari, and H. Hellwagner. ”ES-HAS: An
Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming”.
The 31st edition of the Workshop on Network and Operating System Support for Digital Audio and
Video (NOSSDAV’21), Sept. 28-Oct. 1, 2021, Istanbul, Turkey.
https://athena.itec.aau.at/2021/04/es-has-an-edge-and-sdn-assisted-framework-for-http-adaptive-video-streaming/
Goal
●Minimizing HAS clients’ serving time & network cost,
considering available resources
●Multi-layer architecture + centralized optimization model
executed by SDN controller ⇨ high time complexity
●Three heuristic approaches: CG, FG-I, FG-II
●Experiments on a large-scale cloud-based testbed
including 250 HAS players
⇨ improve QoE by at least 47%
⇨ decrease the streaming cost by at least 47%
⇨ enhance network utilization by at least 48%
compared to state-of-the-art
ARARAT: A Collaborative Edge-Assisted Framework for
HTTP Adaptive Video Streaming
27
27
R. Farahani, M. Shojafar, C. Timmerer, F. Tashtarian, M. Ghanbari and H. Hellwagner, "ARARAT: A Collaborative
Edge-Assisted Framework for HTTP Adaptive Video Streaming," in IEEE Transactions on Network and
Service Management, vol. 20, no. 1, pp. 625-643, March 2023, doi: 10.1109/TNSM.2022.3210595.
https://athena.itec.aau.at/2022/09/ararat/
●Minimizing HAS clients’ serving time & network cost,
considering available resources
●Multi-layer architecture + centralized optimization model
executed by SDN controller ⇨ high time complexity
●Three heuristic approaches: CG, FG-I, FG-II
●Experiments on a large-scale cloud-based testbed
including 250 HAS players
⇨ improve QoE by at least 47%
⇨ decrease the streaming cost by at least 47%
⇨ enhance network utilization by at least 48%
compared to state-of-the-art
ARARAT: A Collaborative Edge-Assisted Framework for
HTTP Adaptive Video Streaming
27
27
R. Farahani, M. Shojafar, C. Timmerer, F. Tashtarian, M. Ghanbari and H. Hellwagner, "ARARAT: A Collaborative
Edge-Assisted Framework for HTTP Adaptive Video Streaming," in IEEE Transactions on Network and
Service Management, vol. 20, no. 1, pp. 625-643, March 2023, doi: 10.1109/TNSM.2022.3210595.
https://athena.itec.aau.at/2022/09/ararat/
Extract some features as metadata during the encoding process ⇨ Reuse
metadata in the transcoding process at the edge
Light-weight Transcoding at the Edge (LwTE)
28
28
A. Erfanian, H. Amirpour, F. Tashtarian, C. Timmerer and H. Hellwagner,
LwTE: Light-Weight Transcoding at the Edge," in IEEE Access, vol. 9, pp. 112276-112289, 2021
https://athena.itec.aau.at/2021/07/lwte-light-weight-transcoding-at-the-edge/
Delivery
3
4
Extract
metadata
1
2
Determine
optimal
policy
3
Optimized
download/
transcode
3
Variations:
●CD-LwTE
●LwTE-Live
metadata in the transcoding process at the edge
Light-weight Transcoding at the Edge (LwTE)
28
28
A. Erfanian, H. Amirpour, F. Tashtarian, C. Timmerer and H. Hellwagner,
LwTE: Light-Weight Transcoding at the Edge," in IEEE Access, vol. 9, pp. 112276-112289, 2021
https://athena.itec.aau.at/2021/07/lwte-light-weight-transcoding-at-the-edge/
Delivery
3
4
Extract
metadata
1
2
Determine
optimal
policy
3
Optimized
download/
transcode
3
Variations:
●CD-LwTE
●LwTE-Live
LwTE: Binary Linear Programming (BLP) Model
29
29
Inputs & Constraints:
●Videos/Segments Size
●Metadata Size
●Resources Cost
●Available Resources
●Probability Function
●Number of Incoming Requests
BLP Optimization Model
Outputs:
●Segments’ Serving Policy
(store/transcode/fetch)
Objective function: Minimize cost (computation,
storage, bandwidth) and serving delay
29
29
Inputs & Constraints:
●Videos/Segments Size
●Metadata Size
●Resources Cost
●Available Resources
●Probability Function
●Number of Incoming Requests
BLP Optimization Model
Outputs:
●Segments’ Serving Policy
(store/transcode/fetch)
Objective function: Minimize cost (computation,
storage, bandwidth) and serving delay
Performance of the proposed CD-LwTE approaches (FGH, CGH) compared with
state-of-the-art approaches in terms of (a) cost, and (b) average serving delay, for various ρ
values (the number of incoming requests at the edge server)
CD-LwTE Comparison with State of the Art
30
30
APAC: T. X. Tran, P. Pandey, A. Hajisami, and D. Pompili, “Collaborative multibitrate video caching and processing in Mobile-Edge
Computing networks,” in 2017 13th Annual Conference on Wireless On-demand Network Systems and Services (WONS), 2017, pp. 165–172.
CoCache: T. X. Tran and D. Pompili, “Adaptive Bitrate Video Caching and Processing in Mobile-Edge Computing Networks,” IEEE
Transactions on Mobile Computing, vol. 18, no. 9, pp. 1965–1978, 2019.
PartialCache: H. Zhao, Q. Zheng, W. Zhang, B. Du, and H. Li, “A Segment-based Storage and Transcoding Trade-off Strategy for
Multi-version VoD Systems in the Cloud,” IEEE Transactions on Multimedia, vol. 19, no. 1, pp. 149–159, 2016.
A. Erfanian, H. Amirpour, F. Tashtarian, C. Timmerer and H. Hellwagner, "CD-LwTE: Cost-and Delay-aware Light-weight
Transcoding at the Edge," in IEEE Transactions on Network and Service Management, doi: 10.1109/TNSM.2022.3229744.
https://athena.itec.aau.at/2022/12/cd-lwte-cost-and-delay-aware-light-weight-transcoding-at-the-edge/
CD-LwTE
CD-LwTE
state-of-the-art approaches in terms of (a) cost, and (b) average serving delay, for various ρ
values (the number of incoming requests at the edge server)
CD-LwTE Comparison with State of the Art
30
30
APAC: T. X. Tran, P. Pandey, A. Hajisami, and D. Pompili, “Collaborative multibitrate video caching and processing in Mobile-Edge
Computing networks,” in 2017 13th Annual Conference on Wireless On-demand Network Systems and Services (WONS), 2017, pp. 165–172.
CoCache: T. X. Tran and D. Pompili, “Adaptive Bitrate Video Caching and Processing in Mobile-Edge Computing Networks,” IEEE
Transactions on Mobile Computing, vol. 18, no. 9, pp. 1965–1978, 2019.
PartialCache: H. Zhao, Q. Zheng, W. Zhang, B. Du, and H. Li, “A Segment-based Storage and Transcoding Trade-off Strategy for
Multi-version VoD Systems in the Cloud,” IEEE Transactions on Multimedia, vol. 19, no. 1, pp. 149–159, 2016.
A. Erfanian, H. Amirpour, F. Tashtarian, C. Timmerer and H. Hellwagner, "CD-LwTE: Cost-and Delay-aware Light-weight
Transcoding at the Edge," in IEEE Transactions on Network and Service Management, doi: 10.1109/TNSM.2022.3229744.
https://athena.itec.aau.at/2022/12/cd-lwte-cost-and-delay-aware-light-weight-transcoding-at-the-edge/
CD-LwTE
CD-LwTE
LwTE Findings
31
31
●Stores the optimal search
decisions in the encoding
process as metadata
●Utilizes the metadata to
avoid search processes
during transcoding at the
edge
●Uses partial-transcoding
●LwTE does transcoding 80%
faster than H.265
●Up to 70% cost saving
compared to
state-of-the-art
LwTE
●Extends LwTE by relaxing
assumptions, new policy, and
serving delay to objective
●Adds new features in
metadata
●BLP model to select optimal
policy to serve requests while
minimizing cost and delay
●Reduces
transcoding time up to 97%
streaming cost up to 75%
delay up to 48%
compared to state-of-the-art
CD-LwTE
●Investigates LwTE’s
performance in live
streaming context
●MBLP model to select
optimal policy (fetching and
transcoding) to serve
requests
●Reduces
streaming cost up to 34%
bandwidth up to 45%
compared to state-of-the-art
LwTE-Live
31
31
●Stores the optimal search
decisions in the encoding
process as metadata
●Utilizes the metadata to
avoid search processes
during transcoding at the
edge
●Uses partial-transcoding
●LwTE does transcoding 80%
faster than H.265
●Up to 70% cost saving
compared to
state-of-the-art
LwTE
●Extends LwTE by relaxing
assumptions, new policy, and
serving delay to objective
●Adds new features in
metadata
●BLP model to select optimal
policy to serve requests while
minimizing cost and delay
●Reduces
transcoding time up to 97%
streaming cost up to 75%
delay up to 48%
compared to state-of-the-art
CD-LwTE
●Investigates LwTE’s
performance in live
streaming context
●MBLP model to select
optimal policy (fetching and
transcoding) to serve
requests
●Reduces
streaming cost up to 34%
bandwidth up to 45%
compared to state-of-the-art
LwTE-Live
Player Adaptation Logic and Quality of Experience
●Bitrate adaptation schemes
Client-based, server-based, network-assisted, hybrid, ML-based
●Application/transport layer enhancements
HTTP/2 (TCP) and HTTP/3 (QUIC), Media over QUIC (MOQ), proprietary formats (SRT, RIST, …),
WebRTC, low-latency/delay
●Client playback improvements
User-/client-aware playback, content-enhancement filters, super-resolution
●Low-latency live streaming
Use of MPEG CMAF, HTTP/1.1 Chunked Transfer Encoding (CTE) , other protocol features
(e.g., HTTP/2 Push); LL-DASH/HLS; specific network functions; CDN support
●Quality of Experience
Content Consumption and End-to-End Aspects
32
32
●Bitrate adaptation schemes
Client-based, server-based, network-assisted, hybrid, ML-based
●Application/transport layer enhancements
HTTP/2 (TCP) and HTTP/3 (QUIC), Media over QUIC (MOQ), proprietary formats (SRT, RIST, …),
WebRTC, low-latency/delay
●Client playback improvements
User-/client-aware playback, content-enhancement filters, super-resolution
●Low-latency live streaming
Use of MPEG CMAF, HTTP/1.1 Chunked Transfer Encoding (CTE) , other protocol features
(e.g., HTTP/2 Push); LL-DASH/HLS; specific network functions; CDN support
●Quality of Experience
Content Consumption and End-to-End Aspects
32
32
Bitrate Adaptation Schemes
Bitrate
Adaptation
Schemes
Client-based
Adaptation
Bandwidth
-based
Buffer-
based
Mixed
adaptation
Proprietary
solutions
MDP-based
Server-based
Adaptation
Network-
assisted
Adaptation
Hybrid
Adaptation
SDN-based
Server and
network-
assisted
A. Bentaleb, B. Taani, A.C. Begen, C. Timmerer and R. Zimmermann, "A Survey on Bitrate Adaptation Schemes for
Streaming Media Over HTTP, " IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 562-585, First Quarter 2019.
33
33
Adaptive bitrate (ABR) algorithms, adaptation logics, ...
Bitrate
Adaptation
Schemes
Client-based
Adaptation
Bandwidth
-based
Buffer-
based
Mixed
adaptation
Proprietary
solutions
MDP-based
Server-based
Adaptation
Network-
assisted
Adaptation
Hybrid
Adaptation
SDN-based
Server and
network-
assisted
A. Bentaleb, B. Taani, A.C. Begen, C. Timmerer and R. Zimmermann, "A Survey on Bitrate Adaptation Schemes for
Streaming Media Over HTTP, " IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 562-585, First Quarter 2019.
33
33
Adaptive bitrate (ABR) algorithms, adaptation logics, ...
E-WISH: A policy driven ABR Algorithm
34
Energy Cost+ ??????
[Data Cost]
Data CostTotal Cost = ɑ
Segment bitrate
Throughput
[Buffer Cost]
Buffer Cost+ β
Download time
Buffer occupancy
[Quality Cost]
Quality Cost+ ɣ
Video quality
Distortion
Instability
Bitrate, Resolution, FPS (↑)
M. Nguyen, E. Çetinkaya, H. Hellwagner and C. Timmerer, "WISH: User-centric Bitrate Adaptation for
HTTP Adaptive Streaming on Mobile Devices," 2021 IEEE 23rd International Workshop on Multimedia
Signal Processing (MMSP), doi: 10.1109/MMSP53017.2021.9733605
Technology Transfer
34
Energy Cost+ ??????
[Data Cost]
Data CostTotal Cost = ɑ
Segment bitrate
Throughput
[Buffer Cost]
Buffer Cost+ β
Download time
Buffer occupancy
[Quality Cost]
Quality Cost+ ɣ
Video quality
Distortion
Instability
Bitrate, Resolution, FPS (↑)
M. Nguyen, E. Çetinkaya, H. Hellwagner and C. Timmerer, "WISH: User-centric Bitrate Adaptation for
HTTP Adaptive Streaming on Mobile Devices," 2021 IEEE 23rd International Workshop on Multimedia
Signal Processing (MMSP), doi: 10.1109/MMSP53017.2021.9733605
Technology Transfer
E-WISH: An Energy-aware ABR Algorithm
35
[Data Cost]
[Quality Cost]
[Buffer Cost]
[Energy Cost]
Data Cost Buffer Cost Quality Cost Energy CostTotal Cost = ɑ + β + ɣ + ??????
Segment bitrate
Throughput
Download time
Buffer occupancy
Video quality
Distortion
Instability
Bitrate, Resolution, FPS
M. Nguyen, E. Çetinkaya, H. Hellwagner and C. Timmerer, "WISH: User-centric Bitrate Adaptation for
HTTP Adaptive Streaming on Mobile Devices," 2021 IEEE 23rd International Workshop on Multimedia
Signal Processing (MMSP), doi: 10.1109/MMSP53017.2021.9733605
D.Lorenzi, M. Nguyen, F. Tashtarian, and C. Timmerer, “E-WISH: An Energy-aware ABR Algorithm For
Green HTTP Adaptive Video Streaming”, 3rd ACM Mile-High Video Conference (MHV '24),
https://doi.org/10.1145/3638036.3640802
35
[Data Cost]
[Quality Cost]
[Buffer Cost]
[Energy Cost]
Data Cost Buffer Cost Quality Cost Energy CostTotal Cost = ɑ + β + ɣ + ??????
Segment bitrate
Throughput
Download time
Buffer occupancy
Video quality
Distortion
Instability
Bitrate, Resolution, FPS
M. Nguyen, E. Çetinkaya, H. Hellwagner and C. Timmerer, "WISH: User-centric Bitrate Adaptation for
HTTP Adaptive Streaming on Mobile Devices," 2021 IEEE 23rd International Workshop on Multimedia
Signal Processing (MMSP), doi: 10.1109/MMSP53017.2021.9733605
D.Lorenzi, M. Nguyen, F. Tashtarian, and C. Timmerer, “E-WISH: An Energy-aware ABR Algorithm For
Green HTTP Adaptive Video Streaming”, 3rd ACM Mile-High Video Conference (MHV '24),
https://doi.org/10.1145/3638036.3640802
E-WISH: An Energy-aware ABR Algorithm
36
D.Lorenzi, M. Nguyen, F. Tashtarian, and C. Timmerer, “E-WISH: An Energy-aware ABR Algorithm For
Green HTTP Adaptive Video Streaming”, 3rd ACM Mile-High Video Conference (MHV '24),
https://doi.org/10.1145/3638036.3640802
36
D.Lorenzi, M. Nguyen, F. Tashtarian, and C. Timmerer, “E-WISH: An Energy-aware ABR Algorithm For
Green HTTP Adaptive Video Streaming”, 3rd ACM Mile-High Video Conference (MHV '24),
https://doi.org/10.1145/3638036.3640802
Based on ACM MMSys’22 keynote by Ali C. Begen
LLL-CAdViSE: Live Low-Latency Cloud-based
Adaptive Video Streaming Evaluation
37
37
B. Taraghi, H. Hellwagner and C. Timmerer, "LLL-CAdViSE: Live Low-Latency Cloud-Based Adaptive Video
Streaming Evaluation Framework," in IEEE Access, vol. 11, pp. 25723-25734, 2023
https://athena.itec.aau.at/2023/03/lll-cadvise-live-low-latency-cloud-based-adaptive-video-streaming-evaluation-framework/
Average latency and ITU-T P.1203 MOS of MPEG-DASH (dash.js) and HLS (hls.js) using
three different ABR algorithms (default, L2A-LL, LoLP) with a given target latency of 3s.
https://github.com/cd-athena/LLL-CAdViSE
LLL-CAdViSE: Live Low-Latency Cloud-based
Adaptive Video Streaming Evaluation
37
37
B. Taraghi, H. Hellwagner and C. Timmerer, "LLL-CAdViSE: Live Low-Latency Cloud-Based Adaptive Video
Streaming Evaluation Framework," in IEEE Access, vol. 11, pp. 25723-25734, 2023
https://athena.itec.aau.at/2023/03/lll-cadvise-live-low-latency-cloud-based-adaptive-video-streaming-evaluation-framework/
Average latency and ITU-T P.1203 MOS of MPEG-DASH (dash.js) and HLS (hls.js) using
three different ABR algorithms (default, L2A-LL, LoLP) with a given target latency of 3s.
https://github.com/cd-athena/LLL-CAdViSE
Live Video Streaming Pipeline
38
LIVE
Origin Server
(Encoding & Packaging)
Live Camera Player CDN Server
manifest.mpd
Bitrate Resolution
5 Mbps 1920x1080
2.8 Mbps 1280x720
1.1 Mbps 960x540
System Admin
Bitrate
Ladder
HTTP Request
HTTP Response
Manifest includes representations:
different versions of the content optimized for various
playback conditions, e.g., different bitrates and resolutions
Tashtarian et al., "ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming", 21st USENIX
Symposium on Networked Systems Design and Implementation (NSDI 24), Santa Clara, CA, 2024
●Fixed bitrate ladder
○Content-aware, e.g., Netflix’s per-title encoding
○Context-aware, e.g., [Lebreton and Yamagishi 2023]
○Agnostic of the content and context (proprietary)
38
LIVE
Origin Server
(Encoding & Packaging)
Live Camera Player CDN Server
manifest.mpd
Bitrate Resolution
5 Mbps 1920x1080
2.8 Mbps 1280x720
1.1 Mbps 960x540
System Admin
Bitrate
Ladder
HTTP Request
HTTP Response
Manifest includes representations:
different versions of the content optimized for various
playback conditions, e.g., different bitrates and resolutions
Tashtarian et al., "ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming", 21st USENIX
Symposium on Networked Systems Design and Implementation (NSDI 24), Santa Clara, CA, 2024
●Fixed bitrate ladder
○Content-aware, e.g., Netflix’s per-title encoding
○Context-aware, e.g., [Lebreton and Yamagishi 2023]
○Agnostic of the content and context (proprietary)
ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming
Select representations
with bitrates that closely
match the desired bitrates
Players
Advertise a large
number of
representations
Encoded
segments
CDN
Server
Mega-Manifest
39
Origin Server
OTL
Process requests
and video qualities
and determine OTL
LIVE
Live
Camera
Utilize the OTL
OTL: Optimized Temporary Ladder
Tashtarian et al., "ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming", 21st USENIX
Symposium on Networked Systems Design and Implementation (NSDI 24), Santa Clara, CA, 2024
Select representations
with bitrates that closely
match the desired bitrates
Players
Advertise a large
number of
representations
Encoded
segments
CDN
Server
Mega-Manifest
39
Origin Server
OTL
Process requests
and video qualities
and determine OTL
LIVE
Live
Camera
Utilize the OTL
OTL: Optimized Temporary Ladder
Tashtarian et al., "ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming", 21st USENIX
Symposium on Networked Systems Design and Implementation (NSDI 24), Santa Clara, CA, 2024
QoE, Latency, and Stall with ARTEMIS vs. Five Static
Bitrate Ladders
4
0
40
Average QoE up 11%
Average Stall down 36%
Average Latency down 18%
Bitrate Ladders
4
0
40
Average QoE up 11%
Average Stall down 36%
Average Latency down 18%
Quality of Experience (QoE) ...
●“... is the degree of delight or annoyance of the user of an application or service.
It results from the fulfillment of his or her expectations with respect to the utility and / or
enjoyment of the application or service in the light of the user’s personality and current state.”
1)
●… can be easily extended to various domains, e.g., immersive media experiences.
2)
41
41
1)
P. Le Callet, S. Möller, A. Perkis, et al. QUALINET White Paper on
Definitions of Quality of Experience. European Network on Quality of
Experience in Multimedia Systems and Services (COST Action IC 1003). 2012.
2)
A. Perkis, C. Timmerer, et al. 2020. QUALINET White Paper on Definitions
of Immersive Media Experience (IMEx). arXiv:2007.07032 [cs.MM]
3)
Jeroen van der Hooft, Tim Wauters, Filip De Turck, Christian Timmerer, and
Hermann Hellwagner. “Towards 6DoF HTTP Adaptive Streaming Through
Point Cloud Compression”.
27th ACM Int’l. Conf. on Multimedia (MM'19). Oct. 2019.
4)
J. van der Hooft, M. T. Vega, C. Timmerer, A. C. Begen, F. De Turck and R.
Schatz, "Objective and Subjective QoE Evaluation for Adaptive Point
Cloud Streaming," 2020 Twelfth International Conference on Quality of
Multimedia Experience (QoMEX), Athlone, Ireland, 2020.
3) 4)
●“... is the degree of delight or annoyance of the user of an application or service.
It results from the fulfillment of his or her expectations with respect to the utility and / or
enjoyment of the application or service in the light of the user’s personality and current state.”
1)
●… can be easily extended to various domains, e.g., immersive media experiences.
2)
41
41
1)
P. Le Callet, S. Möller, A. Perkis, et al. QUALINET White Paper on
Definitions of Quality of Experience. European Network on Quality of
Experience in Multimedia Systems and Services (COST Action IC 1003). 2012.
2)
A. Perkis, C. Timmerer, et al. 2020. QUALINET White Paper on Definitions
of Immersive Media Experience (IMEx). arXiv:2007.07032 [cs.MM]
3)
Jeroen van der Hooft, Tim Wauters, Filip De Turck, Christian Timmerer, and
Hermann Hellwagner. “Towards 6DoF HTTP Adaptive Streaming Through
Point Cloud Compression”.
27th ACM Int’l. Conf. on Multimedia (MM'19). Oct. 2019.
4)
J. van der Hooft, M. T. Vega, C. Timmerer, A. C. Begen, F. De Turck and R.
Schatz, "Objective and Subjective QoE Evaluation for Adaptive Point
Cloud Streaming," 2020 Twelfth International Conference on Quality of
Multimedia Experience (QoMEX), Athlone, Ireland, 2020.
3) 4)
Immersive Video Delivery:
From Omnidirectional Video to Holography
42
42
J. v. d. Hooft, H. Amirpour, M. Torres Vega, Y. Sanchez, R. Schatz; T. Schierl, C. Timmerer, "A Tutorial on
Immersive Video Delivery: From Omnidirectional Video to Holography," in IEEE Communications
Surveys & Tutorials, 2023 (early access) doi: 10.1109/COMST.2023.3263252.
●3DoF Omnidirectional Video
●6DoF Volumetric Video
●6DoF Imagery Video
From Omnidirectional Video to Holography
42
42
J. v. d. Hooft, H. Amirpour, M. Torres Vega, Y. Sanchez, R. Schatz; T. Schierl, C. Timmerer, "A Tutorial on
Immersive Video Delivery: From Omnidirectional Video to Holography," in IEEE Communications
Surveys & Tutorials, 2023 (early access) doi: 10.1109/COMST.2023.3263252.
●3DoF Omnidirectional Video
●6DoF Volumetric Video
●6DoF Imagery Video
NeRV: Neural Representations for Videos, NeurIPS 2021
Deep Learning-Based Video Coding: A Review and a
Case Study, ACM CSUR 2020
●Video Coding
Learned video coding / E2E deep video coding
Future of Video Streaming
43
43
An End-to-End Learning Framework for Video
Compression, TPAMI 2021
●Video Streaming
DeepStream: Video Streaming Enhancements
using Compressed Deep Neural Networks
Deep learning-based video coding
Neural implicit representation for videos
DeepStream: TCSVT 2022
Generative AI for video streaming
Deep Learning-Based Video Coding: A Review and a
Case Study, ACM CSUR 2020
●Video Coding
Learned video coding / E2E deep video coding
Future of Video Streaming
43
43
An End-to-End Learning Framework for Video
Compression, TPAMI 2021
●Video Streaming
DeepStream: Video Streaming Enhancements
using Compressed Deep Neural Networks
Deep learning-based video coding
Neural implicit representation for videos
DeepStream: TCSVT 2022
Generative AI for video streaming
ATHENA team
Samira Afzal, Hadi Amirpour, Jesús Aguilar Armijo, Emanuele Artioli, Christian Bauer, Alexis Boniface, Ekrem
Çetinkaya, Reza Ebrahimi, Alireza Erfanian, Reza Farahani, Mohammad Ghanbari (late), Milad Ghanbari,
Mohammad Ghasempour, Selina Zoë Haack, Hermann Hellwagner, Manuel Hoi, Andreas Kogler, Gregor Lammer,
Armin Lachini, David Langmeier, Sandro Linder, Daniele Lorenzi, Vignesh V Menon, Minh Nguyen, Engin Orhan,
Lingfeng Qu, Jameson Steiner, Nina Stiller, Babak Taraghi, Farzad Tashtarian, Yuan Yuan, Yiying Wei
International Collaborators (selection)
●Prof. Ali C. Begen, Ozyegin University, Turkey
●Prof. Filip De Turck, Ghent University – imec, Belgium
●Dr. Jeroen van der Hooft, Ghent University – imec, Belgium
●Dr. Maria Torres Vega, Ghent University – imec, Belgium
●Prof. Wassim Hamidouche, Technology Innovation Institute (UAE)
●Prof. Roger Zimmermann, NUS, Singapore
●Dr. Abdelhak Bentaleb, Concordia University, Canada
●Prof. Christine Guillemot, INRIA, Rennes, France
●Prof. Patrick Le Callet, Polytech Nantes-Institut Universitaire de France, France
●Prof. Junchen Jiang, University of Chicago, USA
●Dr. Sergey Gorinsky, IMDEA, Spain
Acknowledgments
4
4
44
https://athena.itec.aau.at/
Samira Afzal, Hadi Amirpour, Jesús Aguilar Armijo, Emanuele Artioli, Christian Bauer, Alexis Boniface, Ekrem
Çetinkaya, Reza Ebrahimi, Alireza Erfanian, Reza Farahani, Mohammad Ghanbari (late), Milad Ghanbari,
Mohammad Ghasempour, Selina Zoë Haack, Hermann Hellwagner, Manuel Hoi, Andreas Kogler, Gregor Lammer,
Armin Lachini, David Langmeier, Sandro Linder, Daniele Lorenzi, Vignesh V Menon, Minh Nguyen, Engin Orhan,
Lingfeng Qu, Jameson Steiner, Nina Stiller, Babak Taraghi, Farzad Tashtarian, Yuan Yuan, Yiying Wei
International Collaborators (selection)
●Prof. Ali C. Begen, Ozyegin University, Turkey
●Prof. Filip De Turck, Ghent University – imec, Belgium
●Dr. Jeroen van der Hooft, Ghent University – imec, Belgium
●Dr. Maria Torres Vega, Ghent University – imec, Belgium
●Prof. Wassim Hamidouche, Technology Innovation Institute (UAE)
●Prof. Roger Zimmermann, NUS, Singapore
●Dr. Abdelhak Bentaleb, Concordia University, Canada
●Prof. Christine Guillemot, INRIA, Rennes, France
●Prof. Patrick Le Callet, Polytech Nantes-Institut Universitaire de France, France
●Prof. Junchen Jiang, University of Chicago, USA
●Dr. Sergey Gorinsky, IMDEA, Spain
Acknowledgments
4
4
44
https://athena.itec.aau.at/
Thank you for your attention
45
45
46
46
46
Tags
Download
Download Slideshow Get the original presentation fileQuick Actions
Statistics
- Views 1,197
- Slides 46
- Age 517 days
Related Slideshows
23
Earthquakes_Type of Faults_Science G8.pptx
OctabellFabila1
31 views
15
Quiz #1 Science 10 in the first quarter for jhs
HendrixAntonniAmante
30 views
9
Astronomy history from long ago till doday
ssuserbd9abe
29 views
9
Great history of astronomy from long ago till today
ssuserbd9abe
27 views
20
EARTHQUAKE-DRILL.powerpoint.............
chalobrido8
30 views
9
History of astronomy from old times to the present times
ssuserbd9abe
30 views