Introduction to Audiovisual Communications

Introduction to Audiovisual Communications.
The presentation gives a big picture of different technologies involved in the Audio and Video Communication systems.

1. Audiovisual Communications
Introducción a las Comunicaciones
Audiovisuales
Profesores: Francesc Tarrés, Gabriel Montoro
1

2. Audiovisual Communications
Introduction
•Audiovisual Communications Technologies
•Set of technologies for transmitting or storing signals
representing speech, audio, image or video information in
different formats.
•Audiovisual Technologies Impact:
•Communications among persons through sound and image is
(still today) the media of major social impact.
•Classical Media: Films, TV, Radio, Telephone
•Networked contents: Production webs, Podcast, Video On-Demand
•Contents merchandising: iTunes, Spotify
•Personal Communications: Facetime, WeChat
•Social Networks: Facebook, Instagram, Flickr, etc.
2

3. Audiovisual Communications
Introduction
3

4. Audiovisual Communications
Historical Evolution
•Constantly improving the audiovisual experience
•Technologies are ‘determined’ by the characteristics of human auditory
and visual systems.
•Today almost all AV systems are digital. This course is centered on
digital systems.
•However, many AV paradigms are understood when looking to the
historical evolution of the system. Some KEY historical aspects are
introduced to facilitate the understanding of current systems.
•Audiovisual Systems are evolutionary. The introduction of new
technologies preserves some degree of compatibility with previous
systems.
4

5. Audiovisual Communications
Introduction
•Evolution example (Comercial TV)
•Black & White Television
•Analog Color
•Digital Color (Standard Definition)
•High Definition
•3D
•Ultra High Definition
•Non-evolutionary attempts have usually failed
•Analog High Definition TV in Europe (late 80s, beg. 90s)
•Economic efforts for acquiring new technologies is
almost constant along years
5

6. Audiovisual Communications
Future Systems
•It is difficult to predict the evolution of AV systems. Past experience
show that many promising AV systems have not succeed: HD-MAC,
DCC, Laser-disc, Minidisc, etc.
•A possible future line prediction could be
•Ultra-High Definition (4K and 8K commercially available at low cost)
•Cameras, editing software, displays
•High Quality Audio Portable systems: Pono
•Introduction of Multisense information into the experience
•Location, motion, body sensors, weather, light, etc.
•Advances in 3D technologies
•Displays without glasses (multiuser)
•Introduction of holographic technologies
•Multicamera - Multiview
•Capturing and transmission of otner sensorial signals
•Olphative
•Tactil
6

7. Audiovisual Communications
Future Systems
•No matter how real systems evolve the main characteristics
from the point of view of signal processing will remain the
same
•The following steps will be necessary
1. Signals must be captured and coded efficiently
2. The coding has to be adapted to the transmission channel or to
the storage support.
3. Reproduction technologies must guarantee a quality experience
to the audience
•The design and technologies of these stages has to be
adapted to the psycho-physiology human sensorial
system (auditory, visual, olphactive, etc)
7

8. Audiovisual Communications
Communication Systems
Block Diagram
8
Capture
Source
Coding
Multiplexer
Encryption &
Channel
Coding
Channel
Decoding
& Decryption
Demux
Source
Decoding
Presentation
Storage o Transmission
Modulators & Demodulators
(QAM, OFDM, PSK)
IP data transmission
File Downloading
DVD-Video, Blu-ray
Capture
Presentation
Signal transducer
Source
Coding
Compression
Multiplexing
Collect
several channels
into one stream
Encryption and
Channel
Coding
Encription
Adapt Data to
Media
Error protection
Microphones
Video Cameras
Displays
Speakers
A/D and D/A
PCM coding
FLAC, APE
MP3, AAC
TIFF, PNG
JPEG, JP2
MPEG-2, H.264
MPEG-TS
MPEG- PS
AVI, MP4, MKV
Block Coding
Hamming
Reed-Solomon
Conv. Coding
Trellis
LDPC
Condtional Access
Encription
Scrambling

9. Audiovisual Communications
Digital AV Systems
•Advantages
•Immunity to noise
•Easy reprogramming
•Low dependence on external factors (e.g. temperature)
•Easy encryption, data protection, Digital Rights Management
•Disadvantages
•Introduction of Quantization Noise
•Bandwidth Increase
•Difficulty for managing huge databases (video browsing and retrieval)
9

10. Audiovisual Communications
Quantization Noise
•Example in audio
•The number of levels in the A/D determines the quality of the
resulting digital signal.
•Error in quantizing is equivalent to noise.
•2 different criteria for digitalization
•Quality: Suppose analog signal is noise free. Determine the number of bits
necessary for a human being not being able to distinguish between original analog
and the digital version
•N bits audio = 16 bits (20 bits, 24 also possible in some systems).
•SNR. Suppose the analog signal has a given SNR. Choose the number of bits
(number of levels) to have a quantization noise below the analog noise.
10
SNR=6N+1.76

11. Audiovisual Communications
Bandwidth Increase
•Example audio (CD audio - 1 channel)
•Example video (Standard Definition)
11

12. Audiovisual Communications
Example in Digital Cinema
12
Bandwidth calculation in Digital Cinema Applications
Digital Cinema requires a huge bandwidth to support a full film if no compression is applied

13. Audiovisual Communications
Browsing and Retrieval
•Insert Metadata in Audio and Video Files
•Manually
•Index data by humans: ex. soccer, TV News, others
•Automatic
•Find automatic descriptors (low-level) for audio and video data.
•Use these low level descriptors to infer semantic contents
•Standard MPEG-7
•Other proprietary solutions:
•Google Images, Shazam, Midomi, SoundHound, etc.
13

14. Audiovisual Communications
Digital AV Systems
14
•Advantages
•Immunity to noise
•Easy reprogramming
•Low dependence on external factors (e.g. temperature)
•Easy encryption, data protection, Digital Rights Management
•Disadvantages
•Introduction of Quantization Noise
•Bandwidth Increase
•Compression: MP3, AAC, MPEG-1, MPEG-2, MPEG4, H.264, H.265
•Difficulty for managing huge databases (video browsing and retrieval)
• Metadata description
• MPEG-7

15. Audiovisual Communications
List of Contents
•Introduction
•Image Processing I: Point Transforms
•Image Processing II: Spatial Transforms
•Cameras. Principles of geometric optics
•Audiovisual Systems: Historical Perspective
•Principles of Compression
•Still Image Compression: JPEG, JPEG 2000
•Video Compression: MPEG-2, MPEG4, H.264, H.265
•Fundamentals of audiovisual description: MPEG-7
15

16. Audiovisual Communications
Organization
•Theory
•2 sessions of 1.5 h every week (Prof. Francesc Tarrés)
•Controls and Exams
•Arrange meetings: [email protected]
•Updated slides and textbooks: francesctarres.wordpress.com
•Laboratory
•1 session of 2 h every week (Prof. Gabriel Montoro)
•Lab work and Lab reports
•Arrange meetings: [email protected]
16

17. Audiovisual Communications
Assessment
17
Control (week 4-5) 10%
Mid-Term Exam 20%
Final Exam 30%
Lab work 15%
Lab reports 15%
Participation 10%