CMOS Image Sensor Design_h20_10_jpeg.pdf

in5350_h20_10_jpeg

1. IN5350 –CMOS Image Sensor Design
Lecture 10 – Introduction to JPEG
compression
20-October-2020

2. Key takeaways from previous lecture
•Low-pass filter: average accross neighboring
pixels, h⊗f(x,y)
•High-pass filter (unsharp): g(x,y)={f(x,y)−α⋅
(h⊗f(x,y))} /(1−α)
•3 AWB methods: grey world, brightest point, color
gamut
•Delta-E used to quantify color accuracy
18.10.2020
2

3. Why JPEG compression?
•Reduce output data bandwidth
•Enable higher video frame rate (when output data
bandwidth is the bottleneck)
•Reduce I/O area and power consumption
•Save image storage space on host side
18.10.2020
3

4. Basic concepts used in JPEG
•Human visual system more sensitive to luma
levels than chroma levels
–Sub-sample chroma data
•Small changes between neighboring pixels
–Transform into frequency domain which enables
efficient coding schemes
•Human visual system more sensitive to low
frequency than high frequency content
–Remove HF contents that HVS cannot see
18.10.2020
4

5. Conceptual diagram av JPEG encoder
Y
Cb
Cr
Chroma sub sampling
Compressed
image
18.10.2020 5

6. Step-1: RGB => YCbCr
0.11B + G 0.587 + R0.299 = Y
Y)- (B 0.564 = Cb
Y)- (R 0.713 =Cr
18.10.2020 6

7. Step-2: Sub-sample chroma information
•Selected chroma coding scheme written in JPEG header file
18.10.2020 7

8. Step-3: divide up picture in macro blocks
of 8x8 pixels
•Studies have shown that 8x8 is optimal block size
–High correlation (redundancy) inside pixel cluster
–Relatively inexpensive HW implementation (only 8 rows
of memory required to process 8x8 matrix)
18.10.2020
8

9. VGA example
VGA picture: 640x480 pixels
4:2:0 coding
•4800 (80x60) luma macro blocks (Y)
•2x1200 (40x30) chroma macro blocks (Cb, Cr)
18.10.2020 9

10. Step-4: DCT transformation
•Analogous to FFT. Signal transformed to frequency domain.
•Each macro block described as weighted sum of 64
‘frequencies’ (i.e. jpeg-defined reference pictures)
•Input (X
ij): 8x8 matrix (original picture)
•Output (Y
ij): 8x8 matrix (coefficients for each ‘frequency’)
T
ijij
AXAYsjontransformaDCT ⋅⋅=:
18.10.2020 10

11. Conceptual diagram of DCT transform
JPEG defined
8x8 DCT
patterns
18.10.2020 11

12. DCT example
X
ij
Y
ij
AX
ijA
T
HF
content
close to
zero
18.10.2020 12

13. Step-5: Quantization (lossy)
•Divide by quantisation matrix, Q, and round off decimals
•Level of lossyness defined by Q which is defined in
JPEG header
•Coefficients at high frequencies can be rounded to zero
without visible image artifacts








=
ij
ij
ij
Q
Y
RoundZ
Luma: Chroma:
18.10.2020 13

14. Example
Y’
ij= Z
ijx Q
ij
After DCT (Y
ij)Original picture (X
ij)
Reversing the process
After quantization (Z
ij)
X’
ij
Y’
ij
18.10.2020 14

15. Step-6: Coding of DCT coefficients after
quantization
•DC values (0,0 position) coded with Differential PCM (DPCM)
•AC values (all other positions) coded with Run Length Coding (RLC)
. . . . . .
DC
i-1
DC
i 1−−=∆
iiiDCDCDC
DPCM coding of DC-values
18.10.2020 15

16. Run Level Coding (RLC) of AC values
•Efficient coding method of all zeros in the matrix
•Diagonal (ZigZag) scanning of the matrix
–Count zeroes (A) until non- zero value (B)
–Transmit (A,B), and start counting zeros again
RLC example:
(0, 32), (0, 6), (0, - 1),
(0, -1), (1,- 1), (3, -1),
(2, 1), (EOB)
EOB=End of block
18.10.2020
16

17. Step-7: Entropy encoding
•Lossless
•Frequent codes represented with few bits
•Rare codes represented with many bits
•Also called Variable Length Coding (VLC)
–Available VLC algorithms
•Huffman coding (most common)
•Adaptive Huffmancoding
•Shannon- FanoAlgorithm
•Arithmeticcoding
•Jpeg header contains look- up table for selected
VLC algorithm
18.10.2020
17

18. Huffman coding of DC and AC coefficients
18.10.2020 18
Huffman coding of AC coefficients
Huffman coding of DC coefficients

19. Some JPEG variants
•Progressive JPEG: picture stored as series of
‘scans’ that make the picture gradually sharper
•Lossless JPEG: no quantization
•JPEG 2000: ‘wavelet’ transformation instead of
DCT
18.10.2020
19

20. Other compression standards
•MPEG
•H.264
•Wavelet (e.g. JPEG-2000)
•fractal
•vector quantization
18.10.2020 20

21. JPEG references
•W.B. Pennebaker and J.L. Mitchell, JPEG Still
Image Data Compression Standard, International
Thomson Publishing, Chapman & Hall, NY, 1992
•Digital Image Processing by R.C. Gonzalez and
R.E. Woods
•http://www.wikipedia.org
18.10.2020 21

22. Key takeaways
•JPEG often embedded in CIS with on- chip color
processing (aka SOC imagers) to reduce output
datarate
•Image sectioned into 8x8 blocks
•Color compressed by chromasub-sampling
•DCT transforms image into frequency domain
•Quantization matrix defines level of compression
•Data is transmitted using efficient coding schemes
(DPCM/RLC/Huffman) for minimum redundancy
18.10.2020 22