Digital Image restoration

Digital Image restoration,Image Enhancement,Digital image filters,arithmetic mean,geometric mean etc

1. DIGITAL IMAGE PROCESSING IMAGE RESTORATION

2. Basic Idea With Image Restoration one tries to repair errors or distortions in an image, caused during the image creation process. In general our starting point is a degradation and noise model: g(x, y) = H ( f(x, y) ) +   (x, y) Determined by quality of equipment and image taking conditions. Image restoration is computationally complex. Equipment as degradation free as possible. seen technical and financial limitations Medical: low radiation, little time in magnet-tube. lowest image quality to achieve medical goals Web-cams: cheap lens distortions corrected by CPU in cam.

3. Basic Idea Ultimate goal of restoration techniques is to improve an image in some predefined sense. This above concept obviously have certain overlapping with image enhancement, although there are differences among the both. The fundamental differences in the two concepts are illustrated in the next slide.

4. 4 Image Enhancement vs. Image Restoration Image enhancement: process image so that the result is more suitable for a specific application, is largely a subjective process . Image restoration: recover image from distortions to its original image, is largely an objective process. Image enhancement techniques are basically heuristic procedure designed to manipulate an image in order to take advantage of the psychophysical aspects of human visual systems. In contrast, the restoration approach usually involves formulating a criterion of goodness that will yield an optimal estimate of the desired result.

5. Model of Image Degradation/Restoration Degradation process is modeled as a degradation function that together with an additive noise, operates on an input image f(x, y) to produce a degraded image g(x, y). If H is a linear, position-invariant process, then where h ( x , y) is the spatial representation of the degradation function and h ( x , y ) is the noise.

6. Linear, Position-Invariant Degradation From the model of image degradation, one may write: where H is the degradation function and η is the additive noise. Initially assume that which gives,

7. Linear, Position-Invariant Degradation An operator having input-output relationship g(x, y) = H[f(x, y)] is said to be position- (or space-) invariant if for any f(x, y) and any α , β . The definition indicates that the response at any point in the image depends only on the value of the input at that point, not on its position.

8. 8 Linear, Position-Invariant Degradation The following function is called a Point Spread Function: Now we may express the linear position-invariant degradation (Spatial and Frequency Domain) as follows: The above is nothing but the impulse response of H.

9. Properties of Noise A noise is a white noise when its Fourier Spectrum is constant. We assume that the noise is independent of the spatial coordinates, and that it is uncorrelated with respect to image itself. Noise cannot be predicted, but can be approximately described in statistical way using the probability density function (PDF).

10. Noise Models Gaussian Noise: Rayleigh Noise Erlang (Gamma) Noise

11. Exponential Noise Uniform Noise Impulse (salt & pepper) Noise Noise Models If either P a or P b is zero, the impulse noise is called Uni-polar. Other wise it is called Bi-polar. Bi-polar noise is also called salt & pepper noise.

12. Sources of Noise Gaussian noise arises in an image due to factors such as electronic circuit noise and sensor noise due to poor illumination and/or high temperature. Raleigh noise is helpful in characterizing noise phenomena in range imaging. Exponential and Gamma noises are more common is Laser Imaging. Impulse noise is found in situations where quick transients, such as faulty switching, take place during imaging.

13. Image Degradation with Additive Noise Original image Histogram Degraded images

14. Original image Histogram Degraded images Image Degradation with Additive Noise

15. Estimation of Noise Parameters The parameters of periodic noise typically are estimated by inspection of the Fourier spectrum of the image. The parameters of the noise PDFs may be known partially from sensor specifications. But it is often necessary to estimate them for a particular imaging environment.

16. Restoration in presence of Noise only Clearly, g(x, y) = f(x, y) + η (x, y). In frequency domain, G(u, v) = F(u, v) + N(u, v). In general, neither η (x, y) nor N(u, v) is known in advance. We choose spatial domain filtering techniques when additive random noise is expected to be present. We use frequency domain technique only if we feel that periodic noise is present.

17. Periodic Noise Reduction by Frequency Domain Filtering Periodic noise appears as a concentrated bursts of energy in the Fourier transform, at locations corresponding to frequencies of the periodic interference. The approach for restoration is to use a selective filter to isolate the noise. The type of selective filters we use is either band-reject or band-pass filters. We also use notch-reject or notch-pass filters for the same purpose.

18. Periodic Noise Reduction by Frequency Domain Filtering Band reject filter Restored image Degraded image DFT Periodic noise can be reduced by setting frequency components corresponding to noise to zero.

19. Restoration in presence of Noise only: Spatial Domain Filtering Spatial domain filtering are chosen for restoration if the degradation present in the image is additive noise only. There are two general types of filters used in this category: Mean Filters. Order-statistics Filters.

20. Mean Filters Arithmetic mean filter or moving average filter Geometric mean filter mn = size of moving window Degradation model: To remove this part

21. Geometric Mean Filter: Example Original image Image corrupted by AWGN Image obtained using a 3x3 geometric mean filter Image obtained using a 3x3 arithmetic mean filter AWGN: Additive White Gaussian Noise

22. Harmonic and Contra-harmonic Filters Harmonic mean filter Contra-harmonic mean filter mn = size of moving window Works well for salt noise but fails for pepper noise Q = the filter order Positive Q is suitable for eliminating pepper noise. Negative Q is suitable for eliminating salt noise. For Q = 0, the filter reduces to an arithmetic mean filter. For Q = -1, the filter reduces to a harmonic mean filter.

23. Contra-harmonic Filters: Example Image corrupted by pepper noise with prob. = 0.1 Image corrupted by salt noise with prob. = 0.1 Image obtained using a 3x3 contra- harmonic mean filter With Q = 1.5 Image obtained using a 3x3 contra- harmonic mean filter With Q =-1.5

24. Contra-harmonic Filters: Incorrect Use Example Image corrupted by pepper noise with prob. = 0.1 Image corrupted by salt noise with prob. = 0.1 Image obtained using a 3x3 contra- harmonic mean filter With Q =-1.5 Image obtained using a 3x3 contra- harmonic mean filter With Q =1.5

25. Order-Statistic Filters: Revisited subimage Original image Moving window Statistic parameters Mean, Median, Mode, Min, Max, Etc. Output image

26. Order-Statistics Filters Median filter Max filter Min filter Mid-point filter Reduce “dark” noise (pepper noise) Reduce “bright” noise (salt noise)

27. Median Filter : How it works A median filter is good for removing impulse, isolated noise Degraded image Salt noise Pepper noise Moving window Sorted array Salt noise Pepper noise Median Filter output Normally, impulse noise has high magnitude and is isolated. When we sort pixels in the moving window, noise pixels are usually at the ends of the array. Therefore, it’s rare that the noise pixel will be a median value.

28. Median Filter : Example Image corrupted by salt-and-pepper noise with p a = p b = 0.1 Images obtained using a 3x3 median filter 1 4 2 3

29. Max and Min Filters: Example Image corrupted by pepper noise with prob. = 0.1 Image corrupted by salt noise with prob. = 0.1 Image obtained using a 3x3 max filter Image obtained using a 3x3 min filter

30. Alpha-trimmed Mean Filter where g r ( s , t ) represent the remaining mn - d pixels after removing the d /2 highest and d /2 lowest values of g ( s , t ). This filter is useful in situations involving multiple types of noise such as a combination of salt-and-pepper and Gaussian noise. Formula: Here d can range from 0 to (mn – 1). If d = 0, it reduces to Arithmetic mean filter. If d = mn – 1, it reduces to median filter.

31. Alpha-trimmed Mean Filter: Example Image corrupted by additive uniform noise Image obtained using a 5x5 arithmetic mean filter Image additionally corrupted by additive salt-and- pepper noise 1 2 2 Image obtained using a 5x5 geometric mean filter 2

32. Alpha-trimmed Mean Filter: Example (cont.) Image corrupted by additive uniform noise Image obtained using a 5x5 median filter Image additionally corrupted by additive salt-and- pepper noise 1 2 2 Image obtained using a 5x5 alpha- trimmed mean filter with d = 5 2

33. Alpha-trimmed Mean Filter: Example (cont.) Image obtained using a 5x5 arithmetic mean filter Image obtained using a 5x5 geometric mean filter Image obtained using a 5x5 median filter Image obtained using a 5x5 alpha- trimmed mean filter with d = 5

34. Adaptive Median Filter Adaptive median filter has three main purposes: To remove salt-and-pepper noise. To provide smoothing of other noise that may not be impulsive. To reduce distortion, such as excessive thinning or thickening of object boundaries. The algorithm discussed previously consider z min and z max to be “impulse-like” noise components.

35. Estimation of Degradation Model Degradation model: Purpose: to estimate h ( x , y) or H ( u, v ) Methods: 1. Estimation by Image Observation 2. Estimation by Experiment 3. Estimation by Modeling or Why? If we know exactly h ( x , y ), regardless of noise, we can do deconvolution to get f ( x , y ) back from g ( x , y ) .

36. Restoration Using Degradation Model The degradations are modeled as being result of convolution, and restoration seeks to find filters that apply process in reverse. Due to that reason, the term image deconvolution is used frequently to signify linear image restoration. The filters used in the restoration process often are called deconvolution function. The process of restoring an image by using a degradation function that has been estimated in some way sometimes is called blind deconvolution. It is due to the fact that true degradation function is seldom known completely.

37. Estimation by Image Observation We are given a degraded image without any knowledge about the degradation function H. One way to estimate the function H is to gather information from the image itself. We take a small section of the image with simple structures, where there are strong signal contents. Using the sample gray levels of the object and background, we can construct an un-blurred image of the same size and characteristics as the observed sub-image. Then we can extend the transfer function to the whole image.

38. Estimation by Image Observation f ( x , y ) f ( x , y )* h ( x , y ) g ( x , y ) Subimage Reconstructed Subimage DFT DFT Restoration process by estimation Original image (unknown) Degraded image Estimated Transfer function Observation This case is used when we know only g ( x , y ) and cannot repeat the experiment!

39. Estimation by Experimentation If the equipment similar to the equipment used to acquire the degraded image is available, it is possible in principle to obtain an accurate estimate of the degradation. Images similar to the degraded image can be acquired with various system settings until they are as closely as possible to the image we wish to restore. The idea is to obtain the impulse response of the degradation by imaging an impulse using the same system settings. An impulse is simulated by a bright dot of light, as bright as possible to reduce the effect of noise.

40. Estimation by Experimentation Used when we have the same equipment set up and can repeat the experiment . Input impulse image System H ( ) Response image from the system DFT DFT

41. Estimation by Modeling Used when we know physical mechanism underlying the image formation process that can be expressed mathematically. Atmospheric Turbulence model Example: Original image Severe turbulence k = 0.00025 k = 0.001 k = 0.0025 Low turbulence Mild turbulence K is a constant that depends on the nature of the turbulence. Proposed by Hufnagel and Stanley in 1964

42. Inverse Filter After we obtain H ( u , v ), we can estimate F ( u , v ) by the inverse filter: From degradation model: Noise is enhanced when H ( u , v ) is small. To avoid the side effect of enhancing noise, we can apply this formulation to frequency component ( u , v ) with in a radius D from the center of ( u , v ). In practice, the inverse filtering is not a popular technique.

43. Inverse Filter: Example Original image Blurred image Due to Turbulence Result of applying the full filter Result of applying the filter with D =70 Result of applying the filter with D =40 Result of applying the filter with D =85

44. Wiener Filter: Minimum Mean Square Error Filter The inverse filtering approach discussed so far makes no explicit provision for handling noise. The current approach incorporates both the degradation function and statistical characteristics of noise into the restoration process. The method is founded on considering images and noise as random processes.

45. Wiener Filter: Minimum Mean Square Error Filter The objective is to find an estimate of the uncorrupted f such that the mean square error between the original and the estimate is minimized. It is assumed that: the noise and the image are uncorrelated; one or the other has zero mean; the gray levels in the estimate are a linear function of the levels in the degraded image.

46. Wiener Filter: Minimum Mean Square Error Filter Objective: optimize mean square error: Wiener Filter Formula: where H ( u , v ) = Degradation function S h ( u , v ) = Power spectrum of noise S f ( u , v ) = Power spectrum of the un-degraded image

47. Approximation of Wiener Filter Wiener Filter Formula: Approximated Formula: Difficult to estimate Practically, K is chosen manually to obtained the best visual result!

48. Wiener Filter: Example Original image Blurred image Due to Turbulence Result of the full inverse filter Result of the inverse filter with D =70 Result of the full Wiener filter

49. Wiener Filter: Example (cont.) Original image Result of the inverse filter with D =70 Result of the Wiener filter Blurred image Due to Turbulence

50. Example: Wiener Filter and Motion Blurring Image degraded by motion blur + AWGN Result of the inverse filter Result of the Wiener filter s h 2 =650 s h 2 =325 s h 2 =130 Note: K is chosen manually

51. 51 Different restoration approaches Frequency domain Inverse filter Wiener (minimum mean square error) filter Algebraic approaches Unconstrained optimization Constrained optimization

52. 52 The block-circulant matrix Stacking rows of image f , g , n to make MN x 1 column vectors f , g , and n . (Also called lexicographic representation of the original image). Correspondingly, H should be a MN x MN matrix H is called block- circulant matrix

53. 53 Algebraic approach – Unconstrained restoration vs. Inverse filter Compared to the inverse filter:

54. 54 Algebraic approach – Constrained restoration vs. Wiener filter Compared to:

55. Improvement Wiener Filtering K = S n (u,v)/S f (u,v), Sn(u,v) = |N(u,v)| 2 Sf(u,v) = |F(u,v)| 2 S n (u,v) & S f (u,v) must be known S n (u,v) the power spectrum of the noise, S f (u,v) the power spectrum of the original image

56. Improvement – Cons. Constrained Least Squares Filtering P( u,v ) is the fourier transform of the Laplacian operator Constrain: |g – H | 2 = |η| 2 R( u,v ) = G( u,v ) – H( u,v ) Adjust γ from the constrain – by Newton- Raphson root-finding Apply algorithm from Prof. Hsien-Sen Hung