CV2_CH9 ppt x chapter wise notes from CV

chapter wise notes

1. Motion Estimation Presenter: Yu-Hsiang Shao ( 邵育翔 ) Phone: [phone] E-Mail : [email protected] Advisor: Prof. Chiou - Shann Fuh Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C. Advanced Computer Vision  Chapter 9

2. Slide credits Most of these slides were adapted from: Ioannis ( Yannis ) Gkioulekas  (16-385 Computer Vision, Carnegie Mellon University ). Delbert Gideon ( SlideServe ). 湯竣淋 (ACV, 2020)

3. Overview Image processing Geometry-based vision Physics-based vision Semantic vision Dealing with motion Chapter 9

4. 9.1 Translational alignment  9.2 Optical flow. Horn- Schunck flow. (HW3) 9.3 Parametric motion 9.4 Layered motion Overview of this lecture

5. 9.1 Translational Alignment

6. http://www.humansensing.cs.cmu.edu/intraface/

7. How can I find in the image?

8. Template Matching Alignment between two images or image patches A suitable error metric must first be chosen to compare the images Where they are located in

9. Translational Alignment Minimum of Sum of Squared Difference (SSD) Assumption: corresponding pixel values remains the same in the two images ---- Brightness constancy constraint u =( u,v ): displacement Residual error  (displaced frame difference)

10. Robust Error Metrics Squared Difference is senstive to outliers

11. Robust Error Metrics Robust norm of error ( Huber 1981; Hampel , Ronchetti , Rousseeuw et al. 1986; Black and Anandan 1996; Stewart 1999) Sum of Absolute Difference (L1 norm) Grows less quickly than the quadratic penalty associated with least squares E SAD is NOT differentiable at the origin, not well suited to gradient descent approaches SRD: sum of robust difference SAD: sum of absolue difference

12. Robust Error Metric Smoothly varying function (Black and Rangarajan (1996) ) Quadratic for small values but grows more slowly away from the origin Geman –McClure function a : constant that can be thought of as an outlier threshold   MAD: median absolute deviation 

13. Spatially Varying Weights Pixels that may lie outside of the boundaries Partially or completely downweight the contribution of certain pixels  Erase moving object for background alignment Weighted (or Windowed) SSD function w and w 1 are zero outside the image boundaries

14. Weighted SSD If a large range of potential motions is allowed, the above metric can have a bias towards smaller overlap solutions. Answer: W indowed SSD score can be divided by the overlap area  E WSSD / A Overlap area S SD: S um of Squared Difference RMS: Root Mean Square Root Mean Square:

15. Bias and Gain (Exposure Differences) For images being aligned were not taken with the same exposure Simple model of linear intensity variation --- Bias and Gain model Gain Bias

16. Bias and Gain Least Squares with Bias and Gain Use linear regression to estimate gain and bias Color image Estimate bias and gain for each color channel Weighted prediction in video codecs

17. Correlation Cross-Correlation ( Confused with convolution ) Alternative to taking intensity difference Maximize the product of two aligned images

18. Correlation Cross-Correlation vs.  Convolution

19. Correlation Cross-Correlation Alternative to taking intensity difference Maximize the product of two aligned images If a very bright patch exists,  the maximum product may actually lie in that area.  

20. Normalized Cross-Correlation NCC in [-1,1] Works well when matching images taken with different exposures Performance  degrades for noisy low-contrast regions (Zero variance) Mean images of the corresponding patches 

21. Normalized Cross-Correlation Normalized SSD score ( Criminisi, Shotton, Blake et al., 2007). Produce comparable results to NCC S SD: S um of Squared Difference

22. How can I find in the image?

23. Idea #1: Template Matching Slow, combinatory, global solution

24. Idea #2: Pyramid Template Matching Faster, combinatory, locally optimal

25. Not guaranteed to produce the same results as a full search, but works almost as well and much faster Steps At coarser levels, search over a smaller number of discrete pixels Motion estimation at coarse level is used to initialize a smaller local search at the next finer level Hierarchical Motion Estimation

26. Hierarchical Motion Estimation Problem of image pyramid? N ot possible to coarsen the representation too much before significant features are blurred away.

27. Fourier-based Alignment 

28. Fourier-based Alignment 

29. Fourier-based Alignment  Fourier transform

30. Fourier-based Alignment 

31. Fourier-based Alignment 

32. Fourier transform of a shifted signal has the same magnitude as the original signal but a linearly varying phase Convolution in the spatial domain corresponds to multiplication in the Fourier domain  Fourier-based Alignment 

33. Fourier-based Alignment  Accelerated by Fast Fourier Transform (FFT) "Convolution in the spatial domain corresponds  to multiplication in the Fourier domain" Divide and Conquer​ O( N 2 ) -> O(N log N ) * See proof: https://www.youtube.com/watch?v=M0Sa8fLOajA  

34. Fourier-based Alignment  Consider the SSD formula. Its Fourier transform can be written as Thus, the SSD function can be computed by taking twice the correlation function and subtracting it from the sum of the energies in the two images.  S SD: S um of Squared Difference

35. Windowed Correlation  Convolution is the whole picture, but only a small part needs to be compared The cross-correlation function should be replaced with a windowed (weighted) cross-correlation function The weighting functions  ω and  ω 1 are zero outside the image boundaries

36. Phase Correlation  The spectrum of the two signals being matched is whitened by dividing each per-frequency product by the magnitudes of the Fourier transforms In the case of noiseless signals, we have I 1 ( x + u ) = I ( x )   The output of phase correlation is a single impulse located at the correct value of u . (Kuglin and Hines 1975; Brown 1992)

37. Rotations and Scale Pure rotation: Re-sample the images into polar coordinates Rotation and Scale: Re-sample the images into log-polar coordinates While Fourier-based alignment is mostly used to estimate translational shifts between images, it can also be used to estimate in-plane rotations and scales.

38. How can I find in the image?

39. Some notation before we get into the math… 2D image transformation 2D image coordinate Parameters of the transformation Translation transform coordinate Affine Pixel value at a coordinate affine transform coordinate Warped image can be written in matrix form when linear affine warp matrix can also be 3x3 when last row is [0 0 1]

40. Affine transform affine transform coordinate

41. Image alignment (problem definition) warped image template image Find the warp parameters p such that the SSD is minimized How could you find a solution to this problem?

42. Hard to optimize What can you do to make it easier to solve? assume good initialization, linearized objective and update incrementally This is a non-linear (quadratic) function of a non-parametric function! (Function I is non-parametric)

43. Idea #3: Model refinement Fastest, locally optimal (when you have a good initial solution)

44. Lucas- Kanade alignment A n I terative I mage R egistration T echnique with an A pplication to S tereo V ision ,  1981.

45. If you have a good initial guess p … can be written as … (a small incremental adjustment) (pretty strong assumption) (this is what we are solving for now)

46. How can we linearize the function I for a really small perturbation of p ? Taylor series approximation! This is still a non-linear (quadratic) function of a non-parametric function! (Function I is non-parametric)

47. Multivariable Taylor Series Expansion (First order approximation)

48. Multivariable Taylor Series Expansion (First order approximation) Recall: chain rule short-hand short-hand

49. Linear approximation Multivariable Taylor Series Expansion (First order approximation) What are the unknowns here?

50. The Jacobian (A matrix of partial derivatives)

51. The Jacobian Affine transform Rate of change of the warp (A matrix of partial derivatives)

52. Linear approximation Multivariable Taylor Series Expansion (First order approximation) Now, the function is a linear function of the unknowns

53. pixel coordinate (2 x 1) image intensity (scalar) warp function (2 x 1) warp parameters (6 for affine) image gradient (1 x 2) Partial derivatives of warp function (2 x 6) incremental warp (6 x 1) template image intensity (scalar)

54. Summary (of Lucas-Kanade Image Alignment) Solve for increment Taylor series approximation Linearize Difficult non-linear optimization problem Assume known approximate solution Strategy: Problem: then solve for warped image template image

55. OK, so how do we solve this?

56. Another way to look at it… (moving terms around) Have you seen this form of optimization problem before? vector of variables vector of constants constant

57. Another way to look at it… How do you solve this? Looks like variable constant constant

58. Least squares approximation is solved by is optimized when where Applied to our tasks: after applying H :  Hessian matrix

59. The Hessian (A square matrix of second-order partial derivatives) Jacobian of ▽f

60. Solve for increment Taylor series approximation Linearize Difficult non-linear optimization problem Assume known approximate solution Strategy: Solve: warped image template image Solution: Solution to least squares approximation Hessian

61. Gauss-Newton gradient descent   non-linear optimization! This is called…

62. Warp image Compute error image Compute gradient Evaluate Jacobian Compute Hessian Compute Update parameters Lucas Kanade (Additive alignment) x ’coordinates of the warped image (gradients of the warped image)

63. Compositional alignment Creating Full View Panoramic Image Mosaics and Environment Maps ,  R. Szeliski, H. Y. Shum 1997.

64. Image Alignment (start with an initial solution, match the image and template)

65. Additive Alignment incremental perturbation of parameters Image Alignment Objective Function Given an initial solution…several possible formulations

66. Compositional Alignment incremental warps of image Image Alignment Objective Function Given an initial solution…several possible formulations Additive Alignment incremental perturbation of parameters

67. Additive strategy go back, adjust and try again first shot second shot

68. Compositional strategy start from here first shot second shot

69. W(x;0 + D p) = W(x; D p) T (x) W(x;p) W(x;p) o W(x; D p) Compositional I (x) T (x) Additive I (x) W( x;p ) W(x;p+ D p)

70. Compositional Alignment Assuming an initial solution p and a compositional warp increment Original objective function (SSD) Another way to write the composition Skipping over the derivation…the new update rule is Identity warp

71. So what’s so great about this compositional form?

72. The Jacobian is constant. Jacobian can be precomputed! linearized form linearized form Jacobian of W(x;p) Jacobian of W(x;0) Additive Alignment Compositional Alignment

73. Compositional Image Alignment Minimize W(x;0 + D p) = W(x; D p) T (x) W(x;p) W(x;p) o W(x; D p) Jacobian is simple and can be precomputed

74. Warp image Compute error image Compute gradient Evaluate Jacobian Compute Hessian Compute Update parameters Lucas Kanade (Additive alignment)

75. Warp image Compute error image Compute gradient Evaluate Jacobian Compute Hessian Compute Update parameters Shum-Szeliski (Compositional alignment)

76. Any other speed up techniques?

77. Inverse alignment

78. Baker-Matthews alignment Lucas- kanade 20 years on: A unifying framework,  2004.

79. Why not compute warp updates on the template? Additive Alignment Compositional Alignment What happens if you let the template be warped too? Inverse Compositional Alignment

80. W(x;0 + D p) = W(x; D p) T (x) W(x;p) W( x;p ) o W( x; D p ) T (x) W(x;0 + D p) = W(x; D p) W(x;p) Compositional Inverse compositional W( x;p ) o W( x; D p ) -1 I (x) I (x)

81. Compositional strategy start from here first shot second shot

82. Inverse Compositional strategy first shot move the hole

83. So what’s so great about this inverse compositional form?

84. Inverse Compositional Alignment Minimize Solution Update can be precomputed from template!

85. Properties of inverse compositional alignment Jacobian can be precomputed It is constant - evaluated at W(x;0) Gradient of template can be precomputed It is constant Hessian can be precomputed Warp must be invertible (main term that needs to be computed)

86. Warp image Compute error image Compute gradient Evaluate Jacobian Compute Hessian Compute Update parameters Lucas Kanade (Additive alignment)

87. Warp image Compute error image Compute gradient Evaluate Jacobian Compute Hessian Compute Update parameters Shum-Szeliski (Compositional alignment)

88. Warp image Compute error image Compute gradient Evaluate Jacobian Compute Hessian Compute Update parameters Baker-Matthews (Inverse Compositional alignment)

89. Algorithm Efficient Authors Forwards Additive No Lucas, Kanade Forwards compositional No Shum, Szeliski Inverse Additive Yes Hager, Belhumeur Inverse Compositional Yes Baker, Matthews

90. An Iterative Image Registration Technique with an Application to Stereo Vision. 1981 Lucas Kanade Detection and Tracking of Feature Points. 1991 Kanade Tomasi Good Features to Track. 1994 Tomasi Shi History of the Kanade-Lucas-Tomasi (KLT) Tracker The original KLT algorithm

91. Method for aligning (tracking) an image patch Kanade-Lucas-Tomasi Method for choosing the best feature (image patch) for tracking Lucas- Kanade Tomasi-Kanade How should we select features? How should we track them from frame to frame?

92. 9.2 Optical flow

93. optical flow

94. optical flow used for motion estimation in visual odometry

95. optical flow used for segments the background

96. Image Alignment Lucas-Kanade (Forward additive) Baker-Matthews (Inverse Compositional)

97. Tracking in Video KLT Mean shift Kalman Filtering SLAM

98. Optical Flow

99. Optical Flow Problem Definition Assumptions Brightness constancy Small motion Given two consecutive image frames, estimate the motion of each pixel

100. Optical Flow (Problem definition) Estimate the motion (flow) between these two consecutive images How is this different from estimating a 2D transform?

101. Key Assumptions (unique to optical flow) Color Constancy (Brightness constancy for intensity images) Small Motion (pixels only move a little bit) Implication: allows for pixel to pixel comparison (not image features) Implication: linearization of the brightness constancy constraint

102. Approach Look for nearby pixels with the same color (small motion) (color constancy)

103. Brightness constancy Assumption:Brightness of the point will remain the same constant Scene point moving through image sequence Assumption 1

104. Small motion Optical flow (velocities): Displacement: … the brightness between two consecutive image frames is the same For a really small space-time step … Assumption 2

105. total derivative partial derivative Equa tion is not obvious. Where does this come from? These assumptions yield the … Brightness Constancy Equation

106. For small space-time step, brightness of a point is the same Insight: If the time step is really small, we can linearize the intensity function

107. Multivariable Taylor Series Expansion (First order approximation, two variables)

108. assuming small motion divide by take limit Brightness Constancy Equation Multivariable Taylor Series Expansion (First order approximation, two variables)

109. shorthand notation vector form Brightness Constancy Equation (x-flow) (y-flow) (1 x 2) (2 x 1)

110. (putting the math aside for a second…) What do the term of the brightness constancy equation represent? flow velocities temporal gradient How do you compute these terms? Image gradients (at a point p)

111. spatial derivative temporal derivative Forward difference Sobel filter Derivative-of-Gaussian filter … How do you compute … frame differencing

112. Frame differencing 1 1 1 1 1 1 1 1 1 1 1 10 10 10 10 1 10 10 10 10 1 10 10 10 10 1 10 10 10 10 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 10 10 10 1 1 10 10 10 1 1 10 10 10 9 9 9 9 9 9 9 (example of a forward difference) - =

113. Example: 1 1 1 1 1 1 1 1 1 1 1 10 10 10 10 1 10 10 10 10 1 10 10 10 10 1 10 10 10 10 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 10 10 10 1 1 10 10 10 1 1 10 10 10 y x y x - - - - - 9 - - 9 - - 9 - - 9 - y x - - - - - 9 9 9 9 - - - - - - y x 9 9 9 9 9 9 9 y x 3 x 3 patch -1 0 1 -1 1

114. spatial derivative optical flow Forward difference Sobel filter Derivative-of-Gaussian filter … How do you compute … temporal derivative frame differencing How do you compute this? We need to solve for this! (this is the unknown in the optical flow problem)

115. Solution lies on a straight line The solution cannot be determined uniquely with a single constraint (a single pixel) many combinations of u and v will satisfy the equality

116. known unknown

117. Horn- Schunck Optical Flow (1981) Lucas- Kanade Optical Flow (1981) ‘constant’ flow (flow is constant for all pixels) ‘smooth’ flow (flow can vary from pixel to pixel) brightness constancy method of differences global method (dense) local method (sparse) small motion

118. Constant flow

119. Where do we get more equations (constraints)? Assume that the surrounding patch (say 5x5) has ‘ constant flow’

120. Flow is locally smooth Neighboring pixels have same displacement Using a 5 x 5 image patch, gives us 25 equations Assumptions:

121. Equivalent to solv ing : where the summation is over each pixel p in patch P

122. where the summation is over each pixel p in patch P Sometimes called ‘Lucas- Kanade Optical Flow’ (can be interpreted to be a special case of the LK method with a translational warp model) Equivalent to solv ing :

123. OpenCV Lucas- Kanade

124. When is this solvable?

125. When is this solvable? A T A should be invertible A T A should not be too small A T A should be well conditioned l 1 and l 2 should not be too small l 1 / l 2 should not be too large ( l 1 =larger eigenvalue)

126. Where have you seen this before? Harris Corner Detector! = What are the implications ?

127. Implications Corners are when λ1, λ2 are big; this is also when Lucas-Kanade optical flow works best Corners are regions with two different directions of gradient (at least) Corners are good places to compute flow! What happens when you have no ‘corners’?

128. Barber’s pole illusion

129. Barber’s pole illusion

130. Barber’s pole illusion

131. Aperture Problem In which direction is the line moving? small visible image patch

132. Aperture Problem In which direction is the line moving? small visible image patch

133. Aperture Problem

134. Aperture Problem

135. Aperture Problem

136. Aperture Problem

137. Want patches with different gradients to the avoid aperture problem

138. Want patches with different gradients to avoid aperture problem

139. 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 5 - - - - - 1 1 1 - 2 2 2 - 3 3 3 - 4 4 4 y x y x optical flow: (1,1) H(x,y) = y I(x,y) We recover the v of the optical flow but not the u. This is the aperture problem. Compute gradients Solution:

140. Horn- Schunck optical flow

141. Horn- Schunck Optical Flow (1981) Lucas- Kanade Optical Flow (1981) ‘constant’ flow (flow is constant for all pixels) ‘smooth’ flow (flow can vary from pixel to pixel) brightness constancy method of differences global method (dense) local method (sparse) small motion

142. Smoothness most objects in the world are rigid or deform elastically moving together coherently we expect optical flow fields to be smooth

143. Key idea (of Horn-Schunck optical flow) Enforce brightness constancy Enforce smooth flow field to compute optical flow

144. Key idea (of Horn-Schunck optical flow) Enforce brightness constancy Enforce smooth flow field to compute optical flow

145. For every pixel, lazy notation for Enforce brightness constancy

146. Key idea (of Horn-Schunck optical flow) Enforce brightness constancy Enforce smooth flow field to compute optical flow

147. Enforce smooth flow field u-component of flow

148. Which flow field optimizes the objective? ?

149. small big Which flow field optimizes the objective?

150. Key idea (of Horn-Schunck optical flow) Enforce brightness constancy Enforce smooth flow field to compute optical flow bringing it all together…

151. Horn-Schunck optical flow smoothness brightness constancy weight

152. HS optical flow objective function Brightness constancy Smoothness

153. How do we solve this minimization problem?

154. Compute partial derivative, derive update equations (gradient de s cent!) How do we solve this minimization problem?

155. Compute the partial derivatives of this huge sum! smoothness term brightness constancy

156. Compute the partial derivatives of this huge sum! it’s not so bad… how many u terms depend on k and l? ONE from brightness constancy FOUR from smoothness

157. Compute the partial derivatives of this huge sum! Smoothness - Square of the magnitude of the gradient - Sum of the squares of the Laplacians Approximations of the Laplacian:  

158. Compute the partial derivatives of this huge sum! (variable will appear four times in sum)

159. Compute the partial derivatives of this huge sum! short hand for local average (variable will appear four times in sum)

160. Where are the extrema of E? (set derivatives to zero and solve for unknowns u and v)

161. Where are the extrema of E? (set derivatives to zero and solve for unknowns u and v) how do you solve this? this is a linear system

162. ok, take a step back, why are we doing all this math?

163. We are solving for the optical flow (u,v) given two constraints smoothness brightness constancy We need the math to minimize this (back to the math)

164. Where are the extrema of E? (set derivatives to zero and solve for unknowns u and v) how do you solve this? Partial derivatives of Horn-Schunck objective function E:

165. Recall

166. Recall Adjugate matrix : = Determinant :

167. Recall (det A) Same as the linear system: (det A) (d et A) =

168. Rearrange to get update equations: new value old average

169. new value old average When lambda is small (lambda inverse is big)… Recall:

170. new value old average When lambda is small (lambda inverse is big)… goes to zero goes to zero …we only care about smoothness. Recall:

171. We are solving for the optical flow (u,v) given two constraints smoothness brightness constancy We needed the math to minimize this (now to the algorithm)

172. Horn-Schunck Optical Flow Algorithm Precompute image gradients Precompute temporal gradients Initialize flow field While not converged Compute flow field updates for each pixel:

173. HW3 due June 8, 2021 Implementing Horn &  Schunck  optical flow estimation as above​ Synthetically translate lena.bmp one pixel to the right and downward​ Try

174. HW3 Precompute image gradients Precompute temporal gradients

175. HW3 1 pixel

176. HW3

177. HW3

178. HW3 3.   Initialize flow field 4. While not converged   Compute flow field updates for each pixel: 32 iterations 64 iterations

179. HW3

180. HW3 Iteration = 64 Iteration = 16 Iteration = 4 Iteration = 1 Scale = 10

181. OpenCV Optical Flow Algorithm https://docs.opencv.org/3.4/d4/dee/tutorial_optical_flow.html

182. Rolling shutter wobble removal

183. Rolling shutter wobble removal To compensate for these distortion, which are caused by different exposure times for different scanlines, accurate per-pixel optic flow must be estimated. (Section 9.2.1) Modeling rolling shutter is also important to obtain accurate pose estimates in structure from motion

184. Deep Learning https://www.youtube.com/watch?v=OSEuYBwOSGI ECCV 2020 Best Paper Award | RAFT: A New Deep Network Architecture For Optical Flow |

185. Deep Learning https://www.youtube.com/watch?v=OSEuYBwOSGI ECCV 2020 Best Paper Award | RAFT: A New Deep Network Architecture For Optical Flow |

186. 9.3 Parametric motion

187. Spline-based Motion Traditionally, optical flow algorithms compute an independent motion estimate for each pixel. Alternative approach is to represent the motion field as a two-dimensional spline controlled by a smaller number of control vertices Traditionally, optical flow algorithms compute an independent motion estimate for each pixel. Alternative approach is to represent the motion field as a two-dimensional spline controlled by a smaller number of control vertices Spline-Based Image Registration, R. Szeliski , J. Coughlan, 1997

188. control vertices basis functions  Two-dimensional spline controlled by a smaller number of control vertices ​

189. control vertices basis functions

190. control vertices basis functions

191. Undistorted Barrel distortion Pincushion distorti on

192. https://www.youtube.com/watch?v=yAB6OQsHOyI

193. 9.4 Layered Motion

194. Layered Motion In many situations, visual motion is caused by the movement of a small number of objects at different depths in the scene.

195. Layered Motion In many situations, visual motion is caused by the movement of a small number of objects at different depths in the scene.

196. Layered Motion In many situations, visual motion is caused by the movement of a small number of objects at different depths in the scene.

197. Layered Motion In many situations, visual motion is caused by the movement of a small number of objects at different depths in the scene.

198. Layered motion representations not only lead to compact   representations but they also exploit the information available   in multiple video frames, as well as accurately modeling the   appearance of pixels near motion discontinuities.  

199. Layered Motion How to compute layered representation of a video?  Estimate affine motion models over a collection of non-overlapping patches  Cluster the estimates using K-means  Alternate between assigning pixels to layers and recomputing motion estimates for each layer using the assigned pixels Layers are constructed by warping and merging the various layer   pieces from all of the frames together.   Wang and Adelson (1994)

200. Layered Motion Baker, Szeliski, and Anandan (1998)

201. Transparent Layers and Reflections Reflection in windows, picture frames, … Reflection Model how much intensity each layer contributed to the final image Glass surface Image transmission reflection

202. 11 Reflection Transparent motion separation The amount of reflected light is quite low compared to the transmitted light (the picture of the girl) and yet the algorithm is still able to recover both layers.

203. Video object tracking One of the most widely used applications of computer vision to video analysis is video object tracking https://www.youtube.com/watch?v=LjjJQ81RbX0

204. 10 Questions What is the assumption of the Lucas- Kanade alignment method? P47 Tell the flowchat of Lucas Kanade alignment (Additive alignment)? P64 What’s different between Additive alignment and inverse compositional alignment? P87 What are the assumptions of the Optical flow? P101 What happens when you have no ‘corners’ in optical flow? p130-140

205. 10 Questions What’s the topic of HW3? P142 What’s different between Horn- Schunck optical flow and Lucas- Kanade optical flow? P143 What are two key ideas for Horn- Schunck optical flow? P146 When is the deadline for hw3? P175 What are the applications of Layer motion (Give 3 examples)? P195 - 205

206. 10 Answers If you have a good initial guess Ans : Warp image Compute error image Compute gradient Evaluate Jacobian Compute Hessian Compute Update parameters Ans : Jacobian can be precomputed Gradient of template can be precomputed Hessian can be precomputed Warp must be invertible

207. 10 Answers Ans : Brightness constancy Small motion Ans : Barber’s pole illusion Aperture Problem Horn- Schunck optical flow Smooth flow vs Constant flow

208. 10 Answers Ans : Brightness constancy Smooth flow field June 8, 2021 Ans : segments the background Transparent Layers and Reflections Video object tracking Video object segmentation