stereo_cameras stereoscopic viewing and stereo algorithms

stereoscopic viewing and stereo algorithms

1. A Stereo Image

2. The space of all stereo images
The space/geometry of
all stereo/epipolar images/cameras
The space (geometry) of ray sets (cameras)
that allow row-based stereo analysis
or
•S. Seitz, J. Kim, The Space of All Stereo Images, IJCV 2002 / ICCV 2001.
•T. Pajdla, Epipolar Geometry of Some Non-Classical Cameras, Slovenian
Computer Vision Winter Workshop, 2001.

3. Outline
1) Regular stereo
2) Generalized / non-classical
camera
4) When can we compute stereo
from generalized cameras?
3) Stereo with generalized
cameras (an example)

4. A Stereo Image

5. Stereoscopic illusion
Real world Stereo Illusions

6. Stereoscopic illusion

7. Stereoscopic Imaging
Key property: horizontal parallax
•Enables stereoscopic viewing and stereo algorithms
Thank you, Steve Seitz

8. Capturing stereo pairs
Left
camera
Right
camera

9. Display different image for each eye
1) Separating with vertical paper
2) Color glasses
4)Temporally synchronized
screen and glasses
3) Polarized glasses 0

10. Parallax and disparity
Z L
R
Z=1
Z=1
Z=2
Z=2
L
R
R
L
Original images Aligned images
Z=0
Z=0
1
row
Disparity (1D)Parallax (3D)
Alignment  Measure depth with respect
to this plane

11. Displaying disparity
depth
disparity
1


12. Stereo Algorithms
Photogrammetry (generating maps)

13. Stereoscopic Imaging
Key property: horizontal parallax
•Enables stereoscopic viewing and stereo algorithms
Thank you, Steve Seitz

14. Rectification
or
Why we can focus on 1-row
Line corresponds to line (“epipolar lines”)
Rectified images: epipolar lines are image rows
?
?
Homography

15. Rectification (cont.)

16. Outline
1) Regular stereo
2) Generalized / non-classical
camera
4) When can we compute stereo
from generalized cameras?
3) Stereo with generalized
cameras (an example)

17. Geometric camera model
Mapping:Mapping: world points
We model only the “projection” operation
We do not model: light, color, lens blurring, etc.
 image points (pixels)
We model camera as

18. Example: pinhole camera model
(usual camera)
This projection operation is commonly described
by a 3x4 projective matrix.
Important property:
every ray from the ray set of the camera
projects to one point
A pinhole camera is defined by:
• set of rays (starting from the camera center)
• mapping from this rays to the image plane
For our purposes the following is more convenient:

19. Generalization of the camera model

MultipleMultiple centers of projection
(origins of rays)
Image surface is arbitraryarbitrary

OneOne center of projection
Image surface (film) is planarplanar
Classical camera
Generalized (ray-projective) camera
A generalized camera maps rays to image points
For our purposescamera  set of rays

20. Xerox machine
(non-classical cameras, example 1)
As a multi-prospective camera:

21. Existing Immersive Technology: Imax
®
Can Not Combine Full FOV and Stereopsis
•Imax 3D
•Incredibly realistic three-dimensional
images are projected
onto the giant
IMAX screen with such realism that
you can hear the audience gasp as
they reach out to grab the almost
touchable images.
•Imax Dome
•Experience wraps the audience in
images of unsurpassed size and
impact, providing an amazing sense
of involvement.
thanks to Shmuel Peleg, Hebrew University of Jerusalem

22. Pushbroom camera
(non-classical cameras, example 2)
•1D projective sensor
•… translating
Advantage: large field of
view in one dimension

23. Pushbroom camera
(non-classical cameras, example 2)
The generalized camera model:
X direction - parallel projection
Y direction - perspective projection
Notion GeneratorGenerator – the set of all ray origins
x
y
For other cameras, generator can be a 2D surface

24. t t+1 t+2
Pushbroom camera
(non-classical cameras, example 2)
Imaging process
thanks to Shmuel Peleg, Hebrew University of Jerusalem
Virtual generalized camera = device (usual camera) + software
Images from usual camera
Image from generalized camera

25. Non-classical cameras
•Implementation through cuts of
3D video arrays
•Take images while moving a usual camera
•Stack them into 3D array
•Take a cut along the “time” dimension
t

26. Move “1D sensor” along a circle
record on a cylinder
Circular projective camera
(non-classical cameras, example 3)
Advantage: complete 360° horizontal view
thanks to Shmuel Peleg, Hebrew University of Jerusalem

27. Circular projective camera
(non-classical cameras, example 3)
Note: Generator is a circle
The generalized camera model:

28. Outline
1) Regular stereo
2) Generalized / non-classical
camera
4) When can we compute stereo
from generalized cameras?
3) Stereo with generalized
cameras (an example)

29. Generalized stereo: an example
Inward-facing camera, moving around an object
thanks to Steve Seitz, University of Washington

30. Images for both eyes
•Output: 2 symmetric cuts of 3D video array
thanks to Steve Seitz, University of Washington
•Input: video sequence

31. Results: red-blue stereo image
thanks to Steve Seitz, University of Washington

32. Results: 3D reconstruction
thanks to Steve Seitz, University of Washington
Using usual algorithm (built for usual camera)
with non-classical images

33. How does the set of rays look?
There is a “blind” area in the
center of the scene
Pixel = ray
Ray geometry

34. Ray geometry
What rays go through a point in the scene?
How disparity depends on depth?

35. Outline
1) Regular stereo
2) Generalized / non-classical
camera
4) When can we compute stereo
from generalized cameras?
3) Stereo with generalized
cameras (an example)

36. Parallax and disparity in Cyclographs
(review)
L
R
Choose “ground” plane Z=0
PARRALAX / DISPARITY
L R
Z=1
Z=2
Z=0

37. The generalized camera model
D - image surface; P – ray space
 A view (camera) is a function V: DP
D Does not include:
• Multiple rays to 1-image point
• Curved light paths (mirror, lens)

38. A row (row-continuity)
A row is the set of points in one view image that
corresponds to a ray of the other view.
For rectified images: rows are image rows
Infinitesimality:
 Row has width  0
 Row has no holes

39. Stereo: basic constraints
Rays V(V(uu
11,v,v
11)) and V(V(uu
22,v,v
22)) intersect
vv
11
=vv
22
DD
11
DD
22
((uu
11,v,v
11))
((uu
22,v,v
22))
r
o
w

v

v

=
vv
11
ro
w
v v = vv 22

40. Basic stereo constraints + row-continuity
Surfaces V(V(*,v*,v
11) ) and V(V(*,v*,v
22)) , where vwhere v
11,, and vv
2 2 are
corresponding rows,
intersect in a surface(not a curve)
DD
11
DD
22
((uu
11,v,v
11))
((uu
22,v,v
22))
row v v =vv
11
ro
w
v v = vv 22

41. Example: The intersection of epipolar planes
The red plane ”intersects” the blue plane

42. Ruled surfaces:

43. Ruled surfaces: examples
•Generalized cylinder
Generalized cone

44. The most important slide
Doubly ruled surface
•Left camera “ruling” the scene
•Right camera “ruling” the scene

45. Doubly Ruled surfaces: examples
A plane
A hyperboloid

46. Theorem (D. Hilbert ):
The only doubly ruled surfaces are:
hyperboloid hyperbolic paraboloid plane

47. The Doubly Ruled Surfaces
of cyclograph

48. SUMMARY
The space (geometry)
of ray sets (cameras)
that allow row-based
stereo analysis
are doubly ruled surfaces

49. OmniStereo with Mirrors
Dynamic Scenes
viewing circle
optical center
mirror

50. Stereo Image
viewing circle
Optical center
Spiral mirror acquiring
right eye panorama
Spiral mirror acquiring
left eye panorama

51. Stereo Image