SURF - Speeded Up Robust Features

Computer Vision

1. SURF:
Speeded Up
Robust Features
Computer Vision

2. 1.
Introduction

3. Why is feature/image
matching important?
Used for:
▪Camera calibration
▪3D reconstruction
▪Image registration
▪Object recognition
▪more...

3

4. How do we search for discrete image
correspondences?
First Step
‘Interest Points’ are
selected at distinctive
locations in the image,
such as corners, blobs,
and T-junctions
Second Step
The neighbourhood of
every interest point is
represented by a
feature vector.
Third Step
The descriptor vectors
are matched between
different images.
This matching is often
based on a distance
between the vectors
(e.g. Mahalanobis or
Euclidean distance)
4

5. Main Challenges
Types of variance:
▪Illumination
▪Scale
▪Rotation
▪Affine
▪Perspective

Achieving:
▪Repeatability
▪Distinctiveness

While the
Detector/Descriptor:
▪Is faster to
compute
▪Maintains good
performance
5

6. 2.
What is SURF?

7. Approximates or even outperforms previously proposed
schemes with respect to repeatability, distinctiveness, and
robustness, yet can be computed and compared much faster.

Achieved by:
- relying on integral images for image convolutions
- building on the strengths of the leading existing detectors
and descriptors (using a Hessian matrix-based measure for the
detector, and a distribution-based descriptor)
- simplifying these methods to the essential

This leads to a combination of novel detection, description,
and matching steps.
7

8. 3.
Related Work

9. Interest Point Detection
▪Harris corner detector
▫Most widely used; based on the eigenvalues; not
scale-invariant
▪Automatic scale detection detector
▫Experimented both the determinant of the Hessian matrix
as well as Laplacian
▪Scale-invariant feature detectors
▫Harris-Laplace and Hessian-Laplace; The location is
selected by the determinant of Hessian matrix; The scale is
selected by the Laplacian

▪SIFT - most appealing descriptor for practical uses
▫Approximated the LoG by a DoG filter
9

10. New detector-descriptor scheme
10
Fast-Hessian
detector

Based on the
Hessian matrix but a
very basic
approximation - DoG
+
Integral image
Match

Present a new
indexing step
based on the sign
of the Laplacian
(Speed up and
increase the
robustness)
SURF Descriptor

Describes a
distribution of
Haar-wavelet
responses within the
interest point
neighborhood
+
Integral image

11. 4.
SURF
DETECTION AND DESCRIPTION

12. 4.1.
SURF Detection
FAST-HESSIAN DETECTOR

13. Hessian Matrix H(x, σ) in x at scale σ is defined as:

13
x = (�,�)
σ : scale (Stan drad derivation of
Gaussian)

with the image I in point x

Gaussian second order derivative

y-dir xy-dir

14. Approximation LoG with box filters => DoG

14
Box filters (instead of Gaussian)

y-dir xy-dir
9x9 box filter with σ =1.2

15. Instead of iteratively reducing the image size (left), the use of
integral images allows the up-scaling of the filter at constant
cost (right)
15
The scale space is divided into octaves. An octave represents a
series of increasing filter response maps.
It is selected as the interest point only if it
is larger than all of the neighbors.
For each new octave, the filter size
increase is doubled.

16. Scale space interpolation is especially important in our case, as the
difference in scale between the first layers of every octave is
relatively large.
16
In the figure we can see the detected interest points for a Sunflower
field. This kind of scenes shows clearly the nature of the features from
Hessian-based detectors.

17. 4.2.
SURF Descriptor
Orientation Assignment & Descriptor Components

18. SURF Descriptor
18
Features
Interest
Points
Orientation
Assignment
Fixing a reproducible
orientation based on
information from a
circular region around
the interest point.
Descriptor
Components
Construct a square
region aligned to the
selected orientation,
and extract the SURF
descriptor from it.

19. Orientation Assignment
▪The Haar wavelet responses are represented
as vectors;
▪Sum all responses within a sliding orientation
window covering an angle of 60º;
▪The longest vector is the dominant orientation.
19

20. Orientation Assignment - Haar wavelet filters
▪We can use integral images for fast filtering.
The used filters are shown in the image below.
Only six operations are needed to compute the
response in x or y direction at any scale
20
Haar wavelet filters to compute the responses in x (left) and y direction
(right). The dark parts have the weight -1 and the light parts +1.

21. Descriptor Components
▪Split the interest region (20s x 20s) up into 4x4
square sub-regions (example on the left);
▪Calculate Haar wavelet response d
x
and d
y
and
weight the response with a Gaussian kernel;
▪Sum the response over each sub-region for d
x

21

22. Descriptor Components - Properties
▪The image below shows the properties of the
descriptor for three distinctively different image
intensity patterns, resulting in a distinctive descriptor
22
The descriptor entries of a sub-region represent the nature of the
underlying intensity pattern.

23. Matching stage
▪For fast indexing during this stage, the sign of the
Laplacian for the underlying interest point is
included;
▪Features are only compared if they have the same
type of contrast - this minimal information allows
for faster matching and gives a slight increase in
performance
23

24. 5.
Experimental Results

25. Standard Evaluation
▪Using images with real textures and structured scenes.
▪Two viewpoint changes (Graffiti and Wall), one zoom and
rotation (Boat) and lighting changes (Leuven)
▪The detectors are evaluated using repeatability score
that indicates how many of the detected interest points
are found in both images, relative to the lowest total
number of interest points found
▪The descriptors are evaluated using recall-(1-precision)
graphs.
▪SURF is compared to GLOH, SIFT and PCA-SIFT
25

26. 5.1
Detector Results

27. Object Recognition
▪Recognising 22 objects in an art museum with a test set
of 116 images taken under various and sometimes
extreme conditions.
27
Number of detected points and calculation time for the detectors
Average recognition rates:
SURF-128 - 85.7%; U-SURF - 83.8%; SURF - 82.6%
GLOH - 78.3%; SIFT - 78.1%; PCA-SIFT - 72.3%

28. 28
▪Test keypoint repeatability for:
▫Viewpoint Change
▫Lighting Change
▫Zoom and Rotation

29. 29
Repeatability score:

30. 5.2
Descriptor Results

31. ▪SURF outperforms the other descriptors for almost all
comparisons.
31
▪The SURF descriptor outperforms the other descriptors in
a systematic and significant way, with sometimes more
than 10% improvement in recall for the same level of
precision. At the same time, it is fast to compute.

32. 6.
Conclusion

33. -SURF is a fast and performant interest point
detection-description scheme;
-SURF outperforms the current state-of-the art
both in speed and accuracy;
-SURF descriptor is easily extendable for the
description of affine invariant regions
-Future work will aim at optimising the code for
additional speed up
-A binary of the latest version is available on the
internet
33

34. THANKS!
Any questions?
34