Linear Algebra and Matrices used for ML.ppt

Linear Algebra and Matrices

1. Linear Algebra and Matrices
J. Mercy Arokia Rani
Assistant Professor
Department of Mathematics
St. Joseph’s College (Autonomous)
Trichy – 620 002

2. Talk Outline
•Scalars, vectors and matrices
•Vector and matrix calculations
•Identity, inverse matrices &
determinants
•Solving simultaneous equations
•Relevance to SPM
Linear Algebra & Matrices, MfD 2009

3. Scalar
•Variable described by a single number
e.g. Intensity of each voxel in an MRI scan
Linear Algebra & Matrices, MfD 2009

4. Vector
•Not a physics vector (magnitude, direction)
•Column of numbers e.g. intensity of same voxel at
different time points
Linear Algebra & Matrices, MfD 2009










3
2
1
x
x
x

5. Matrices
•Rectangular display of vectors in rows and columns
•Can inform about the same vector intensity at different
times or different voxels at the same time
•Vector is just a n x 1 matrix











476
145
321
A
Square (3 x 3)Rectangular (3 x 2) d
i j : i
th
row, j
th
column
Defined as rows x columns (R x C)











83
72
41
C











333231
232221
131211
ddd
ddd
ddd
D
Linear Algebra & Matrices, MfD 2009

6. Matrices in Matlab
•X=matrix
•;=end of a row
•:=all row or column










987
654
321
Subscripting – each element of a matrix
can be addressed with a pair of
numbers; row first, column second
(Roman Catholic)
e.g. X(2,3) = 6
X(3, :) =
X( [2 3], 2) =

 987








8
5
“Special” matrix commands:
• zeros(3,1) =
• ones(2) =
• magic(3) =








11
11










0
0
0
Linear Algebra & Matrices, MfD 2009










294
753
618

7. Design matrix
=
















+
= +Y X
d
a
t a

v e c t o
r
d
e s i g n

m
a t r i x
p
a
r a
m
e t e r s
e r r o
r
v e c t o
r

=
t h
e b
e t a
s
( h
e r e : 1
t o
9
)
Linear Algebra & Matrices, MfD 2009

8. Transposition











2
1
1
b
 211
T
b  943d
column row row column











476
145
321
A











413
742
651
T
A
Linear Algebra & Matrices, MfD 2009











9
4
3
T
d

9. Matrix Calculations
Addition
–Commutative: A+B=B+A
–Associative: (A+B)+C=A+(B+C)
Subtraction
- By adding a negative matrix





























65
43
1532
0412
13
01
52
42
BA
Linear Algebra & Matrices, MfD 2009

10. Scalar multiplication
•Scalar x matrix = scalar multiplication
Linear Algebra & Matrices, MfD 2009

11. Matrix Multiplication
“When A is a mxn matrix & B is a kxl matrix,
AB is only possible if n=k. The result will be
an mxl matrix”
A
1 A
2 A
3
A
4 A
5 A
6
A
7 A
8 A
9
A
10 A
11 A
12
m
n
x
B
13 B
14
B
15 B
16
B
17 B
18
l
k
Number of columns in A = Number of rows in B
= m x l matrix
Linear Algebra & Matrices, MfD 2009

12. Matrix multiplication
•Multiplication method:
Sum over product of respective rows and columns

10
23





 







13
12








2221
1211
cc
cc








1)(31)(23)(32)(2
1)(01)(13)(02)(1








513
12
X =
=
=
Define output
matrix
A B
• Matlab does all this for you!
• Simply type: C = A * B
Linear Algebra & Matrices, MfD 2009

13. Matrix multiplication
•Matrix multiplication is NOT commutative
•AB≠BA
•Matrix multiplication IS associative
•A(BC)=(AB)C
•Matrix multiplication IS distributive
•A(B+C)=AB+AC
•(A+B)C=AC+BC
Linear Algebra & Matrices, MfD 2009

14. Vector Products
Inner product X
T
Y is a scalar
(1xn) (nx1)
Outer product XYOuter product XY
TT
is a matrix is a matrix
(nx1) (1xn) (nx1) (1xn)

xy
T

x
1
x
2
x
3










y
1y
2y
3 
x
1y
1x
1y
2x
1y
3
x
2y
1x
2y
2x
2y
3
x
3
y
1
x
3
y
2
x
3
y
3










 
i
i
i
T
yxyxyxyx
y
y
y
xxx 













3
1
332211
3
2
1
321yx
Inner product = scalar
Two vectors:











3
2
1
x
x
x
x











3
2
1
y
y
y
y
Outer product = matrix
Linear Algebra & Matrices, MfD 2009

15. Identity matrix
Is there a matrix which plays a similar role as the number 1 in
number multiplication?
Consider the nxn matrix:
For any nxn matrix A, we have A I
n
= I
n
A = A
For any nxm matrix A, we have I
n
A = A, and A I
m
= A (so 2 possible matrices)
Linear Algebra & Matrices, MfD 2009

16. Identity matrix
112233 110000 1+0+01+0+0 0+2+00+2+0 0+0+30+0+3
445566 XX 001100==4+0+04+0+0 0+5+00+5+0 0+0+60+0+6
778899 000011 7+0+07+0+0 0+8+00+8+0 0+0+90+0+9
Worked
example
A I
3
= A
for a 3x3 matrix:
• In Matlab: eye(r, c) produces an r x c identity matrix
Linear Algebra & Matrices, MfD 2009

17. Matrix inverse
•Definition. A matrix A is called nonsingular or invertible if
there exists a matrix B such that:
•Notation. A common notation for the
inverse of a matrix A is A
-1
. So:
• The inverse matrix is unique when it exists. So if A is invertible, then
A
-1
is also invertible and then (A
T
)
-1
= (A
-1
)
T
1111XX
2 2
33
-1-1

33
==
22 + + 11
3 33 3
-1-1 + + 11
3 33 3
==1100
-1-122
1 1
33
1 1
33
-2-2+ + 22
3 3
33
11 + + 22
3 3 3 3
0011
• In Matlab: A
-1
= inv(A) •Matrix division: A/B= A*B
-1
Linear Algebra & Matrices, MfD 2009

18. Matrix inverse
•For a XxX square matrix:
•The inverse matrix is:
Linear Algebra & Matrices, MfD 2009
•E.g.: 2x2 matrix

19. Determinants
• Determinants are mathematical objects that are very useful in the
analysis and solution of systems of linear equations (i.e. GLMs).
• The determinant is a function that associates a scalar det(A) to every
square matrix A.
–Input is nxn matrix
–Output is a single
number (real or
complex) called the
determinant
Linear Algebra & Matrices, MfD 2009

20. Determinants
•Determinants can only be found for square matrices.
•For a 2x2 matrix A, det(A) = ad-bc. Lets have at closer look at that:
• A matrix A has an inverse matrix A
-1
if and only if det(A)≠0.
• In Matlab: det(A) = det(A)
Linear Algebra & Matrices, MfD 2009
a b
c d
det(A) = = ad - bc
[ ]

21. Solving simultaneous equations
For one linear equation ax=b where the unknown is x and a
and b are constants,
3 possibilities:
If 0a then ba
a
b
x
1
 thus there is single solution
If 0a, 0b then the equation bax becomes 00
and any value of x will do

If 0a, 0bthen baxbecomes b0 which is a
contradiction
Linear Algebra & Matrices, MfD 2009

22. With >1 equation and >1 unknown
•Can use solution from the single
equation to solve
•For example
•In matrix form
bax
1

12
532
21
21


xx
xx
A X = B




















 1
5
21
32
2
1
x
x
X =A
-1
B
Linear Algebra & Matrices, MfD 2009






4
1

23. •X =A
-1
B
•To find A
-1
•Need to find determinant of matrix A
•From earlier
(2 -2) – (3 1) = -4 – 3 = -7
•So determinant is -7
bcad
dc
ba
A )det(






21
32
Linear Algebra & Matrices, MfD 2009

A
1

1
det(A)
db
ca







24. 


















21
32
7
1
21
32
)7(
1
1
A
1
2
7
14
7
1
4
1
21
32
7
1























X






4
1
if B isif B is
SoSo
bax
1

Linear Algebra & Matrices, MfD 2009

25. Linear Algebra & Matrices, MfD 2009
How are matrices relevant
to fMRI data?

26. NormalisationNormalisation
Statistical Parametric MapStatistical Parametric Map
Image time-seriesImage time-series
Parameter estimatesParameter estimates
General Linear ModelGeneral Linear ModelRealignmentRealignment SmoothingSmoothing
Design matrix
AnatomicalAnatomical
referencereference
Spatial filterSpatial filter
StatisticalStatistical
InferenceInference
RFTRFT
p <0.05p <0.05
Linear Algebra & Matrices, MfD 2009

27. Linear Algebra & Matrices, MfD 2009
T
i
m
e
BOLD signal
T
i
m
e
single voxel
time series
Voxel-wise time series analysis
Model
specificatio
n
Parameter
estimation
Hypothesis
Statistic
SPM

28. How are matrices relevant to fMRI data?
=
















+
= +Y X
d
a
t a

v e c t o
r
d
e s i g n

m
a t r i x
p
a
r a
m
e t e r s
e r r o
r
v e c t o
r

Linear Algebra & Matrices, MfD 2009
GLM equation

29. How are matrices relevant to fMRI data?
Y
d
a
t a

v e c t o
r
Response variable
e.g BOLD signal at a particular
voxel
A single voxel sampled at
successive time points.
Each voxel is considered as
independent observation.
Preprocessing .
..
Intens
ity
T
i
m
e
Y
T
i
m
e
Y = X . β + ε
Linear Algebra & Matrices, MfD 2009

30. How are matrices relevant to fMRI data?
















X
d
e s i g
n

m
a
t r i x
p
a
r a
m
e t e r s

Explanatory variables
–These are assumed to be
measured without error.
–May be continuous;
–May be dummy,
indicating levels of an
experimental factor.
Y = X . β + ε
Solve equation for β – tells us
how much of the BOLD signal is
explained by X
Linear Algebra & Matrices, MfD 2009

31. In Practice
•Estimate MAGNITUDE of signal changes
•MR INTENSITY levels for each voxel at
various time points
•Relationship between experiment and
voxel changes are established
•Calculation and notation require linear
algebra
Linear Algebra & Matrices, MfD 2009

32. Summary
•SPM builds up data as a matrix.
•Manipulation of matrices enables
unknown values to be calculated.
Y = X . β +
ε
Observed = Predictors * Parameters + Error
BOLD = Design Matrix * Betas + Error
Linear Algebra & Matrices, MfD 2009

33. References
•SPM course http://www.fil.ion.ucl.ac.uk/spm/course/
•Web Guides
http://mathworld.wolfram.com/LinearAlgebra.html
http://www.maths.surrey.ac.uk/explore/emmaspages/
option1.html
http://www.inf.ed.ac.uk/teaching/courses/fmcs1/
(Formal Modelling in Cognitive Science course)
•http://www.wikipedia.org
•Previous MfD slides
Linear Algebra & Matrices, MfD 2009

34. Thanking You….!!!!!