Linear and Genralized linear modeling (GLm)

Generalized Linear Modeling (GLM) for analysis, modeling, or prediction. This can include:

- Regression analysis data
- Statistical modeling output
- Research data for GLM applications
- Data for predictive modeling using GLM
- Any other file relevant to Generalized Linear Modeling

Accepted file formats: CSV,

1. General Linear
Model
Beatriz Calvo
Davina Bristow

2. Overview
Summary of regression
Matrix formulation of multiple regression
Introduce GLM
Parameter Estimation
Residual sum of squares
GLM and fMRI
fMRI model
Linear Time Series
Design Matrix
Parameter estimation
Summary

3. Summary of Regression
Linear regression models the linear relationship between a
single dependent variable, Y, and a single independent
variable, X, using the equation:
Y = βX + c + ε
The regression coefficient, β, reflects how much of an
effect X has on Y
ε is the error term and is assumed to be independently,
identically, and normally distributed (mean 0 and variance
σ
2
)

4. Summary of Regression
Multiple regression is used to determine the effect of a
number of independent variables, X
1
, X
2
, X
3
etc, on a
single dependent variable, Y
The different X variables are combined in a linear way and
each has its own regression coefficient:
Y = β
1
X
1
+ β
2
X
2
+…..+ β
L
X
L
+ ε
The β parameters reflect the independent contribution of
each independent variable, X, to the value of the
dependent variable, Y.
i.e. the amount of variance in Y that is accounted for by
each X variable after all the other X variables have been
accounted for

5. Matrix formulation
Multiplying matrices reminder:
a b c
d e f
G
H
I
=x
G x a + H x b + I x c
G x d + H x e + I x f

6. Matrix Formulation
Write out equation for each observation of variable Y from 1 to J:
Y
1
= X
11
β
1
+…+X
1l
β
l
+…+ X
1L
β
L
+ ε
1

Y
j = X
j1β
1 +…+X
jlβ
l +…+ X
jLβ
L + ε
j
Y
J = X
J1β
1 +…+X
Jlβ
l +…+ X
JLβ
L + ε
J
Y
1
Y
j
Y
J
=
X
11
… X
1l
… X
1L
X
j1
… X
1l
… X
1L
X
11
… X
1l
… X
1L
Can turn these simultaneous equations into matrix form to get a single
equation:
β
1
β
j
β
J
+
ε
1
ε
j
ε
J
Y = X x β + ε
Observed data
Design Matrix Parameters Residuals/Error

7. General Linear Model
This is simply an extension of multiple regression
Or alternatively Multiple Regression is just a simple form
of the General Linear Model
Multiple Regression only looks at ONE dependent (Y)
variable
Whereas, GLM allows you to analyse several
dependent, Y, variables in a linear combination
i.e. multiple regression is a GLM with only one Y variable
ANOVA, t-test, F-test, etc. are also forms of the GLM

8. GLM - continued..
In the GLM the vector Y, of J observations of a single Y
variable, becomes a MATRIX, of J observations of N
different Y variables
An fMRI experiment could be modelled with matrix Y of
the BOLD signal at N voxels for J scans
However SPM takes a univariate approach, i.e. each
voxel is represented by a column vector of J fMRI signal
measurements, and it processed through a GLM
separately
(this is why you then need to correct for multiple
comparisons)

9. GLM and fMRI
How does the GLM apply to fMRI experiments?
Y = X . β + ε
Observed data:
SPM uses a mass
univariate
approach – that is
each voxel is
treated as a
separate column
vector of data.
Y is the BOLD
signal at various
time points at a
single voxel
Design matrix:
Several components which
explain the observed data, i.e.
the BOLD time series for the
voxel
Timing info: onset vectors, O
m
j
,
and duration vectors, D
m
j
HRF, h
m
, describes shape of
the expected BOLD response
over time
Other regressors, e.g.
realignment parameters
Parameters:
Define the
contribution of each
component of the
design matrix to the
value of Y
Estimated so as to
minimise the error, ε,
i.e. least sums of
squares
Error:
Difference
between the
observed
data, Y, and
that predicted
by the model,
Xβ.
Not assumed
to be spherical
in fMRI

10. Parameter estimation
In linear regression the parameter β is estimated so that
the best prediction of Y can be obtained from X
i.e. sums of squares of difference between predicted
values and observed data, (i.e. the residuals, ε) is
minimised
Remember last week’s talk & graph!
The method of estimating parameters in GLM is
essentially the same, i.e. minimising sums of squares
(ordinary least squares), it just looks more complicated

11. Last week’s graph
ε
y = βx + c
ε = residual error
= y
i
, true value
= ỹ , predicted value

12. Residual Sums of Squares
Take a set of parameter estimates, β
Put these into the GLM equation to obtain estimates of Y from X, i.e.
fitted values, Y:
Y = X x β
The residual errors, e, are the difference between the fitted and
actual values:
e = Y - Y = Y - Xβ

Residual sums of squares is: S = Σ
j
J
e
j
2
When written out in full this gives:
S = Σ
j
J
(Y
j
- X
j1
β
1
-…- X
jL
β
L
)
2

13. Minimising S
If you plot the sum
of squares value
for different
parameter, β,
estimates you get
a curve
parameter estimates(B)
s
u
m
s
o
f s
q
u
a
re
s
(S
)
S is minimised
when the
gradient of this
curve is zero
Gradient = 0
min S
S = Σ(Y - Xβ)
2
e = Y - Xβ
S = Σ
j
J
e
j
2

14. Minimising S cont.
 so to calculate the values of β which gives you the least
sums of squares you must find the partial derivative of

S = Σ
j
J
(Y
j - X
j1β
1 -…- X
jLβ
L)
2
Which is

∂S/∂β = 2Σ(-X
jl)(Y
j – X
j1β
1-…- X
jLβ
L)
and solve this for ∂S/∂β = 0
In matrix form of the residual sum of squares is
S = e
T
e
this is equivalent to Σ
j
J
e
j
2
(remember how we multiply matrices)
 e = Y - X β therefore S = (Y - X β )
T
(Y - X β )

15. Minimising S cont.

Need to find the derivative and solve for ∂S/∂β = 0
The derivative of this equation can be rearranged to give
X
T
Y = (X
T
X)β
when the gradient of the curve = 0, i.e. S is minimised
This can be rearranged to give:
β = X
T
Y(X
T
X)
-1

But a solution can only be found, if (X
T
X) is invertible
because you need to divide by it, which in matrix terms is
the same as multiplying by the inverse!

16. GLM and fMRI
How does the GLM apply to fMRI experiments?
Y = X . β + ε
Observed data:
SPM uses a mass
univariate
approach – that is
each voxel is
treated as a
separate column
vector of data.
Y is the BOLD
signal at various
time points at a
single voxel
Design matrix:
Several components which
explain the observed data, i.e.
the BOLD time series for the
voxel
Timing info: onset vectors, O
m
j
,
and duration vectors, D
m
j
HRF, h
m
, describes shape of
the expected BOLD response
over time
Other regressors, e.g.
realignment parameters
Parameters:
Define the
contribution of each
component of the
design matrix to the
value of Y
Estimated so as to
minimise the error, ε,
i.e. least sums of
squares
Error:
Difference
between the
observed
data, Y, and
that predicted
by the model,
Xβ.
Not assumed
to be spherical
in fMRI

17. fMRI models
Completed the experiment, after preprocessing, the data
are ready for STATS.
STATS: (estimate parameters, β, inference)
indicating evidence against the Ho of no effect at
each voxel are computed->an image of this statistic
is produce
This statistical image is assessed (other talk will
explain that)

18. Example: 1 subject. 1 session
Moving finger vs rest
7 cycles of rest and moving
Time series of BOLD response in one voxel
Time seconds
R
e
s
p
o
n
s
e
s

a
t

v
o
x
e
l

(
x
,

y
,

z
)
Question:
Is there any change in the
BOLD response between
moving and rest?
Each epoch 6 scans
Whole brain acquisition data

19. TIME SERIES:
consist on the sequential
measures of fMRI data signal
intensities over the period of the
experiment
The same temporal model is
used at each voxel
Mass-univariated model and
perform the same analysis at
each voxel
Therefore, we can describe the
complete temporal model for
fMRI data by looking at how it
works for the data from a voxel. Single Voxel Time Series
Time
Linear Time Series Model

20. Y: My data/ observations
Single Voxel Time Series
My Data
Time
Y
1
Y
s
Y
N
Time series of N observations
Y
1
,…,Y
s
,…,Y
n
.
N= scan number
Acquired at one voxel
at times t
s, where S=1:N

21. Model specification
The overall aim of regressor generation is to come
up with a design matrix that models the expected
fMRI response at any voxel as a linear
combinations of columns.
Design matrix – formed of several components
which explain the observed data.
Two things SPM need to know to construct the
design matrix:

Specify regressors

Basis functions that explain my data

22. Model specification …
Specify regressors X
Timing information consists of onset vectors
O
m
j and duration vectors D
m
Other regressors e.g. movement parameters
Include as many regressors as you consider
necessary to best explain what’s going on.
Basis functions that explain my data (HRF)
Expected shape of the BOLD response due to
stimulus presentation

23. GLM and fMRI data
Model the observed time series at each voxel as a linear
combination of explanatory functions, plus an error term
Y
s
= β
1
X
1
(tS)
+ …+ β
l
X
l
(tS)
+ …+ β
L
X
L
(tS)
+ ε
s
Here, each column of the design matrix X contains the
values of one of the continuous regressors evaluated at
each time point t
s of fMRI time series
That is, the columns of the design matrix are the discrete
regressors

24. Consider the equation for all time points, to give a set of equations
Y
1
Y
s
Y
N
β
1
β
l
β
L
ε
N
ε
s
ε
N
+=
X
1
(t1) X
l
(t1)X
L
(t1)
X
1
(tS)
X
l
(tS)
X
L
(tS)
X
1
(tN)
X
l
(tN)
X
L
(tN)
Y
1= β
1 X
1
(t1)+ …+ β
l X
l
(t1)+ …+ β
L X
L
(t1) + ε
1
Y
s= β
1 X
1
(tS)+ …+ β
l X
l
(tS)+ …+ β
L X
L
(tS) + ε
s
Y
N= β
1 X
1
(tN)+ …+ β
l X
l
(tN)+ …+ β
L X
L
(tN) + ε
N
Y = X β + εIn matrix notation:
GLM and fMRI data …
In matrix form:

25. Getting the design matrix
Regressors
ε
Errors are
normally and
independently
and identical
distributed
Intensity
T
im
e
= β
1 β
2+ +
Observations
y = x
1
x
2

26. Getting the design matrix …
Regressors
Intensity
T
im
e
= β
1
β
2+ +
Observations
y = β
1
x
1
+ β
2
x
2
+

ε
Error

27. Design matrix
Regressors
=
β
2
β
1
+
Observations
Y = X β + ε
Error
x

28. Design matrix
Regressors
β
1
β
2
Observations Error
Y = X β + ε
N N
N
l
L
l
l
L
N: nuber of scans
P: number of regressors
Y = X β + ε
Model is specified by:
•design matrix
•Assumptions about ε
Y
1
Y
s
Y
N

X
1
(t1)


X
l
(t1)
X
L
(t1)

X
1
(tS)
X
l
(tS)
X
L
(tS)
X
1
(tN)
X
l
(tN)
X
L
(tN)
β
1
β
l
β
L
ε
N
ε
s
ε
N

29. Parametric estimation
=
β
1
β
2
+
+
Y X ε
Estimate parameters
β
The error is minimal when
Least squares
Parameter estimates
β = X
T
Y(X
T
X)
-1
ε = Y - Y = Y - Xβ
S = Σ
t
J
ε
t
2
(Get this by putting into matrix form
and finding derivative)

30. Summary
The General Linear Model allows you to find the
parameters, β, which provide the best fit with your data, Y
The optimal parameters estimates, β, are found by
minimising the Sums of Squares differences between your
predicted model and the observed data
The design matrix in SPM contains the information about
the factors, X, which may explain the observed data
Once we have obtained the βs at each voxel we can use
these to do various statistical tests
but that is another talk….

31. THE END
Thank you to
Lucy, Daniel and Will
and to
Stephan for his chapter and slides about GLM
and to
Adam for last year’s presentation
Links:
http://www.fil.ion.ucl.ac.uk/~wpenny/notes03/slides/glm/slide1.htm
http://www.fil.ion.ucl.ac.uk/spm/HBF2/pdfs/Ch7.pdf
http://www.fil.ion.ucl.ac.uk/~wpenny/mfd/GLM.ppt