Overview of Inference in Linear Chain CRF's

Overview of inference in linear chain conditional random fields (CRF's)

1. INFERENCE IN LINEAR CHAIN
CRF’S
Christian Duff au-Rasmussen
16. August 2022
1

2. DISCRIMINATIV VS GENERATIVE MODEL
Generative model: A model of
(Probabilistic) discriminative model: A model of
P(Y,X)=P(X|Y)P(Y).
P(Y|X).
2

3. DISCRIMINATIV VS GENERATIVE MODEL
3

4. NAIVE BAYES - GENERATIVE EXAMPLE
4

5. HMM - GENERATIVE EXAMPLE
5

6. LOGISTIC REGRESSION - DISCRIMINATIVE
6

7. LINEAR CHAIN CRF - DISCRIMINATIVE
7

8. LINEAR CHAIN CRF - DISCRIMINATIVE
8

9. LINEAR CHAIN CRF - DISCRIMINATIVE
Mimicking HMM assumptions of P(yt|x)=P(yt|xt)
9

10. LOGISTIC REGRESSION AS A CRF
10

11. HMM AS A LINEAR CHAIN CRF
θij=logp(y
′
=i|y=j)
μoi=logp(x=o|y=i)
11

12. COMPUTING MARGINAL PROBS - HUGO
LAROCHELLE
12

13. COMPUTING MARGINAL PROBS - HUGO
LAROCHELLE

14. 13

15. COMPUTING MARGINAL PROBS - HUGO
LAROCHELLE

16. 14

17. A 3X3 EXAMPLE - FORWARD-BACKWARD
15

18. A 3X3 EXAMPLE - FORWARD-BACKWARD
16

19. A 3X3 EXAMPLE - FORWARD-BACKWARD
17

20. A 3X3 EXAMPLE - FORWARD-BACKWARD
Computing the partition function
Great unit test.
18

21. THE ISSUE WITH start AND end
TRANSITIONS
How to include start and end transition weights?
How to check the calculations are correct?
19

22. THE HMM AS A LINEAR CHAIN CRF
Stationary distribution:
: Transitions probabilities
: Matrix of ones.
: Row vector of ones.
: The identity matrix.
δ=1C(IC−Γ+U)
−1
Γ
U
1C
IC
20

23. EXPERIMENTS - NO START OR END
21

24. EXPERIMENTS - INCLUDING START, NO END
22

25. EXPERIMENTS - INCLUDING START AND END
23