Lecture 1,2 maths presentation slides.pdf

This is a maths ppt presentation explaning an important topic.

1. Introduction to Probability
Chapter 1: Introduction
Dr. Nitin Gupta
Department of Mathematics
Indian Institute of Technology Kharagpur,
Kharagpur - 721 302, INDIA.
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 1 / 21

2. Outline
1
Motivation
2
Introduction to the concepts of probability
3
Assigning probabilities to events
4
Continuity theorem in probability
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 2 / 21

3. References
1
Probability and statistics in engineering by Hines et al (2003) Wiley.
2
Mathematical Statistics by Richard J. Rossi (2018) Wiley.
3
Probability and Statistics with reliability, queuing and computer
science applications by K. S. Trivedi (1982) Prentice Hall of India
Pvt. Ltd.
4
https://medium.com/@jrodthoughts/statistical-learning-in-articial-
intelligence-systems-e68927792175 by Jesus Rodriguez
(2017)
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 3 / 21

4. Motivation
In articial intelligence (AI) environment, uncertainty is a key element.
Due to uncertainty the AI agent does not know the precise outcome of the
given situation. Uncertainty is the typical result of random/probabilistic or
partially observable environment. Statistical learning is helpful in these AI
situations.
For example Bayes' theorem helps in dealing with uncertainty in the real
world:
P(causejeect) =P(eect)xP(eectjcause)=P(cause);
whereP(AjB) is the probability of occurrence of A given B. Replacing
cause and eect with the probabilities of any state-action combination in
an AI environment we arrive to the fundamentals of Bayesian learning.
Many AI algorithms are based on Bayesian learning or statistical learning.
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 4 / 21

5. Motivation
In reliability computation of r-out-of n system the probability concepts are
used where the components are assumed to have random life. r-out-of n
system is a system which functions if atleast r out of its n components
functions. Series and parallel system are n-out of n system and 1-out of n
system, respectively.
Consider a situation where a redundant component or spare is provided to
the system to increase its reliability. Then using probability concepts we
can nd increase in the reliability of the system.
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 5 / 21

6. Introduction
Random Experiment (E): is an experiment whose outcome may not
be perdicted in advance.
Sample Space (): Collection of all possible outcomes of random
experiment.
Example
IfE1: Toss a coin, then 1=fH;Tg.
IfE2: Toss a coin till we get a head, then 2=fH;TH;TTH; : : :g.
IfE3: Lifetime of a bulb, then 3= [0;1).
IfE4: Radioactive particles emitted by a radioactive substance, then
4=f0;1;2; : : :g.
IfE5: Roll a pair of dice and see up face, then
5=f(i;j);i= 1;2;3;4;5;6;j= 1;2;3;4;5;6g.
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 6 / 21

7. Introduction
Event: is subset of sample space. Event is denoted by capital letter.
The set of all subsets is power set for a nite sample space .
Example
InE1the event is the toss yield a headA1=fHg
InE2we are getting head in third toss then event isA2=fTTHg.
InE3an event isA3= (0;2).
InE4if the radioactive particles emitted is 2, thenA4=f2g.
InE5if sum of number on up faces is 4, thenA5=f(1;3);(2;2);(3;1)g.
IfA1;A2; : : :are events in and eventsA1;A2; : : :are mutually exclusive,
then
S
1
i=1
Ai.
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 7 / 21

8. Sigma Field
SupposeEis an experiment with sample space as . Letfbe a collection
of subsets of . Thenfis said to be a sigma eld if
1
2f.
2
IfA2f, thenA2f.
3
IfA1;A22f, thenA1[A22f.
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 8 / 21

9. Example-Sigma Field
Example (1)
E: Toss a coin, then =fH;Tg. Then
f1=f;fHg;fTg;gis the power set and is a sigma eld.
f2=f;gis trivial sigma eld.
Example (2)
E: Toss a two coin, then =fHH;HT;TH;TTg. Then
f1=f;gis trivial sigma eld.
f2=Power set of , is a sigma eld.
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 9 / 21

10. Probability
Denition
Consider a random experimentEhaving sample space . Letfbe a sigma
eld of subsets of . Consider an eventAdened onf, thenP(A) is a real
number called a probability of eventAifP() satises following axioms:
1
P() = 1, 2f.
2
P(A)0,A2f.
3
IfA1;A2; : : :are mutually exclusive events inf, then
P

1
[
i=1
Ai
!
=
1
X
i=1
P(Ai):
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 10 / 21

11. How to assign the probabilities
The assignment of probability is done on the basis of
1
prior experience or prior observations;
2
analysis of the experimental conditions;
3
assumptions.
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 11 / 21

12. Relative frequency approximation
Relative frequencyfA=
number of times event A occurs
number of times experiment was done
=
mA
m
P(A) = lim
m!1
fA:
For large number of trails, the approximate probability obtained is quite
near to the exact probability. The disadvantage of this approach is that
the experiment should be repeated and is not a one o situation.
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 12 / 21

13. Classical Method
Here we assume that the possible outcomes of random experiment are
equally likely and their total number is nite. Then
P(A) =
n(A)
n()
=
number of favourable cases to eventA
number of cases in
:
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 13 / 21

14. Simple consequences of axioms
Consider the experimentEon (;f).
1
P() = 0; =fg 2f.
2
AandBare two events inf, then
P(A[B) =P(A) +P(B)P(A\B):
3
IfA;B2fandAB, thenP(A)P(B).
4
LetAi2f;i= 1;2; : : : ;n, then
P

n
[
i=1
Ai
!
=
n
X
i=1
P(Ai)
n
X
i;j=1
i<j
P(Ai\Aj) +
n
X
i;j;k=1
i<j<k
P(Ai\Aj\Ak)
+ + (1)
n1
P(A1\A2\ \An):
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 14 / 21

15. Example
Example
Suppose we are rolling two fair dices independently. We want to nd the
probability that
1.
2.
Solution: 1.Let eventA1denote sum of faces up is 7. Favourable cases for
A1=f(1;6);(6;1);(2;5);(5;2);(3;4);(4;3)g. Also total number of cases
are 36. Hence
P(A1) =
Number of favourable cases
Total number of cases
=
6
36
=
1
6
:
2. EventA2denote sum of numbers on up-faces is greater than 9. Then
A2=f(4;6);(6;4);(5;5);(5;6);(6;5);(6;6)g. Therefore
P(A2) =
6
36
=
1
6
:
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 15 / 21

16. Example
Example
An urn contains 5 red, 2 black and 4 yellow balls. Two balls are drawn at
random from the urn. Find the probability that both balls are of same
colour.
Total number of balls are 11. Two balls are drawn out of 11 balls in

11
2

ways. Let eventE1denote that both balls are of same colour. If balls are
red the number of ways of choosing them are

5
2

. If balls are black the
number of ways of choosing them are

2
2

. If balls are yellow the number
of ways of choosing them are

4
2

. Therefore number of favourable cases
toE1is

5
2

+

2
2

+

4
2

. Hence required probability is
P(E1) =

5
2

+

2
2

+

4
2


11
2
=
17
55
:
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 16 / 21

17. Example
Example
Four personsA;B;C;andDtake turns (in the sequence
A;B;C;D;A;B;C;D;A; : : :) in tossing a biased coin.The biased coin has
probability 3/4 of head up. The rst person to get a tail wins. We want to
determine the probability thatBwins. The probability of getting a tail in
tossing the coin isp= 1=4 andq= 1p. Then required probability is
P(Bwins) =qp+q
5
p+q
9
p+
=pq(1 +q
4
+q
8
+ )
=
pq
1q
4
= 0:274:
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 17 / 21

18. Denition
(a) A sequence of eventsfAng
1
n=1
,An2fare said to be monotonically
increasing if, for alln,AnAn+1
(b) A sequence of eventsfAng
1
n=1
,An2fare said to be monotonically
decreasing if, for alln,AnAn+1
Denition (Limit of sequence)
(a) For monotonically increasing sequence of eventsfAng
1
n=1
,An2fthe
limit of sequence of events is dened as
lim
n!1
An=
1
[
n=1
An
(b) For monotonically decreasing sequence of eventsfAng
1
n=1
,An2fthe
limit of sequence of events is dened as
lim
n!1
An=
1
\
n=1
An
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 18 / 21

19. Example
Example
LetAn1=f!: 0< ! <
n2
n1
g;n= 2;3; : : : :Then
A1=;A2= (0;
1
2
);A3= (0;
2
3
); : : :. Hence sequence of eventsfAng
1
n=1
is monotonically increasing. The limit of sequence is
lim
n!1
An=
1
[
n=1
An
=f!: 0< ! <1g:
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 19 / 21

20. Continuity theorem in probability
Let (;f;P) be a probability model.
(a) fAng
1
n=1
,An2f, be monotonically increasing sequence of events,
then
P

lim
n!1
An

= lim
n!1
P(An)
(b) fAng
1
n=1
,An2f, be monotonically decreasing sequence of events,
then
P

lim
n!1
An

= lim
n!1
P(An)
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 20 / 21

21. Summary
To analyze the the algoritmns and computer systems, computer scientists
need powerful tools. Many of the tools require the foundation in the
probability theory. Hence we require to study the concepts of probability.
The Russian mathemematican Kolmogorov (1903-1987) provided
foundational work of the modern probability theory. In this chapter we
introduced the concepts of random experiment, the sample space and the
mathematical denition of probability. Followed by the assignment of the
probabilities to the events. Some simple consequences of the axioms of the
denition of the probability were also discussed. Finally we discussed the
continuity theorem of probability. Various examples are provided to
understand the concepts.
N. Gupta (IIT Kharagpur) Chapter 1: Introduction 21 / 21