algorithm_lec_1eregdsgdfgdgdfgdfgdfg.ppt

Student of Midnapore College (Autonomous).

1. Design & Analysis of
Algorithms
Design the Algorithm
Analysis the Algorithm

2. Introduction
•Algorithm:Idea behind the Program
•Pseudo-Code:Natural Language + Flavour of
Programming Language (Even that can be used to
describe an algorithm)
•Program:Set of Instruction
•Process:Program in Execution

3. Algorithms
Finiteness:Analgorithmmustalwaysterminateaftera
finitenumberofsteps.
Definiteness:Eachstepofanalgorithmmustbeprecisely
defined;theactionstobecarriedoutmustberigorously
andunambiguouslyspecifiedforeachcase.
Input:Analgorithmhaszeroormoreinputs,i.e,quantities
whicharegiventoitinitiallybeforethealgorithmbegins.
Output:Analgorithmhasoneormoreoutputsi.e,
quantitieswhichhaveaspecifiedrelationtotheinputs.
Effectiveness:Analgorithmisalsogenerallyexpectedto
beeffective.Thismeansthatalloftheoperationstobe
performedinthealgorithmmustbesufficientlybasicthat
theycaninprinciplebedoneexactlyandinafinitelength
oftime.

4. The problem of sorting

5. Insertion Sort

6. Example of Insertion Sort

7. Example of Insertion Sort

8. Example of Insertion Sort

9. Example of Insertion Sort

10. Example of Insertion Sort

11. Example of Insertion Sort

12. Example of Insertion Sort

13. Example of Insertion Sort

14. Example of Insertion Sort

15. Example of Insertion Sort

16. Example of Insertion Sort

17. Running Time
• The running time depends on the input: an already
sorted sequence is easier to sort.
• Parameterize the running time by the size of the
input, since short sequences are easier to sort than
long ones.
• Generally, we seek upper bounds on the running
time, because everybody likes a guarantee.

18. Kinds of analyses
Worst-case:(usually)
• T(n) = maximum time of algorithm on any input of
size n.
Average-case:(sometimes)
• T(n) = expected time of algorithm over all inputs of
size n.
• Need assumption of statistical distribution of inputs.
Best-case:
• Cheat with a slow algorithm that works fast on some
input.

19. Machine-Independent time
The RAM Model
Machine independent algorithm design depends on a
hypothetical computer called Random Acces Machine (RAM).
Assumptions:
• Each simple operation such as +,-,if ...etc takes exactly
one time step.
• Loops and subroutines are not considered simple
operations.
• Each memory acces takes exactly one time step.

20. Machine-independent time
What is insertion sort’s worst-case time?
• It depends on the speed of our computer,
• relative speed (on the same machine),
• absolute speed (on different machines).
BIG IDEA:
• Ignore machine-dependent constants.
• Look at growth of
“Asymptotic Analysis”nnT as )(

21. Machine-independent time: An example
A pseudocodefor insertion sort ( INSERTION SORT ).
INSERTION-SORT(A)
1 for j2 to length [A]
2 do key A[ j]
3 Insert A[j] into the sortted sequence A[1,..., j-1].
4 i j –1
5 while i > 0 and A[i] > key
6 do A[i+1] A[i]
7 i i–1
8 A[i +1] key

22. Analysis of INSERTION-SORT(contd.)1]1[8
)1(17
)1(][]1[6
][05
114
10]11[ sequence
sorted theinto][Insert 3
1][2
][21
timescost SORT(A)-INSERTION
8
2
7
2
6
2
5
4
2
1















nckeyiA
tcii
tciAiA
tckeyiAandi
ncji
njA
jA
ncjAkey
ncAlengthj
n
j
j
n
j
j
n
j
j
do
while
do
tofor

23. Analysis of INSERTION-SORT(contd.))1()1()1()(
2
6
2
5421  

n
j
j
n
j
j tctcncnccnT ).1()1(
8
2
7 

nctc
n
j
j
The total running time is

24. Analysis of INSERTION-SORT(contd.)
The best case: The array is already sorted.
(t
j=1 for j=2,3, ...,n))1()1()1()1()(
85421  ncncncncncnT ).()(
854285421 ccccnccccc 

25. Analysis of INSERTION-SORT(contd.)
•The worst case: The array is reverse sorted
(t
j=j for j=2,3, ...,n).)12/)1(()1()(
521  nncncncnT )1()2/)1(()2/)1((
876  ncnncnnc ncccccccnccc )2/2/2/()2/2/2/(
8765421
2
765  2
)1(
1



nn
j
n
j cbnannT 
2
)(

26. Growth of Functions
Although we can sometimes determine the exact
running time of an algorithm, the extra precision is not
usually worth the effort of computing it.
For large inputs, the multiplicative constants and lower
order terms of an exact running time are dominated by
the effects of the input size itself.

27. Asymptotic Notation
The notation we use to describe the asymptotic running
time of an algorithm are defined in terms of functions
whose domains are the set of natural numbers.
Asymptotic notations are the
mathematicalnotationsusedtodescribetherunningtime
ofanalgorithmwhentheinputtendstowardsaparticular
valueoralimitingvalue. ...,2,1,0N

28. O-notation
•For a given function , we denote by the set
of functions
•We use O-notation to give an asymptotic upper bound of
a function, to within a constant factor.
• means that there existes some constant c
s.t. is always for large enough n. )(ng ))((ngO 







0
0
allfor )()(0
s.t.and constants positiveexist there:)(
))((
nnncgnf
ncnf
ngO ))(()( ngOnf )(ncg )(nf

29. Ω-Omega notation
•For a given function , we denote by the
set of functions
•We use Ω-notation to give an asymptotic lower bound on
a function, to within a constant factor.
• means that there exists some constant cs.t.
is always for large enough n. )(ng ))((ng 







0
0
allfor )()(0
s.t.and constants positiveexist there:)(
))((
nnnfncg
ncnf
ng ))(()( ngnf  )(nf )(ncg

30. -Theta notation
•For a given function , we denote by the set
of functions
•A function belongs to the set if there exist
positive constants and such that it can be “sand-
wiched” between and or sufficienly large n.
• means that there exists some constant c
1
and c
2 s.t. for large enough n.)(ng ))((ng 







021
021
allfor )()()(c0
s.t.and,, constants positiveexist there:)(
))((
nnngcnfng
nccnf
ng )(nf ))((ng 1c 2c )(
1ngc )(
2ngc Θ ))(()( ngnf  )()()(
21
ngcnfngc 

31. Asymptotic notation
Graphic examples of and .,,O

32. 2
2
22
1 3
2
1
ncnnnc  21
3
2
1
c
n
c  Example 1.
Show that
We must find c
1 and c
2such that
Dividing bothsides by n
2yields
For )(3
2
1
)(
22
nnnnf  )(3
2
1
,7
22
0 nnnn 

33. Theorem
•For any two functions and , we have
if and only if)(ng ))(()( ngnf  )(nf )).(()( and ))(()( ngnfngOnf 

34. Because :)
2
(52
2
3 nnn 
Example 2.)
2
(52
2
3)( nnnnf  )
2
(52
2
3 nOnn 

35. Example 3. 610033,3forsince)(61003
2222
 nnncnOnn

36. Example 3. 3when61003,1forsince)(61003
610033,3forsince)(61003
2332
2222


nnnncnOnn
nnncnOnn

37. Example 3. cnncncnOnn
nnnncnOnn
nnncnOnn



when3,any forsince)(61003
3when61003,1forsince)(61003
610033,3forsince)(61003
22
2332
2222

38. Example 3. 100when610032,2forsince)(61003
when3,any forsince)(61003
3when61003,1forsince)(61003
610033,3forsince)(61003
2222
22
2332
2222




nnnncnnn
cnncncnOnn
nnnncnOnn
nnncnOnn

39. Example 3. 3when61003,3forsince)(61003
100when610032,2forsince)(61003
when3,any forsince)(61003
3when61003,1forsince)(61003
610033,3forsince)(61003
3232
2222
22
2332
2222





nnnncnnn
nnnncnnn
cnncncnOnn
nnnncnOnn
nnncnOnn

40. Example 3. 100when61003,any forsince)(61003
3when61003,3forsince)(61003
100when610032,2forsince)(61003
when3,any forsince)(61003
3when61003,1forsince)(61003
610033,3forsince)(61003
22
3232
2222
22
2332
2222






nnncncnnn
nnnncnnn
nnnncnnn
cnncncnOnn
nnnncnOnn
nnncnOnn

41. Example 3. apply. and both since)(61003
100when61003,any forsince)(61003
3when61003,3forsince)(61003
100when610032,2forsince)(61003
when3,any forsince)(61003
3when61003,1forsince)(61003
610033,3forsince)(61003
22
22
3232
2222
22
2332
2222







Onnn
nnncncnnn
nnnncnnn
nnnncnnn
cnncncnOnn
nnnncnOnn
nnncnOnn

42. Example 3. applies. only since)(61003
apply. and both since)(61003
100when61003,any forsince)(61003
3when61003,3forsince)(61003
100when610032,2forsince)(61003
when3,any forsince)(61003
3when61003,1forsince)(61003
610033,3forsince)(61003
32
22
22
3232
2222
22
2332
2222
Onnn
Onnn
nnncncnnn
nnnncnnn
nnnncnnn
cnncncnOnn
nnnncnOnn
nnncnOnn









43. Example 3. applies. only since)(61003
applies. only since)(61003
apply. and both since)(61003
100when61003,any forsince)(61003
3when61003,3forsince)(61003
100when610032,2forsince)(61003
when3,any forsince)(61003
3when61003,1forsince)(61003
610033,3forsince)(61003
2
32
22
22
3232
2222
22
2332
2222









nnn
Onnn
Onnn
nnncncnnn
nnnncnnn
nnnncnnn
cnncncnOnn
nnnncnOnn
nnncnOnn

44. Standard notations and common functions
•Floors and ceilings  11  xxxxx

45. Standard notations and common functions
•Logarithms:)lg(lglglg
)(loglog
logln
loglg
2
nn
nn
nn
nn
kk
e





46. Standard notations and common functions
•Logarithms:
For all real a>0, b>0, c>0, and nb
a
a
ana
baab
ba
c
c
b
b
n
b
ccc
a
b
log
log
log
loglog
loglog)(log
log





47. Standard notations and common functions
•Logarithms:b
a
ca
aa
a
b
ac
bb
bb
log
1
log
log)/1(log
loglog




48. Standard notations and common functions
•Factorials
For the Stirling approximation:




















ne
n
nn
n
1
12! 0n )lg()!lg(
)2(!
)(!
nnn
n
non
n
n




49. Types of Functions
O(1) ------Constant
Ex.: f(n)= 2 or 5 or 5000 (any constant) O(1)
O(logn) ------logarithmic
Ex.: alogn+b O(logn)
O(n) Linear
Ex.: an+b =O(n)
O(n^2) Quadratic
Ex.: an^2+bn+c
O(n^3) Cubic
Ex.: an^3+bn^2+cn+d
O(a^n) like 2^n, 3^n, n^n Exponential

50. Compare Classes of Functions
1<logn<√n<n<nlogn< n^2<n^3<…<2^n<3^n<…<n^n

51. Machine Independent
•TimeComplexityofanAlgorithm:
NumberofPrimitivecomputations.
Relationshipbetweentherunningtime
ofanalgorithmandthesizeofitsinput
•DoesnotdependonSoftwareand
Hardware

52. A Model of the Computer: RAM
•An informal model of the random-access machine (RAM)
•Basic types in the RAM model
•◮integer and floating-point numbers
•◮limited size of each “word” of data (e.g., 64 bits)
•Basic steps in the RAM model
•◮operations involving basic types
•◮load/store: assignment, use of a variable
•◮arithmetic operations: addition, multiplication, division,
etc.
•◮branch operations: conditional branch, jump
•◮subroutine call
A basic step in the RAM model takes a constant time