DATA STRUCTURES AND

ALGORITHMS

Lecture Notes 1

Prepared by nan TAHRALI
2
ROAD MAP
What is an algorithm ?
What is a data structure ?
Analysis of An Algorithm
Asymptotic Notations
Big Oh Notation
Omega Notation
Theta Notation
Little o Notation
Rules about Asymptotic Notations
3
What is An Algorithm ?
Definition :
A finite, clearly specified sequence of
instructions to be followed to solve a
problem.

4
What is An Algorithm ?
int Sum (int N)

PartialSum 0
i 1

foreach (i > 0) and (i<=N)
PartialSum PartialSum + (i*i*i)
increase i with 1

return value of PartialSum

int Sum (int N)
{
int PartialSum = 0 ;

for (int i=1; i<=N; i++)
PartialSum += i * i * i;

return PartialSum;
}
Problem : Write a program to calculate

=
N
i
i
1
3
5
Properties of an Algorithm
Effectiveness
simple
can be carried out by pen and paper
Definiteness
clear
meaning is unique
Correctness
give the right answer for all possible cases
Finiteness
stop in reasonable time
6
Is function F an algorithm?
int F (int m)
{
n = m;
while (n>1)
{
if (n/2 == 0)
n=n/2;
else
n=3*n+1;
}
return m;
}
7
What is a Data Structure ?
Definition :
An organization and representation of data
representation
data can be stored variously according to their type
signed, unsigned, etc.
example : integer representation in memory
organization
the way of storing data changes according to the
organization
ordered, inordered, tree
example : if you have more then one integer ?
8
Properties of a Data Structure ?
Efficient utilization of medium
Efficient algorithms for
creation
manipulation (insertion/deletion)
data retrieval (Find)
A well-designed data structure allows using little
resources
execution time
memory space
9
The Process of Algorithm Development
Design
divide&conquer, greedy, dynamic
programming
Validation
check whether it is correct
Analysis
determine the properties of algorithm
Implementation
Testing
check whether it works for all possible cases
10
Analysis of Algorithm
Analysis investigates
What are the properties of the algorithm?
in terms of time and space
How good is the algorithm ?
according to the properties
How it compares with others?
not always exact
Is it the best that can be done?
difficult !
11
Mathematical Background
Assume the functions for running times of two
algorthms are found !

For input size N
Running time of Algorithm A = T
A
(N) = 1000 N
Running time of Algorithm B = T
B
(N) = N
2


Which one is faster ?
12
N T
A
T
B
10 10
-2
sec 10
-4
sec
100 10
-1
sec 10
-2
sec
1000 1 sec 1 sec
10000 10 sec 100

sec
100000 100

sec 10000

sec
If the unit of running time of algorithms A and B is sec
Mathematical Background
So which algorithm is faster ?
13
Mathematical Background
If N<1000 T
A
(N) > T
B
(N)
o/w T
B
(N) > T
A
(N)
TB
TA
T (Time)
N (Input size)
1000
Compare their relative growth ?
14
Mathematical Background
Is it always possible to have definite results?
NO !

The running times of algorithms can change
because of the platform, the properties of the
computer, etc.

We use asymptotic notations (O, , , o)
compare relative growth
compare only algorithms
15
Big Oh Notation (O)
Provides an upper bound for the function f

Definition :
T(N) = O (f(N)) if there are positive constants c
and n
0
such that
T(N) cf(N) when N n
0

T(N) grows no faster than f(N)
growth rate of T(N) is less than or equal to growth rate
of f(N) for large N
f(N) is an upper bound on T(N)
not fully correct !
16
Big Oh Notation (O)
Analysis of Algorithm A
O(N) N 1000 (N) T
A
= =
1000 N cN


if c= 2000 and n
0
= 1 for all N
is right
0
n N >
O(N) N 1000 (N) T
A
= =
17
Examples
7n+5 = O(n)
for c=8 and n
0
=5
7n+5 8n n>5 = n
0

7n+5 = O(n
2
)
for c=7 and n
0
=2
7n+5 7n
2
nn
0

7n
2
+3n = O(n) ?

18
Advantages of O Notation
It is possible to compare of two algorithms
with running times
Constants can be ignored.
Units are not important
O(7n
2
) = O(n
2
)
Lower order terms are ignored
O(n
3
+7n
2
+3) = O(n
3
)
19
Running Times of Algorithm A and B

T
A
(N) = 1000 N = O(N)
T
B
(N) = N
2
= O(N
2
)


A is asymptotically faster than B !
20
Omega Notation ()
Definition :
T(N) = (f(N)) if there are positive constants c
and n
0
such that T(N) c f(N) when N n
0

T(N) grows no slower than f(N)
growth rate of T(N) is greater than or equal to growth
rate of f(N) for large N
f(N) is a lower bound on T(N)
not fully correct !

21
Omega Notation
Example:
n
1/2
= O (lg n) .
for c = 1 and n
0
= 16
Let n > 16
c*(lg n) n
1/2
22
Omega Notation
Theorem:
f(N) = O(g(n)) <=> g(n) = (f(N))
Proof:
f(N) c
1
g(n) <=> g(n) c
2
f(N)
divide the left side with c
1

1/c
1
f(N) g(n) <=> g(n) c
2
f(N)
if we choose c
2
as 1/c
1
then theorem is right.

23
Omega Notation
7n
2
+ 3n + 5 = O(n
4
)
7n
2
+ 3n + 5 = O(n
3
)
7n
2
+ 3n + 5 = O(n
2
)
7n
2
+ 3n + 5 = (n
2
)
7n
2
+ 3n + 5 = (n)
7n
2
+ 3n + 5 = (1)

n
2
and 7n
2
+ 3n + 5 grows at the same rate
7n
2
+ 3n + 5 = O(n
2
) = (n
2
) = (n
2
)
24
Theta Notation ()
Definition :
T(N) = (h(N)) if and only if
T(N) = O(h(N)) and T(N) = (h(N))

T(N) grows as fast as h(N)
growth rate of T(N) and h(N) are equal for
large N
h(N) is a tight bound on T(N)
not fully correct !
25
Theta Notation ()
Example :
T(N) = 3N
2


T(N) = O(N
4
)


T(N) = O(N
3
)


T(N) = (N
2
) best


26
Little O Notation (o)
Definition :
T(N) = o(p(N)) if
T(N) = O(p(N)) and T(N)(p(N))

f(N) grows strictly faster than T(N)
growth rate of T(N) is less than the growth
rate of f(N) for large N
f(N) is an upperbound on T(N) (but not tight)
not fully correct !
27
Little O Notation (o)
Example :

T(N) = 3N
2

T(N) = o(N
4
)

T(N) = o(N
3
)

T(N) = (N
2
)


28
RULES
RULE 1:

if T
1
(N) = O(f(N)) and T
2
(N) = O(g(N)) then

a) T
1
(N) + T
2
(N) = max (O(f(N)), O(g(N)))
b) T
1
(N) * T
2
(N) = O(f(N) * g(N))

You can prove these ?
Is it true for notation ?
What about notation?
29
RULES
RULE 2:

if T(N) is a polynomial of degree k

T(N) = a
k
N
k
+ a
k-1
N
k-1
+ + a
1
N + a
0


then
T(N) = (N
k
)

30
RULES
RULE 3:

log
k
N = o(N) for any constant k

logarithm grows very slowly !

31
Some Common Functions
c = o (log N) => c=O(log N) but c(logN)
log N = o(log
2
N)
log
2
N = o(N)
N = o(N log N)
N = o (N
2
)
N
2
= o (N
3
)
N
3
= o (2
N
)
32
Example :
T(N) = 4N
2

T(N) = O(2N
2
)
correct but bad style
T(N) = O(N
2
)
drop the constants
T(N) = O(N
2
+N)
correct but bad style
T(N) = O(N
2
)
ignore low order terms

33
Another Way to Compute Growth Rates
)) ( ( ) ( 0
) (
) (
lim N g o N f
N g
N f
N
= =

relation no is there oscilate
N f o N g
N g N f c
=
= =
= = =
)) ( ( ) (
)) ( ( ) ( 0 u
34
Example :
f(N) = 7N
2
g(N) = N
2
+ N




35
Example :
f(N) = N logN g(N) = N
1.5


compare logN with N
0.5

compare log
2
N with N


compare log
2
N with o(N)

N logN = o(N
1.5
)