CS51: Abstraction and Design in Computation Using Lazy Streams

CS51:
Abstraction and Design in Computation

Stuart M. Shieber
John A. Paulson School of Engineering and Applied Sciences

Harvard University
CS Advising Fair
2:30-4:00 today after class
Maxwell Dworkin lobby
2
CS Advising Fair
2:30-4:00 today after class
Maxwell Dworkin lobby
2
11: Lazy evaluation and

infinite data structures
Big ideas
Streams
Postponing computation
Lazy recomputation
Laziness through the ages
An application: Taylor series and !
Streams
type 'a stream =
Cons of 'a * 'a stream ;;
find this code in: streams.ml
Streams
type 'a stream =
let head (s: 'a stream) : 'a =
match s with
| Cons (h, _) -> h ;;
Streams
type 'a stream =
let head (Cons(h,_): 'a stream) : 'a =
h ;;
let tail (Cons(_,t): 'a stream) : 'a stream =
t ;;
Streams
# type 'a stream =
type 'a stream = Cons of 'a * 'a stream
#
Streams
# type 'a stream =
# let ones = Cons(1, ones) ;;
Streams
# type 'a stream =
Integer stream goes here.
Streams
# type 'a stream =
10
Streams
# type 'a stream =
Error: Unbound value ones
#
11
Streams
# type 'a stream =
# let rec ones = Cons(1, ones) ;;
12
Streams
# type 'a stream =
val ones : int stream = Cons (1, <cycle>)
#
13
Streams
# type 'a stream =
# head ones;;
- : int = 1
14
# type 'a stream =

Streams

# head ones;;
- : int = 1
# head (tail ones);;
- : int = 1
# head (tail (tail ones));;
- : int = 1
14
How to think about streams

15

Allocate space for the cons

without initializing it, and
have ones point to it.
Cons( , )
15

Allocate space for the cons

have ones point to it.
ones
15
Cons( , )

Initialize the contents of the

cons with the values of the
arguments
ones
16
Cons(1, )

Initialize the contents of the

cons with the values of the
arguments
ones
17
Cons(1, )

This explains where the

<cycle> annotation
comes from.
ones
18
Cons(1, )
More recursive data

# let rec x = 1 + x ;;
19
More recursive data

# let rec x = 1 + x ;;
Allocate space for the int

have x point to it.
19
More recursive data

# let rec x = 1 + x ;;
Allocate space for the int

have x point to it.
x
19
More recursive data

# let rec x = 1 + x ;;
Initialize the contents with

the value of the definition
expression 1 + x
x
20
More recursive data

# let rec x = 1 + x ;;
Error: This kind of expression is not allowed as righthand side of `let rec'
x
21
Stream operations: mapping

# type 'a stream =
# let rec smap (f : 'a -> 'b) (s : 'a stream)
: ('b stream) =
match s with
| Cons(h, t) -> Cons(f h, map f t) ;;
22

# type 'a stream =
: ('b stream) =
Cons(f (head s), smap f (tail s)) ;;
# let twos = smap ((+) 1) ones ;;
23

# type 'a stream =
: ('b stream) =
23

# type 'a stream =
: ('b stream) =
Stack overflow during evaluation (looping recursion?).
24
# type 'a stream =
: ('b stream) =
25
# type 'a stream =
: ('b stream) =
fun x -> Cons(f (head s), smap f (tail s))
Postpone computation by wrapping in a function

25
Lazy streams
# type 'a str = Cons of 'a * 'a stream
and 'a stream = unit -> 'a str ;;
Mutually recursive type

definitions, using the
and connective
26
Lazy streams
# let rec ones : int stream =
fun () -> Cons(1, ones) ;;
27
Lazy streams
# let rec ones () : int str =
Cons(1, ones) ;;
# let head (s : 'a stream) : 'a =
match s() with
| Cons(h, t) -> h ;;
# let tail (s : 'a stream) : 'a stream =
match s() with
28
| Cons(h, t) -> t ;;
Lazy streams
Cons(1, ones) ;;
match s() with
| Cons(h, t) -> h ;;
match s() with
28
| Cons(h, t) -> t ;;

Lazy
streams
Cons(1,
ones) ;;

match s() with
| Cons(h, t) -> h ;;
match s() with
| Cons(h, t) -> t ;;
: ('b stream) =
fun () -> Cons(f (head s), smap f (tail s)) ;
29

Lazy
streams
Cons(1,
ones) ;;

match s() with
| Cons(h, t) -> h ;;
Returning
a function
# let tail (s : 'a stream)
: 'a stream
=
match s() with
| Cons(h, t) -> t
delays
evaluating the recursive call
to smap.
;;

: ('b stream) =
29
| Cons(h, t) -> t ;;
Lazy streams

: ('b stream) =

# head twos ;;
- : int = 2
# head (tail twos) ;;
- : int = 2
# head (tail (tail twos)) ;;
- : int = 2
30
| Cons(h, t) -> t ;;
Lazy streams

: ('b stream) =
fun () ->
An
infinite
map...
Cons(f (head s), smap
f (tail s)) ;

# head twos ;;
- : int = 2
- : int = 2
- : int = 2
30
| Cons(h, t) -> t ;;
Lazy streams

: ('b stream) =
fun () ->
An
infinite
map...
Cons(f (head s), smap
f (tail s)) ;

# head twos ;;
- : int = 2
...over an infinite sequence of ones

- : int = 2
- : int = 2
30
Lazy streams: extracting prefix

# let rec first (n : int) (s : 'a stream) : 'a list =
if n = 0 then []
else head s :: first (n-1) (tail s) ;;
val first : int -> 'a stream -> 'a list = <fun>
# first 10 ones ;;
- : int list = [1; 1; 1; 1; 1; 1; 1; 1; 1; 1]
# first 10 twos ;;
- : int list = [2; 2; 2; 2; 2; 2; 2; 2; 2; 2]
31
Lazy streams: natural numbers
32

Natural numbers
32
7 ...

Natural numbers
7 ...
1 ...
Ones
32

Natural numbers
7 ...
1 ...
8 ...
Ones
Add them
32

Natural numbers
7 ...
1 ...
8 ...
7 ...
8 ...
Ones
Add them
Prepend a zero
0
32

# let rec nats () = Cons(0, smap ((+) 1) nats) ;;
val nats : int stream = <fun>
# first 10 nats ;;
- : int list = [0; 1; 2; 3; 4; 5; 6; 7; 8; 9]
33

# let rec nats () = Cons(0, smap ((+) 1) nats) ;;
val nats : int stream = <fun>
# first 10 nats ;;
- : int list = [0; 1; 2; 3; 4; 5; 6; 7; 8; 9]
33
Lazy streams: mapping multiples

# let rec smap2 f s1 s2 =
fun () -> Cons(f (head s1) (head s2),
smap2 f (tail s1) (tail s2)) ;;
# let threes = smap2 (+) ones twos ;;
# first 10 threes ;;
- : int list = [3; 3; 3; 3; 3; 3; 3; 3; 3; 3]
34

# let rec fibs =
fun () -> Cons(0,
fun () -> Cons(1,
(smap2 (+) fibs (tail fibs)))) ;;
# first 10 fibs ;;
- : int list = [0; 1; 1; 2; 3; 5; 8; 13; 21; 34]
35

# let rec fibs =
fun () -> Cons(0,
fun () -> Cons(1,
# first 10 fibs ;;
- : int list = [0; 1; 1; 2; 3; 5; 8; 13; 21; 34]
35
Lazy streams: Fibonacci numbers
36

First two elements
36
...

First two elements
...
Fibonaccis
36
...

First two elements
...
Fibonaccis
...
Tail of Fibonaccis
36
...

First two elements
...
Fibonaccis
...
...
Tail of Fibonaccis
Sum
36
...

First three elements
...
...
...
Fibonaccis
Tail of Fibonaccis
Sum
37
...

First four elements
...
...
...
Fibonaccis
Tail of Fibonaccis
Sum
38
...

First five elements
...
...
...
Fibonaccis
Tail of Fibonaccis
Sum
39
...
Lazy recomputation
40
Lazy recomputation
Every time we access the tail

of a stream we have to rerun
the function.
40
Lazy recomputation
# let rec fibs =
fun () -> Cons(0,
fun () -> Cons(1,
41
Lazy recomputation
# let rec fibs =
fun () -> Cons(0,
fun () -> Cons(1,
Every time we ask for the nth

fibonacci number, we
recalculate all previous ones
more than once!
41
Avoiding recomputation: thunks

# type 'a thunk =
| Unevaluated of unit -> 'a
| Evaluated of 'a
# let rec force (t : 'a thunk ref) : 'a =
match !t with
| Evaluated v -> v
| Unevaluated f ->
t := Evaluated (f());
force t ;;
42

# type 'a thunk =
| Evaluated of 'a
match !t with
| Evaluated v -> v
| Unevaluated f ->
Already evaluated; just return the

previously calculated value
force t ;;
42

# type 'a thunk =
| Evaluated of 'a
match !t with
| Evaluated v -> v
| Unevaluated f ->
force t ;;
42
Evaluate (f()) and

store the calculated
value as a side effect...

# type 'a thunk =
| Evaluated of 'a
match !t with
| Evaluated v -> v
| Unevaluated f ->
force t ;;
42
...and retrieve the value just calculated.
Avoiding recomputation
# let rec fib x =
if x < 2 then x
else (fib (x-1)) + (fib (x-2)) ;;
# let fib42 : int thunk ref =
ref (Unevaluated (fun () -> fib 42)) ;;
# force fib42 ;;
- : int = 267914296
# force fib42 ;;
- : int = 267914296
43
# let rec fib x =
if x < 2 then x
Naive and extremely slow

computation of Fibonacci
numbers
else (fib (x-1)) + (fib (x-2)) ;;

# force fib42 ;;
- : int = 267914296
# force fib42 ;;
- : int = 267914296
43
# let rec fib x =
if x < 2 then x

numbers
else (fib (x-1)) + (fib (x-2)) ;;

# force fib42 ;;
- : int = 267914296
12.7 seconds
# force fib42 ;;
- : int = 267914296
43
# let rec fib x =
if x < 2 then x

numbers
else (fib (x-1)) + (fib (x-2)) ;;

# force fib42 ;;
- : int = 267914296
12.7 seconds
# force fib42 ;;
- : int = 267914296
43
0 microseconds
The OCaml Lazy module

# type 'a thunk =
| Evaluated of 'a
match !t with
| Evaluated v -> v
| Unevaluated f ->
force t ;;
44

# type 'a thunk =
| Evaluated of 'a
match !t with
| Evaluated v -> v
| Unevaluated f ->
force t ;;
44

# type 'a thunk =
| Unevaluated of unit
| Evaluated of 'a
A delayed memoized
computation of an 'a:
-> 'a
'a Lazy.t

match !t with
| Evaluated v -> v
| Unevaluated f ->
force t ;;
44

# type 'a thunk =
| Evaluated of 'a
A delayed memoized
-> 'a
'a Lazy.t

match !t with
| Evaluated
v -> v
Lazy.force:
'a Lazy.t -> 'a
| Unevaluated f ->
force t ;;
44

# type 'a thunk =
| Evaluated of 'a
A delayed memoized
-> 'a
'a Lazy.t

match !t with
| Evaluated
v -> v
Lazy.force:
'a Lazy.t -> 'a
| Unevaluated f ->
force t ;;
44
lazy expr
ref (Unevaluated (fun () -> expr))

# let rec fib x =
if x < 2 then x
else (fib (x-1)) + (fib (x-2)) ;;
# let fib42 : int Lazy.t =
lazy (fib 42) ;;
# Lazy.force fib42 ;;
- : int = 267914296
- : int = 267914296
45

# let rec fib x =
if x < 2 then x
else (fib (x-1)) + (fib (x-2)) ;;
lazy (fib 42) ;;
- : int = 267914296
12.7 seconds
- : int = 267914296
45

# let rec fib x =
if x < 2 then x
else (fib (x-1)) + (fib (x-2)) ;;
lazy (fib 42) ;;
- : int = 267914296
12.7 seconds
- : int = 267914296
45
0 microseconds
and 'a stream = 'a str Lazy.t ;;
match Lazy.force s with
| Cons(h, _) -> h ;;
# ...
46
and 'a stream = 'a str Lazy.t ;;
# ...
# let rec fibs =
lazy (Cons(0,
lazy (Cons(1,
smap2 (+) fibs (tail fibs))))) ;;
# first 10 fibs ;;
- : int list = [0; 1; 1; 2; 3; 5; 8; 13; 21; 34]
47
A history of laziness
1965: Landin describes streams saying There
is a relationship between lists and functions.
In this relationship a nonnull list L is mirrored
by a none-adic function S that produces a 2list consisting of (1) the head of L, and (2) the
function mirroring the tail of L.
Peter J. Landin
48
1965: Landin describes streams
1971: Wadsworth proposes the lazy lambda
calculus
1976: Friedman and Wise publish Cons
should not evaluate its arguments
1980: Burstall proposes Hope, a lazy
functional programming language (with
algebraic data types)
Rod Burstall
49
1965: Landin describes streams
1971: Wadsworth proposes the lazy lambda
calculus
1976: Friedman and Wise publish Cons
should not evaluate its arguments
1980: Burstall proposes Hope, a lazy
functional programming language (with
algebraic data types)
1985: The Miranda lazy functional language is
released
1990: First version of Haskell leads to
convergence of lazy languages
50
Haskell Curry
Application: approximating !
arctan x = x
4
=1
=4
Brook Taylor (16851731)
51
x3 x5
+
3
5
1 1
+
3 5
4 4
+
3 5
x7
+
7
1
+
7
4
+
7
!
arctan x = x
4
=1
=4
51
arctan 1
x3 x5
+
3
5
1 1
+
3 5
4 4
+
3 5
x7
+
7
1
+
7
4
+
7
!
arctan x = x
4
=1
=4
51
arctan 1
x3 x5
+
3
5
1 1
+
3 5
4 4
+
3 5
x7
+
7
1
+
7
4
+
7
!
arctan x = x
4
=1
=4
51
arctan 1
x3 x5
+
3
5
1 1
+
3 5
4 4
+
3 5
x7
+
7
1
+
7
4
+
7
Application:
=4
4/3 + 4/5
4/7 +
let to_float = smap float_of_int ;;

let odds = smap (fun x -> x * 2 + 1) nats ;;
let neg_evens =
smap (fun x -> if x mod 2 = 0 then -x else x) ;;
let alt_signs =
smap2 ( / ) (neg_evens (tail nats)) (tail nats) ;;
let pi_stream = smap2 ( /. )
(to_float (smap (( * ) 4) alt_signs))
(to_float odds) ;;
52
Application:
=4
4/3 + 4/5
4/7 +
# first 5 odds ;;
- : int list = [1; 3; 5; 7; 9]
# first 5 alt_signs ;;
- : int list = [1; -1; 1; -1; 1]
# first 5 pi_stream ;;
- : float list =
[4.; -1.33333333333333326; 0.8; -0.571428571428571397;
0.44444444444444442]
53
Application:
=4
4/3 + 4/5
4/7 +
# let pi_approx n =
List.fold_left ( +. ) 0.0 (first n pi_stream) ;;
# pi_approx 10 ;;
- : float = 3.04183961892940324
# pi_approx 100 ;;
- : float = 3.13159290355855369
# pi_approx 1000 ;;
- : float = 3.14059265383979413
# pi_approx 10000 ;;
- : float = 3.14149265359003449
# pi_approx 100000 ;;
- : float = 3.14158265358971978
54
Partial sums
55
a series
partial sums
6 10 15 21 28 ...
shifted partial sums 0
6 10 15 21 ...
+ series
= partial sums
6 10 15 21 28 ...
7 ...
7 ...
Partial sums
56
a series
partial sums
6 10 15 21 28 ...
6 10 15 21 ...
+ series
= partial sums
6 10 15 21 28 ...
7 ...
7 ...
Partial sums
57
a series
partial sums
6 10 15 21 28 ...
6 10 15 21 ...
+ series
= partial sums
6 10 15 21 28 ...
7 ...
7 ...
Partial sums
58
a series
partial sums
6 10 15 21 28 ...
6 10 15 21 ...
+ series
= partial sums
6 10 15 21 28 ...
7 ...
7 ...
Partial sums
a series
partial sums
6 10 15 21 28 ...
6 10 15 21 ...
+ series
= partial sums
6 10 15 21 28 ...
7 ...
7 ...
let rec sums s =

smap2 ( +. ) s (lazy (Cons(0.0, sums s))) ;;
58
Application:
=4
4/3 + 4/5
4/7 +
# let rec sums s =

# let pi_sums = sums pi_stream ;;
# first 10 pi_sums ;;
- : float list = [4.; 2.66666666666666696;
3.46666666666666679; 2.89523809523809561;
3.33968253968254025; 2.97604617604617649;
3.28373848373848443; 3.01707181707181782;
3.25236593471887669; 3.04183961892940324]
#
59
Application:
=4
4/3 + 4/5
4/7 +
# let rec sums s =

# let rec within eps s =
let (h, t) = (head s, tail s) in
if abs_float (h -. (head t)) < eps then h
else within eps t ;;
#
60
Application:
=4
4/3 + 4/5
4/7 +
# let rec sums s =

# let rec within eps s =
let (h, t) = (head s, tail s) in
if abs_float (h -. (head t)) < eps then h
else within eps t ;;
# within 0.01 pi_sums ;;
- : float = 3.13659268483881615
# within 0.0001 pi_sums ;;
- : float = 3.14154265358982476
61
Laziness elsewhere
OCaml is an eager language by default but with minimal support
for lazy constructs
Other functional languages (especially Haskell) are lazy by default
Much easier to program with infinite structures
Harder to reason about space and time
Complex interactions with side effects
Echoes in many areas, especially concurrency:
Unix pipes
TCP sockets
async libraries
62
7: The substitution model
63
The substitution model revisited

To evaluate
let x = Q in P:
Evaluate Q to Q'
Evaluate P[x Q'] to R
Result is R
64

To evaluate
let x = Q in P:
Evaluate Q to Q'
Result is R
64
This doesn't work for

let rec
let rec f =
fun n -> if n = 0 then 1
else n * f (n-1) in
f 3
65
let rec f =
else n * f (n-1) in
f 3
(fun n -> if n = 0 then 1

else n * f (n-1)) 3
65
let rec f =
else n * f (n-1) in
f 3

else n * f (n-1)) 3
if 3 = 0 then 1
else 3 * f (n-1)
65
let rec f =
else n * f (n-1) in
f 3

else n * f (n-1)) 3
if 3 = 0 then 1
else 3 * f (n-1)
Free variable?!
65

To evaluate
let x = Q in P:
Evaluate Q to Q'
Result is R
66

To evaluate
let rec x = Q in P:
Evaluate Q[x let rec x = Q in x] to Q'
Result is R
67
let rec f =
else n * f (n-1))
in f 2

else n * (let rec f =
else n * f (n-1))
in f)
(n-1))
2
if 2 = 0 then 1
else 2 * (let rec f =
68
else n * f (n-1))
in f 2

else n * f (n-1))
in f)
(n-1))
2
if 2 = 0 then 1
else n * f (n-1))
in f)
(2-1)
if false then 1
69
if 2 = 0 then 1
else n * f (n-1))
in f)
(2-1)
if false then 1
else n * f (n-1))
in f)
(2-1)
2 * (let rec f =
70

else n * f (n-1))
in f)
(2-1)
if false then 1
else n * f (n-1))
in f)
(2-1)
2 * (let rec f =
else n * f (n-1))
in f)
(2-1)
2 * (let rec f =
71
else n * f (n-1))
in f)
(2-1)
2 * (let rec f =
else n * f (n-1))
in f)
1
2 * (fun n -> if n = 0 then 1

else n * f (n-1))
in f) (n-1))
1
2 * (if 1 = 0 then 1
72
2 * (1 * (if 0 = 0 then 1
else n * f (n-1))
in f) (0-1)))
2 * (1 * (if true then 1

else n * f (n-1))
in f) (0-1)))
2 * (1 * 1)
2 * 1
2
73
2 * (1 * (if 0 = 0 then 1
else n * f (n-1))
in f) (0-1)))
2 * (1 * (if true then 1

This is not the kind of exercise

human beings should have to do.
else n * f (n-1))
in f) (0-1)))
2 * (1 * 1)
2 * 1
2
73

CS51: Abstraction and Design in Computation Using Lazy Streams

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CS51: Abstraction and Design in Computation Using Lazy Streams

Uploaded by

Copyright:

Available Formats

CS51:

Abstraction and Design in Computation

John A. Paulson School of Engineering and Applied Sciences

11: Lazy evaluation and

find this code in: streams.ml

find this code in: streams.ml

Integer stream goes here.

find this code in: streams.ml

find this code in: streams.ml

# type 'a stream =

# let ones = Cons(1, ones) ;;

find this code in: streams.ml

How to think about streams

How to think about streams

Allocate space for the cons

How to think about streams

Allocate space for the cons

How to think about streams

Initialize the contents of the

How to think about streams

Initialize the contents of the

How to think about streams

This explains where the

More recursive data

More recursive data

Allocate space for the int

More recursive data

Allocate space for the int

More recursive data

Initialize the contents with

More recursive data

Stream operations: mapping

Stream operations: mapping

find this code in: streams.ml

Stream operations: mapping

find this code in: streams.ml

Stream operations: mapping

fun x -> Cons(f (head s), smap f (tail s))

Postpone computation by wrapping in a function

Mutually recursive type

find this code in: streams.ml

find this code in: streams.ml

and 'a stream = unit -> 'a str ;;

# let head (s : 'a stream) : 'a =

find this code in: streams.ml

and 'a stream = unit -> 'a str ;;

# let head (s : 'a stream) : 'a =

# let rec smap (f : 'a -> 'b) (s : 'a stream)

find this code in: streams.ml

# let rec smap (f : 'a -> 'b) (s : 'a stream)

fun () -> Cons(f (head s), smap f (tail s)) ;

find this code in: streams.ml

# let rec smap (f : 'a -> 'b) (s : 'a stream)

# let twos = smap ((+) 1) ones ;;

find this code in: streams.ml

# let rec smap (f : 'a -> 'b) (s : 'a stream)

# let twos = smap ((+) 1) ones ;;

...over an infinite sequence of ones

# head (tail twos) ;;

find this code in: streams.ml

Lazy streams: extracting prefix

find this code in: streams.ml