CS 541 Lecture -*- Outline -*- * Streams (Sequences) and Lazy evaluation see page 166 on in ML and the working programmer true call by need found in section 8.4, but uses refs ** Goals *** capture common patterns (paradigms) once and for all *** efficiently and perspiciously express certain algorithms *** Example from Abelson and Sussman, 3.4.1 show code first, then discuss **** compute sum of squares of leaves of a tree that are odd enumerate -> filter(odd) -> map(square) -> accumulate(op +)(0) ------------------------------------------ (* SUM SQUARES OF ODD LEAVES *) fun odd x = (x mod 2 = 1); fun square (x:int) = x * x; fun sum_odd_squares Lf = 0 | sum_odd_squares (Br(v,t1,t2)) = (if odd(v) then square(v) else 0) + sum_odd_squares t1 + sum_odd_squares t2; ------------------------------------------ **** construct list of all odd Fibonacci numbers less than or equal to n enumerate -> map(fib) -> filter(odd) -> accumulate(op ::)([]) ------------------------------------------ (* LIST THE ODD FIBONACCI NUMBERS *) fun fib 0 = 0 | fib 1 = 1 | fib n = fib(n-2) + fib(n-1); fun odd_fibs n = let fun next k = if k > n then [] else let val f = (fib k) in if odd(f) then f :: next(1 + k) else next (1 + k) end in next(1) end; ------------------------------------------ Q: how are these similar? Q: what parts of each are enumeration? mapping? filtering?accumulation? want to separate out the enumeration, mapping, filtering, accumulation ** lazy lists in SML traditionally lazy lists called streams, Paulson calls them sequences because stream in SML is for I/O ------------------------------------------ (* SEQUENCES (aka. LAZY LISTS, STREAMS) *) datatype 'a seq = Nil | Cons of 'a * (unit -> 'a seq); fun head (Cons(x,xf)) = x; fun force f = f(); fun tail (Cons(x,xf)) = force xf; fun consq (x,xq) = Cons(x, fn() => xq); ------------------------------------------ tail has to force its argument. Note: consq is not lazy, so to do lazy evaluation have to use Cons(x, fn () => E); ------------------------------------------ val s1 = Nil; val s1 = Nil : 'a seq ------------------------------------------ val s2 = Cons(2, fn () => Nil); val s2 = Cons (2,fn) : int seq might want to give more examples ... No magic, could use macros if we had them to define ------------------------------------------ (* M4 MACROS FOR LAZY LISTS *) define(delay, (fn () => $1)) define(lazy_cons, Cons($1,delay($2))) ------------------------------------------ delay(E) as (fn () => E) cons_seq(E1,E2) as Cons(E1,delay(E2)) can't use these directly in ML programs, have to use m4 first, then feed it's output to SML (e.g., note examples streams-lazy.txt | m4 >streams-lazy.sml ) so lazy_cons(1,Nil) = Cons(1,delay(Nil)) = Cons(1,(fn () => Nil)) *** express the basic operations using functionals over streams **** enumeration of tree leaves ------------------------------------------ (* ENUMERATION OF TREE LEAVES *) val rec appendq = fn (Nil, yq) => yq | (Cons(x,xf), yq) => lazy_cons(x, appendq(xf(),yq)); val rec enumerate_tree = fn Lf => Nil | Br(v,t1,t2) => appendq(consq(v,Nil), appendq(enumerate_tree(t1), enumerate_tree(t2))); ------------------------------------------ **** enumeration of integers from 1 to k ------------------------------------------ (* ENUMERATION OF INTEGERS FROM 1 to k *) val to = fn k => let val rec to_aux = fn (i,j) => if i > j then Nil else lazy_cons(i, to_aux(i+1, j)) in to_aux(1,k) end; val rec from = fn k => lazy_cons(k, from(k+1)); ------------------------------------------ to(3) = to_aux(1,3) = lazy_cons(1,to_aux(2,3)) = Cons(1,(fn () => to_aux(2,3))) **** accumulation ------------------------------------------ (* ACCUMULATION *) fun accumulate combiner initial = fn Nil => initial | Cons(x,xf) => combiner(x, (accumulate combiner initial (xf ()))); ------------------------------------------ (* type is ('a * 'b -> 'b) -> 'b -> 'a seq -> 'b *) **** filtering ------------------------------------------ (* FILTERING *) fun filterq pred = fn Nil => Nil | Cons(x,xf) => if pred(x) then lazy_cons(x, filterq pred (xf())) else filterq pred (xf()); ------------------------------------------ **** mapping ------------------------------------------ (* MAPPING *) val rec mapq = fn f => fn Nil => Nil | Cons(x,xf) => lazy_cons(f(x), mapq f (xf())); ------------------------------------------ *** can now put the pieces together ------------------------------------------ (* PUTTING THE PIECES TOGETHER *) val sum_odd_squares = fn tree => (accumulate (op +) 0 (mapq square (filterq odd (enumerate_tree tree)))); val list_odd_fibs = fn n => (accumulate (op ::) [] (filterq odd (mapq fib (to n)))); ------------------------------------------ easy to combine these in different ways: e.g., list of squares of fibionnaci numbers list of odd squares... ** Power of map, filter, accumulate paradigm R. Waters (1979), about 60% of traditional fortran programs have loops that can be viewed as instances of map, filter and accumulate *** representing standard data as streams polynomials as stream of coefficients vectors as streams of numbers matricies as streams of vectors ** Nested Mappings (can omit) *** Example, enumerate pairs i,j such that j < i < n and i+j is prime **** idea ------------------------------------------ (* NESTED MAPPINGS *) (* PROBLEM: To enumerate pairs (i,j) such that j < i < n and i+j is prime PLAN: enumerate 1 to n -> form pairs -> filter(for primes) *) ------------------------------------------ form pairs by enumerating j < i, form pair: (i,j) the following computes stream of streams of pairs (mapq (fn i => (mapq (fn j => (i, j)) (to(i-1)))) (to n)) **** some tools ------------------------------------------ (* TOOLS *) val flatten = accumulate appendq Nil; (* type is 'a seq seq -> 'a seq *) fun flatmap f s = flatten (mapq f s); ------------------------------------------ **** code for prim-sum-pairs ------------------------------------------ (* REALIZATION OF PLAN *) fun prime(x:int) = (* ... *) true; val prime_sum_pairs = fn n => (mapq (fn (x,y) => (x,y,(x+y))) (filterq (fn (x:int,y) => prime (x+y)) (flatmap (fn i => (mapq (fn j => (i,j)) (to (i-1)))) (to n)))); ------------------------------------------ ** Stream comprehension (omit) like { x | x > 0 }, but for sequences for above example, want to say form stream of all pairs (i, j) where i is element of to(n) and j is element of to(i-1) such that prime(i + j) *find these parts in code above* what does this call for? *syntactic sugar* a la KRC (D. Turner) val rec prime_sum_pairs = fn n => collect (i, j) such_that i in to(n), j in to(i-1), prime(i+j) why does this have to be a sugar? ** Lazy evaluation, computing with infinite objects *** Infinite sequences (streams) **** explicit definition ------------------------------------------ (* INFINITE SEQUENCES (STREAMS) *) val integers = from(1); ------------------------------------------ **** computations with infinite streams page 171 sieve takes a stream S and returns a stream consisting of head of S and all numbers in the tail not divisible by the head ------------------------------------------ val divisible = fn (x,y) => (x mod y) = 0; val rec sieve = fn Cons(p,nf) => lazy_cons(p, sieve(filterq (fn x => not(divisible(x,p))) (nf ()))); val primes = sieve (from 2); val rec nth_stream = fn n => fn Cons(x,xf) => if n = 0 then x else (nth_stream (n-1) (xf())); nth_stream 50 primes; (* Value 233 *) ------------------------------------------ **** implicit definition use lazy evaluation ------------------------------------------ (* IMPLICITLY DEFINED SEQUENCES *) val rec ones = fn () => lazy_cons(1, ones ()); val rec add_seq = fn (Cons(x1:int,xf1), Cons(x2,xf2)) => lazy_cons(x1+x2, add_seq(xf1(), xf2())); val rec integers = fn () => lazy_cons(1, add_seq(ones (), integers ())); val rec fibs = fn () => lazy_cons(0, lazy_cons(1, add_seq(tail (fibs()), fibs()))); ------------------------------------------