Power
Reasoning About Programs: More Data Types
In the last note, we discussed how to prove things about ML programs that manipulate lists. The same basic ideas can be used to prove things about programs that manipulate all kinds of inductive data structures. In general, you can use induction to reason about any data structure you can define using ML data types, tuples, and basic types like characters, natural numbers and (immutable) strings. All those data structures have the property that one can easily define their "size". Moreover, if you start with some particular data structure d1, there is never an infinite series of data structures d1, d2, d3, ... such that:
size(d1) > size(d2) > size(d3) > ...For example, if you start with any natural number n (n is perhaps 3 or 16 or 5233), there is always only a finite number of other naturals that are smaller than it. Likewise, if you start with a list l there is no infinitely long sequence of lists starting with l for which every list in the sequence is smaller (has a shorter length) than the previous one. This means that our inductive reasoning is sound because it will always "bottom out."
Contrast that situation with the integers (or the real numbers). If you start with the integer 3, there are infinitely many smaller integers (2, 1, 0, -1, -2, -3, ...). Hence, we cannot do induction on the integer data type (at least not directly).
In this note, we shall take a look at reasoning techniques for programs that manipulate natural numbers, trees and other data types.
Natural Numbers
Recall that every natural n is either:
- 0, or
- n' + 1 where n' is also a natural number
OCaml does not have a built-in type for natural numbers (unfortunately!) so we have to represent these natural numbers somehow using the data structures that O'Caml has. Normally, we would represent the natural numbers using the integers. This is a little bit ugly when doing proofs because the integers contain "extra numbers" (-1, -2, ...) that are not naturals, but we can certainly do it and carry out lots of interesting proofs. Instead of doing that however, let's represent the naturals exactly, with no extra garbage, using a data type.
type nat = Z | S of nat;;
Z will stand for the natural 0
and S n' will stand for the natural n' + 1.
(S n' is often pronounced "the successor of n'").
For example, the number 1 will be represented using S Z and
the number 3 will be represented using (S (S (S Z))). Indeed,
writing a function to convert natural numbers to integers is easy enough:
let rec to_int (n:nat) : int =
match n with
Z -> 0
| S n' -> 1 + to_int n'
;;
A useful theorem about the conversion from naturals to integers follows.
Theorem 1: For any valuable expression v,
to_int (S v) == 1 + to_int v.
Proof: By simple equational reasoning.
to_int (S v) == match S v with Z -> 0 | S n' -> 1 + to_int n' (eval value) == 1 + to_int v (eval value)
Now, let's consider the function double on naturals below.
let rec double (n:nat) : nat =
match n with
Z -> Z
| S n' -> S (S (double n'))
;;
How do we prove the following theorem?
Theorem 2: for all natural numbers n,
to_int (double n) == 2 * to_int n.
To do our proof, we will break down the reasoning in to two cases, one for Z and one for S n' -- this covers both cases of the data type definition. (As usual, the structure of the data guides the structure of our programs and the structure of our proofs.) When considering the case for S n', we may assume that our induction hypothesis holds on n':
to_int (double n') == 2 * to_int n'We obtained that induction hypothesis just by substituting n' for n in to the statement of the theorem that we were trying to prove. (We are justified in doing so because n' is a smaller natural number than n.) Ok, here is the proof:
Proof: By induction on structure of the natural number n.
case n = Z:
to_int (double Z)
== to_int (match Z with Z -> Z | S n' -> S (S (double n'))) (eval)
== to_int Z (eval)
== 0 (eval)
== 2*0 (math)
== 2*(to_int Z) (eval reversed)
case n = S n':
to_int (double (S n'))
== to_int (match S n' with Z -> Z | S n' -> S (S (double n'))) (eval value)
== to_int (S (S (double n'))) (eval value)
== 1 + to_int (S (double n')) (Theorem 1)
== 1 + 1 + to_int (double n') (Theorem 1)
== 2 + to_int (double n') (math)
== 2 + 2 * to_int n' (IH)
== 2*(1 + to_int n') (math)
== 2*(to_int (S n')) (Theorem 1)
And there you have it!
Representing Naturals as Integers
Just to show that there is nothing special about representing natural numbers using data types, we will carry out the same proof a second time, this time representing natural numbers as integers. Consider the new code below.
type nat = int;; let to_int (n:nat) : int = n;; let rec double (n:nat) : nat = if n <= 0 then 0 else 1 + 1 + double (n-1) ;;
In order to show the similarities between the two proofs, the last line of the double function is phrased:
1 + 1 + double (n-1)which of course is the same as:
1 + (1 + (double (n-1)))and that expression mimics the last line of the prior double function:
S (S (double n'))as "S" represents "1 +" and n' represents "n-1" (when n is S n').
As before, we will prove a straightforward theorem about the to_int function:
Theorem 1': For any valuable natural number v,
to_int (1+v) == 1 + to_int v.
Proof: By simple equational reasoning.
to_int (1 + v) == 1 + v (eval value) == 1 + to_int v (eval value, reverse)Now let's prove our main theorem.
Theorem 2': for all natural numbers n,
to_int (double n) == 2 * to_int n.
Proof: By induction on the structure of natural numbers. In the case where n = n'+1, the induction hypothesis is
to_int (double n') == 2 * to_int n'We obtain our induction hypothesis by substituting the smaller natural number n' for n in the statement of the theorem. Now the proof:
case n = 0:
to_int (double 0)
== to_int (if 0 <= 0 then 0 else 1 + 1 + double (0-1)) (eval)
== 0 (eval)
== 2*0 (math)
case n = n' + 1
to_int (double (n'+1))
== to_int (if n'+1 <= 0 then 0 else 1 + 1 + double (n'+1-1)) (eval value)
== to_int (1 + 1 + double (n'+1-1)) (eval value)
== to_int (1 + 1 + double n') (math*)
== 1 + to_int (1 + double n') (Theorem 1')
== 1 + 1 + to_int (double n') (Theorem 1')
== 2 + to_int (double n') (math)
== 2 + 2 * to_int n' (IH)
== 2*(1 + to_int n') (math)
== 2*(to_int (n'+1)) (Theorem 1')
If you look back at the proof of the program using data types, you will find it is
almost identical. The one difference is the line marked math*
where when we used data types, the pattern matching facilities
had the effect of performing the arithmetic for us. I like the
proof involving data types, and in general I like programming with data types more
than integers because I don't have to worry about doing arithmetic in my head and
making a mistake (something I do all the time). Managing data types
and pattern matching is easier so I do it that way when I can. Unfortunately, however,
representing numbers in unary using data types is too inefficient to do in genereal.
Proofs about Trees
Consider the following definition of trees and a couple of functions over trees.
type tree = Leaf | Node of int * tree * tree;;
let rec lmember (l:int list) (x:int) : bool =
match l with
[] -> false
| hd::tail -> hd = x || lmember tail x
;;
let rec tmember (t:tree) (x:int) : bool =
match t with
Leaf -> false
| Node (j,left,right) -> j = x || tmember left x || tmember right x
;;
(* converts a tree in to a list while preserving the contents of the data structure.
* More precisely:
*
* for all t:tree,
* for all x:int, tmember t x == lmember (listify t) x
*
*)
let rec listify (t:tree) : int list =
match t with
Leaf -> []
| Node (i,left,right) -> (listify left) @ [i] @ (listify right)
;;
Recall also that the standard library function list append (@) is defined as follows.
let rec (@) (l1:'a list) (l2:'a list) : 'a list =
match l1 with
[] -> l2
| hd::tail -> hd::(tail @ l2)
;;
Now, what we would really like to know is whether listify satisfies
it's informal specification: Does it really transform a tree in to a list while
preserving the contents? More precisely, is the following theorem true?
Conjecture: For all t:tree, for all x:int, tmember t x == lmember (listify t) x.
It turns out that the conjecture is true, but before we can prove it, we need to
know a simple fact about list membership and its relationship with the list concatenation
function @. The following lemma says that checking for membership of x in the
concatenation of two lists is the same as checking for membership in each of the lists individually and
combining the results in the appropriate way (ie, using disjunction).
Lemma 3: For all l1:int list, l2:int list, for all x:int,
lmember (l1 @ l2) x == lmember l1 x || lmember l2 x.
Proof: By induction on the structure of the list l1.
case l1 == []:
To show:
lmember ([] @ l2) x == lmember [] x || lmember l2 x
Proof:
lmember ([] @ l2) x
== lmember l2 x (eval @)
== false || lmember l2 x (property of || and false)
== lmember [] x || lmember l2 x (eval lmember in reverse)
case l1 == hd::tail:
IH: lmember (tail @ l2) x == lmember tail x || lmember l2 x
To show:
lmember ((hd::tail) @ l2) x == lmember (hd::tail) x || lmember l2 x
Proof:
lmember ((hd::tail) @ l2) x
== lmember (hd::(tail @ l2)) x (eval @)
== hd = x || lmember (tail @ l2) x (eval lmember,
hd::(tail @ l2) valuable)
== hd = x || (lmember tail x || lmember l2 x) (IH)
== (hd = x || lmember tail x) || lmember l2 x (|| is associative)
== lmember (hd::tail) x || lmember l2 x (eval lmember in reverse)
QED!
With our lemma proven, we can move on to the proof of the theorem we are actually interested in. We'll state the theorem and then we'll show the proof. After the proof, we'll discuss some of the interesting parts of it.
Theorem 4: For all t:tree, for all x:int, tmember t x == lmember (listify t) x.
Proof: By induction on the structure of the tree t.
case t == Leaf:
To show: tmember Leaf x == lmember (listify Leaf) x
Proof:
tmember Leaf x
== false (eval tmember)
== lmember [] x (eval lmember, reverse)
== lmember (listify Leaf) x (eval listify, reverse)
case t == Node(i, left, right):
IH1: tmember left x == lmember (listify left) x
IH2: tmember right x == lmember (listify right) x
To show: tmember (Node(i, left, right)) x
== lmember (listify (Node(i, left, right))) x
Proof:
tmember (Node(i, left, right)) x
== i = x ||
tmember left x ||
tmember right x (eval tmember)
== i = x ||
lmember (listify left) x ||
tmember right x (IH1)
== i = x ||
lmember (listify left) x ||
lmember (listify right) x (IH2)
== lmember (listify left) x ||
i = x ||
lmember (listify right) x (commutativity of ||)
== lmember (listify left) x ||
lmember [i] x ||
lmember (listify right) x (eval lmember, reverse)
== lmember (listify left @ [i]) x ||
lmember (listify right) x (lemma 3, symmetry)
== lmember (listify left @ [i] @ listify right) x (lemma 3, symmetry)
== lmember (listify (Node (i,left,right))) x (eval listify, reverse)
QED!!
Ok, so the things to notice about the proof are primarily as follows:
- Proofs about trees typically have two cases. This is unsurprising because the datatype definition of trees suggests that there are two kinds of trees: Trees that are just a Leaf, and trees that are Nodes carrying some integer i, a left subtree and a right subtree. There is one case of the proof for each kind of tree.
- The case for Leaf is ususually pretty straightforward, and it is straightforward here. It is straightforward for a similar reason that the case for the empty list [] is typically straightforward -- there is no recursion here so we do not have to use an induction hypothesis.
- The case for Node(i,left,right) is more involved. The first thing to notice is that there are two induction hypotheses: An induction hypothesis for the left subtree and an induction hypothesis for the right subtree. The reason there are two is because both the left and the right subtrees are strictly smaller than the tree Node(i,left,right), which contains them.
Other than those items, a proof about trees is quite similar to a proof about lists. All the same equational reasoning techniques apply. I hope also that you can see the general pattern involved in reasoning about data types. Typically, an inductive proof of a property P for all elements of a datatype d defined as follows:
type d =
B1 of t1
| ...
| Bm of tm
| C1 of s1 * d * ... * d
...
| Cn of sn * d * ... * d
will involve m + n different cases -- one case for each element of the datatype. Moreover, the base cases B1 ... Bm, which do not involve recursive elements of d, will be proven without use of an inductive hypothesis. The cases C1 ... Cn, which do refer recursively to the type d will have inductive hypotheses. There will be one inductive hypothesis for each smaller occurrence of the recursive data structure.
Summary
One can carry out proofs by induction on data such as lists, trees, and natural numbers.
Proofs on natural numbers
Proofs by induction on the natural numbers n typically have two cases: one case for n=0 and one case for n=n'+1 where n' is also a natural number. The inductive hypothesis may be applied to any number smaller than n (and typically to the number n' in the second case of the proof). Whether we represent natural numbers using integers or data types, the structure of the proofs stays the same. Here is a template for a proof of property P(n) for any natural number n.
Proof: Proof by induction on the natural number n.
case n = 0:
To show: P(0)
... proof that property P holds of 0
case n = n' + 1:
To show: P(n' + 1)
IH: P(n')
... proof that property P holds of n' + 1
... use IH on n' ...
Proofs on trees
Proofs by induction on trees t typically have two cases: one for t = Leaf and one for t = Node(i,left,right). Here is a template for a proof of property P(t) for any tree t.
Proof: Proof by induction on the tree t.
case t = Leaf:
To show: P(Leaf)
... proof that property P holds of Leaf ...
case n = Node(i,left,right):
To show: P(Node(i,left,right))
IH1: P(left)
IH2: P(right)
... proof that property P holds of Node(i,left,right)
... use IH1 on left and IH2 on right ...