recurrence relations

(rev. 2017/10/15)

Recurrence Relations

This document shows how to solve a few recurrence relations for some special cases of work functions. Reading this may help you to get a feel for the kind of calculations that work well. Usually we'll deal with work functions, T, that tell us something about how much work, T(n), an algorithm A does depending on the size, n, of the problem. Typically T(n) will denote the maximum amount of work (measured as the number of suitably-chosen simple operations) done by A, on a problem of size n or less. In addition, we'll assume T(n) > 0 for all n > 0. For purposes of simplifying this discussion, assume n = 2^m is a power of two. Suppose also, for some positive constant c and integer q > 2 (1) T(n) ≤ qT(n/2) + cn when n > 1 and (2) T(1) ≤ c (An example of this would be where T(n) ≤ 3T(n/2) + 8.75n for n > 1 and T(1) ≤ 8.75) By line (1), T(2) ≤ qT(1) + 2c, which by line (2) is less than or equal to cq+2c = c(q+2) Again by line (1), T(4) ≤ qT(2) + 4c ≤ q(c(q+2)) + 4c = c(q²+2q+4) Using line (1) a third time, we get T(8) ≤ qT(4) + 8c ≤ q(c(q²+2q+4)) + 8c = c(q³+2q²+4q+8) So we see there is a pattern: T(2^m) ≤ c(q^m + 2q^m-1 + ... + 2^m-1q + 2^m), for all integers m>1 (We could prove this by induction on m.) Some algebra yields: T(2^m) ≤ c2^m((q/2)^m + (q/2)^m-1 + ... + (q/2) + 1), for all integers m>1 = c2^m((q/2)^m+1-1)/((q/2)-1) ( We used the following identity to get the last equality above. (x-1)(x^m + x^m-1 + ... + x + 1) = (x^m+1-1) ) So we have T(2^m) ≤ c2^m((q/2)^m+1-1)/((q/2)-1) (By the way, because q is not equal to 2, (q/2)-1 is not zero) Remembering that 2^m = n, noting that m = log₂(n), and using log to denote the base-2 logarithm, we see that we have T(n) ≤ cn(r^log(n)+1-1)/(r-1), where r=q/2 (a constant), which is the same as T(n) ≤ cn(r^log(n)r¹-1)/(r-1) We see that T(n) is O(nr^log(n)). (Proof? Verify that the ratio [cn(r^log(n)r¹-1)/(r-1)]/[nr^log(n)] converges to [cr/(r-1)] as n--->∞, and ([cr/(r-1)] is a constant.) Also, since log(r^log(n)) = log(n)log(r) = log (n^log(r)), we see that r^log(n) = n^log(r) = n^log(q/2) = n^log(q)-1 And so nr^log(n) = n(n^log(q)-1) = n^log(q). And since T(n) is O(nr^log(n)), T(n) is O(n^log(q)). Since q > 2, O(n^log(q)) is "polynomial time". For example, when q=4, that means log(q)=2 and T(n) is O(n²). And if q=3, then log(q) is less than 1.59, so in that case T(n) is O(n^1.59) Note that the calculation above is valid if q=1. When q=1 we find that T(2^m) ≤ c(q^m + 2q^m-1 + ... + 2^m-1q + 2^m) = c(1 + 2 + 4 + ... + 2^m-1 + 2^m) = c(2^m+1-1) = 2c(2^m-(1/2)) = 2c(n-(1/2)) So it's clear that T(n) is O(n) when q=1. When q=2, we can carry the derivation above to the point where we get: T(2^m) ≤ c2^m((q/2)^m + (q/2)^m-1 + ... + (q/2) + 1) and at this point it becomes clear that T(2^m) ≤ c2^m(1^m + 1^m-1 + ... + 1¹ + 1⁰) and this just says that T(2^m) ≤ c2^m(1^m + 1^m-1 + ... + 1¹ + 1⁰) = c(m+1)2^m Since n=2^m, this last expression is clearly O(nlog(n)). Suppose we have a recurrence where n = 2^m and for some positive constant c (3) T(n) ≤ 2T(n/2) + cn² when n > 1 and (4) T(1) ≤ c In this case we have T(2) ≤ 2T(1) + c2² ≤ c(2+2²) = c(2+4) T(4) ≤ 2T(2) + c4² ≤ 2c(2+2²) + c4² = c(4+8+16) T(8) ≤ 2T(4) + c8² ≤ 2c(4+8+16) + c64 = c(8+16+32+64) The pattern developing is: T(2^m) ≤ c(2^m + 2^m+1 + ...+ 2^m+m) = c2^m (1 + 2 + 4 + 8 + ...+ 2^m) = c2^m(2^m+1-1) This is clearly O(n²).