(rev. 2017/10/13)

Analysis of the Recurrence Relation for Merge Sort

For merge sort, the work function T() satisfies this recurrence:

(1) T(n) ≤ 2T(n/2) + Cn, for all n such that 4 ≤ n ∈ ℤ

where n is the size of the list to be sorted (assumed for simplicity to be a power of 2), C is some positive constant, and the value of the work function T(n) is the maximum number of "simple operations" required to mergesort a list of size n or less. We also assume the following base case

(2) T(2) ≤ 2C

which simply means that the mergesort of a two-element list requires some minimal amount of work, bounded by the constant, 2C. We can assume that the values of C in (1) and (2) are the same. If they happen to be different, if we have C1 and C2, then choosing C = max{C1, C2} provides a value that makes the inequalities in both (1) and (2) true.

So we have:

(1) T(n) ≤ 2T(n/2) + Cn, for all n such that 4 ≤ n ∈ ℤ
(2) T(2) ≤ 2C

Expressing n = 2k as a power of 2, we get

(3) T(2k) ≤ 2T(2(k-1)) + C2k, for all k such that 2 ≤ k ∈ ℤ
(4) T(21) ≤ 2C

How can we get an expression for T in closed form? We can proceed like this:

Using (3): T(4) = T(22) ≤ 2T(21) + C22

Using (4), this gives us T(4) ≤ 2(2C) + C22 = 8C = 2C22

In summary, we have
(2) T(21) ≤ 2C
(5) T(22) ≤ 2C22

Using (3): T(8) = T(23) ≤ 2T(22) + C23
Using (5), this gives us T(8) ≤ 2[2C22] + C23 = 3C23

In summary, we have
(2) T(21) ≤ 2C = 1C21
(5) T(22) ≤ 2C22
(6) T(23) ≤ 3C23

Using (3): T(16) = T(24) ≤ 2T(23) + C24
Using (6), this gives us T(16) ≤ 2[3C23] + C24 = 4C24

In summary, we have
(2) T(21) ≤ 2C = 1C21
(5) T(22) ≤ 2C22
(6) T(23) ≤ 3C23
(7) T(24) ≤ 4C24

So there appears to be a pattern that shows up, to the effect that

(8) T(2k) ≤ kC2k, for all k such that 1 ≤ k ∈ ℤ

Since k=log2(2k), if (8) is true, then T(n) is O(nlog(n)), (for values of n that are powers of 2).

Now that we have found the pattern and formed the theory that (8) is true, can we confirm the theory? Yes, we can prove it, using mathematical induction.

Proposition: Suppose

(3) T(2k) ≤ 2T(2(k-1)) + C2k, for all k such that 2 ≤ k ∈ ℤ
(4) T(21) ≤ 2C

Then
(8) T(2k) ≤ Ck2k, for all k such that 1 ≤ k ∈ ℤ

Proof:

Base case: for k=1, we have, by (4), T(21) ≤ 2C = C(1)21, which satisfies (8).

So now assume the inductive hypothesis, that (8) is true for some k=s, where 1 ≤ s ∈ ℤ, in other words

The inductive hypothesis is: T(2s) ≤ Cs2s

Now, thinking about T(2(s+1)), (3) tells us that

T(2(s+1)) ≤ 2T(2s) + C2s+1

and the inductive hypothesis allows us to conclude that

T(2(s+1)) ≤ 2Cs2s + C2s+1
= Cs2s+1 + C2s+1
= (s)C2s+1 + (1)C2s+1
= (s+1)C2s+1
= C(s+1)2s+1

That completes the inductive step, and so we have proved that Merge Sort is O(nlog(n)), for the case where n is a power of 2.

We have also demonstrated an example of a general technique for determining the big-O of divide-and-conquer algorithms that have work functions T that satisfy recurrences like this:

T(N) ≤ qT(N/b) + CNh, for all suitable integers N ≥ b, where q, b, and h are positive integers and C is a positive constant.