(rev. 2017/03/26)

Analysis of the Recurrence Relation for Merge Sort

For merge sort, the work function T() satisfies this recurrence:

(1) T(n) ≤ 2T(n/2) + Cn, for all n such that 2 ≤ n ∈ ℤ

where n is the size of the list to be sorted (assumed for simplicity to be a power of 2), C is some positive constant, and the value of the work function T(n) is the maximum number of "simple operations" required to mergesort a list of size n or less. We also assume the following base case

(2) T(1) ≤ C

which simply means that the mergesort of a one-element list requires some minimal amount of work, bounded by a constant, C. We can assume that the values of C in (1) and (2) are the same. If they happen to be different, if we have C1 and C2, then choosing C = max{C1, C2} provides a value that makes the inequalities in both (1) and (2) true.

So we have:

(1) T(n) ≤ 2T(n/2) + Cn, for all n such that 2 ≤ n ∈ ℤ
(2) T(1) ≤ C

Expressing n = 2k as a power of 2, we get

(3) T(2k) ≤ 2T(2(k-1)) + C2k, for all k such that 1 ≤ k ∈ ℤ
(4) T(20) ≤ C

How can we get an expression for T in closed form? We can proceed like this:

Using (3): T(2) = T(21) ≤ 2T(20) + C21

Using (4), this gives us T(2) ≤ 2C + 2C = 2C21

In summary, we have
(2) T(20) ≤ C
(5) T(21) ≤ 2C21

Using (3): T(4) = T(22) ≤ 2T(21) + C22
Using (5), this gives us T(4) ≤ 2[2C21] + C22 = 3C22

In summary, we have
(2) T(20) ≤ C
(5) T(21) ≤ 2C21
(6) T(22) ≤ 3C22

Using (3): T(8) = T(23) ≤ 2T(22) + C23
Using (6), this gives us T(8) ≤ 2[3C22] + C23 = 4C23

In summary, we have
(2) T(20) ≤ C (= 1C20)
(5) T(21) ≤ 2C21
(6) T(22) ≤ 3C22
(7) T(23) ≤ 4C23

So there appears to be a pattern that shows up, to the effect that

(7) T(2k) ≤ (k+1)C2k, for all k such that 0 ≤ k ∈ ℤ

Since k=log2(2k), if (7) is true, then T(n) is O(nlog(n)), (for values of n that are powers of 2).

Now that we have found the pattern and formed the theory that (7) is true, can we confirm the theory? Yes, we can prove it, using mathematical induction.

Proposition: Suppose

(3) T(2k) ≤ 2T(2(k-1)) + C2k, for all k such that 1 ≤ k ∈ ℤ
(4) T(20) ≤ C

Then
(8) T(2k) ≤ Bk2k, for some positive constant B, and for all k such that 1 ≤ k ∈ ℤ

Proof:

Base case: for k=1, we have,
by (3) T(21) ≤ 2T(20) + C21,
which by (4) says that T(21) ≤ 2C+C21,
which is equivalent to T(21) ≤ 4C.
On the other hand, for k=1, k2k = 2. So if we set B = 2C, then we have
T(21) ≤ 4C = 2C(2) = 2C*1*21 = Bk2k, for k=1.
So this establishes the base case, that (8) is true, for k=1 and B=2C.

So now assume the inductive hypothesis, that (8) is true for some k=s, where 1 ≤ s ∈ ℤ:
T(2s) ≤ Bs2s

Now, thinking about T(2(s+1)), (3) tells us that

T(2(s+1)) ≤ 2T(2s) + C2s+1
and the inductive hypothesis allows us to conclude that
T(2(s+1)) ≤ 2Bs2s + C2s+1
= B(s+1-1)2(s+1) + C2s+1
= B(s+1)2(s+1) + C2s+1 - B2(s+1)
= B(s+1)2(s+1) + [C-B]2s+1

Note that if we assign the value B=2C, which is what we observed would make the base case true, it will allow us to complete the inductive step of the proof. We will have

T(2(s+1)) ≤ B(s+1)2(s+1) + [C-B]2s+1
= B(s+1)2(s+1) - C2s+1 < B(s+1)2(s+1).

Therefore, the proof is complete, with the value of the constant B chosen to be B=2C. We have proved that Merge Sort is O(nlog(n)), for the case where n is a power of 2.

We have also demonstrated an example of a general technique for determining the big-O of divide-and-conquer algorithms that have work functions T that satisfy recurrences like this:

T(N) ≤ qT(N/b) + CNh, for all suitable integers N ≥ b, where q, b, and h are positive integers and C is a positive constant.