Suppose we want to multiply two 8-bit binary numbers
0 1 0 0 0 0 1 1
x 1 1 0 1 1 1 1 1
-------------------
We can rewrite 0 1 0 0 0 0 1 1 as
0 1 0 0 0 0 1 1 = 24(0 1 0 0) + (0 0 1 1)
= 24x1 + x0
and similarly
1 1 0 1 1 1 1 1 = 24(1 1 0 1) + (1 1 1 1)
= 24y1 + y0
0 1 0 0 0 0 1 1
x 1 1 0 1 1 1 1 1
-------------------
is the same as
[24(0 1 0 0) + (0 0 1 1)][24(1 1 0 1) + (1 1 1 1)]
= [24x1 + x0][24y1 + y0]
= [28][x1][y1]
+ [24][x1y0+y1x0]
+ [x0y0]
= [28][(0 1 0 0)][(1 1 0 1)]
+ [24][(0 1 0 0)(1 1 1 1)+(1 1 0 1)(0 0 1 1)]
+ [(0 0 1 1)(1 1 1 1)]
In other words, a multiplication of two 8-bit numbers can be reduced to four multiplications of numbers half the length (4-bit), plus some additional work that
is proportional to the number of bits - splitting the numbers, doing shifts,
and additions.
This idea works generally for two n-bit numbers, if n is even. In other words,
we could do the same kind of trick to reduce, say, the multiplication of two
256-bit numbers to four multiplications of 128-bit numbers. We can
see the algorithm has a work function that satisfies this relation:
W(n) < 4W(n/2) + Cn, where C is some constant.
(The idea is that we recursively perform the smaller multiplications using the same
divide-and-conquer strategy, until reaching some base-case size that is handled
non-recursively, say by just using a computer hardware multiplication instruction.
Thus the relation above would apply to all problem sizes that are greater than the
base case.)
However, as we will see proved later, this work function W
is Θ(n2), and therefore working the multiplication
problem with this form of divide-and-conquer strategy does not make
things any more efficient than the "traditional" Θ(n2)
multiplication algorithm (in the "Big-O sense").
The "trick" of Karatsuba-Ofman multiplication (1960) is to perform these
three multiplications:
[x1][y1], [x0][y0],
and [(x1+x0)(y1+y0)],
to get the cross term [x1y0+y1x0] by
subtraction as
[(x1+x0)(y1+y0)] - [x1][y1] - [x0][y0]
= [x1y1 + x1y0 + x0y1 + x0y0
- x1y1 - x0y0
= [x1y0 + y1x0],
and then to just do everything else as described before to get the result of the
multiplication as
= [28][x1][y1]
+ [24][x1y0+y1x0]
+ [x0y0]
Using the Karatsuba-Ofman method, we get a work function that satisfies this relation:
W(n) < 3W(n/2) + Kn, where K is some constant.
(Actually, the number of bits in (x1+x0) and/or (y1+y0)
could be as much as (n/2)+1, so the relation above is not quite precise, but a deeper analysis
shows that nevertheless we can use the relation above to find the big-Θ()
of the work function of Karatsuba-Ofman multiplication.