12.5.3 NP-complete problems

We don't know if P = NP.

If we could prove, for one problem Y in NP, that Y cannot be solved with a polynomial time algorithm, then that would prove that P != NP.

Perhaps surprisingly, there are problems X in NP, such that to prove X has a polynomial time solution is to prove P = NP. In other words if we prove X has a polynomial time solution, that proves every NP problem has a polynomial time solution!

DEFINITION: A decision problem X is NP-complete if
  1. X is in NP, and
  2. [Y] <=PT [X] for every problem Y in NP
If there is one NP-complete problem that can be solved in polynomial time, then it follows that all problems in NP can be solved in polynomial time, and therefore that P=NP.

In some sense this "simplifies" things for researchers in theoretical computer science. To prove that P=NP, all you have to do is find a polynomial time algorithm for "one little problem".

THEOREM: Let X be an NP-complete problem. Consider a decision problem Z in NP such that [X] <=PT [Z]. Then Z is also NP-complete.

PROOF: This follows directly from the fact that <=PT is a transitive relation. QED

The theorem above gives us a way to find more NP-complete problems once we have found one. Quite a lot of the work done in the theory of algorithms has been of this nature. This has resulted in the fact that there is a very large set of problems known to be NP-complete.

If you are interested in creating a program to solve a problem it is useful to know whether the problem is NP-complete. If so, then you aren't very likely to be able to find a polynomial time algorithm that solves the problem. You would want to know that ahead of time, so you won't waste your time looking for a solution you (amost certainly) cannot hope to find.

If you did find a polynomial time algorithm for an NP-complete problem, then you would also have proved, in effect, that P=NP. Computer scientists have tried and failed for many years to find out whether P=NP. We still do not know if P=NP. Most researchers doubt that anyone ever will prove P=NP. Most think P != NP (but they don't have proof).

Thus if you are just trying to write a program that solves an NP-complete problem (and not trying to solve the biggest mystery in computer science) you should not try to find a polynomial time algorithm.

There is one big theorem we need now. It will provide us with our first, and thus most important, example of an NP-complete problem. The theorem is called Cook's Theorem, and it declares that a problem called "satisfiabilty" is NP-complete. We need to define a few things to make the theorem intelligible.

DEFINITION: A Boolean formula (i.e. a Boolean expression like we see in programs involving Boolean variables) is satisfiable if there exists at least one way of assigning values to its variables so as to make it true. We denote by SAT the problem of deciding, given a Boolean formula, whether or not it is satisfiable.

EXAMPLE OF A SATISFIABLE BOOLEAN FORMULA:

PROPOSITION: (p || q) ==> (p && q) is satisfiable.

PROOF: The statment A ==> B means "It is not the case that both !B and A are true." Other ways of saying the same thing are:
  1. !(!B && A)
  2. (B || !A)
Therefore (p || q) ==> (p && q) is equivalent to (p && q) || !(p || q), which is equivalent to (p && q) || (!p && !q). This is satisifed (i.e. it's value is true) if both p and q are false or if both are true. QED

EXAMPLE OF A NON-SATISFIABLE BOOLEAN FORMULA:

PROPOSITION: !p && (p || q) && !q is not satisfiable.

PROOF: It's clear that the formula requires that (p || q) be true, which means that at least one of p or q must be true. On the other hand the two 'ends' of the formula also require that neither p nor q be true. QED.

The size of a Boolean formula can be measured in terms of the number of variables N, and the maximum number of occurrences of each variable in the formula. The "obvious" algorithm for testing a formula for satisfiability would be to generate the possible combinations of the truth values of the variables one at a time, and to repeatedly check the formula to see if the current combination satisfies the formula (i.e. makes it have the value true). This could require checking 2N cases.

DEFINITION: A literal is either a Boolean variable x or its logical negation !x. A clause is a literal or a disjunction (OR-ing) of literals. A Boolean formula is in conjunctive normal form (CNF) if it is a clause or a conjunction (AND-ing) of clauses. It is in k-CNF for some positive integer k if it is a conjunction of clauses, each of which contains at most k literals.

EXAMPLE: (p || q || !s) && (!p || !q) is in CNF. It is also in 3-CNF.

DEFINITION: SAT-CNF is the restriction of SAT to Boolean formulas in CNF. For any positive k, SAT-k-CNF is the restriction of SAT-CNF to Boolean formulas in k-CNF.

It is clear that SAT, SAT-CNF and SAT-k-CNF are all in NP. It's easy to figure out how to check a proposed solution in linear time.

A Boolean formula can simulate an algorithm (think about groups of bits in RAM storage being AND-ed and OR-ed by the CPU. Also, think about the way that Prolog programs work.).

It is beyond the scope of the text to explicitly prove most of the details of Cook's theorem, but we give a sketch of the proof here. This should help you appreciate the significance of the theorem.

The gist of a large section of the proof is:

Given a polynomial-time algorithm Al that solves a decision problem X, and given the size in bits N of an instance x of X, it is possible to efficiently construct a Boolean formula ψ(N, Al). This formula contains many variables, among which there are variables x1, x2, ..., xN which correspond directly to the N bits of the input x. After setting the values of x1, x2, ..., xN in ψ(N, Al) according to the bit values of x, the resulting formula is satisfiable if and only if algorithm Al accepts instance x. (The formula is satisfiable if and only if x ε X.)

This Boolean formula is a function only of N and Al. There is a polynomial time algorithm Z for constructing such Boolean formulas. Z can input (N, Al), and output the Boolean formula, ψ(N, Al). (What Z does is a lot like what a compiler does.)

As stated above, ψ(N, Al) contains the variables x1, x2, ..., xN that correspond to the N bits of an input x of size N for algorithm Al. If we have a specific input x, we can set each xi to true or false, depending on whether the corresponding bit of x is 1 or 0. The result is another Boolean formula (called a specialization of ψ(N, Al)). That Boolean formula is satisfiable if and only if x is a yes-instance of algorithm Al.

It's beyond the scope of the course to give much detail about the structure of the formula ψ(N, Al). However, consider this: Let I be the set of all bits of storage that might be used by algorithm Al, and let T be the set of all units of time that Al might use to solve an instance of size N. The formula ψ(N, Al) contains a distinct variable Bi,t for each element of the cross product (I x T) (i.e. for each possible ordered pair (i,t)).

Now suppose that we have an NP problem Y. This means that there is a "proof" (also called a certificate) q(y) of each y in Y. The size of q(y) is big-O of some polynomial in the size of y. There is also an algorithm Al* that can, given y and q(y) as input, verify that y is in Y by outputting "yes" in polynomial time. Also, Al* outputs "no" if q(y) is not a proof that y is in Y. Without loss of generality, we can assume that whenever the size of y is M, the size of q(y) is p(M), where p is some suitable polynomial.

Now to this set of facts we add the knowledge that Al* and N=M+p(M) can be transformed efficiently into the corresponding CNF formula, ψ(N, Al*).

Since the inputs to this algorithm Al* are, in effect, ordered pairs of the form x=(y,q(y)), ψ(N, Al*) will have variables corresponding to the bits of y and the bits of q(y). As we discussed above, if we "plug in" the bits of x=(y,q(y)), then the resulting specialization of ψ(N, Al*) is a Boolean formula that is satisfiable if and only if y is in Y and q(y) is a certificate of that fact.

Now consider the following "wrinkle": If we plug in only the bits of y, and not the bits of q(y) then the resulting (partial) specialization of ψ(N, Al*) can be used to simulate an algorithm that determines if y is in Y using only y as its input.

This is so because this formula is satisfiable if and only if there exists a proof q(y) (expressed as a sequence of truth-values to be substituted in for the rest of the xi's) such that algorithm Al* accepts input x=(y,q(y)).

Once we have this partial specialization of ψ(N, Al*), we can determine, without being given a proof q(y) ahead of time, whether y is in Y. How? We construct the formula. If we determine that the formula is satisfiable, this knowledge tells us that there must be a valid proof, and therefore y is in Y. On the other hand, if the formula is found to be unsatisfiable, then that knowledge tells us that there is no valid proof, and therefore y is not in Y.

That is, the formula described is satisfiable if and only if y is in Y. In effect, it simulates an algorithm that solves problem Y.

The details of the construction are such that the resulting algorithm for deciding Y will run in polynomial time if the algorithm for determining the satisfiability of the Boolean formula runs in polynomial time.

This completes our outline of the proof that any NP problem Y can be polynomially reduced to solving an instance of SAT-CNF.

The statement above is the content of Cook's Theorem:

COOK'S THEOREM:

SAT-CNF is NP-complete.

COROLLARY to Cook's Theorem:

If SAT-CNF can be done with a polynomial time algorithm, then