Greedy Algorithms

• The idea of a greedy algorithm is that it builds up a solution piece-by-piece, choosing each new piece according to some seemingly shortsighted rule-of-thumb. Once the algorithm makes a choice, it never has to undo the choice. In the end the solution constructed has some global optimality property.
  
  
• For example, we have a familiar greedy algorithm for making change. Suppose you are a cashier with a very large supply of quarters, dimes, nickels and pennies. If you owe the customer an amount X, you choose a coin with the largest value possible (except the value of the coin must not be greater than X). You subtract the value of the coin from X. This gives you a value X₂. If X₂ is not zero, you repeat. Here the rule of thumb is to use the largest coin possible at each step. The overall optimal result is that you make change for the customer and the number of coins you give him is as small as it can possibly be.
  
  
• For example, to make change for 57¢ you choose a quarter, another quarter, then a nickel, then a penny, and another penny. The total number of coins is five. If you have only quarters, dimes, nickels, and pennies, there is no way to make 57¢ in coin with fewer than five coins.
  
  
• It's fairly common to have a situation where it's pretty easy to understand how to perform the actions of a greedy algorithm, but not obvious that the algorithm actually works. (For example, how do we know for sure that the algorithm for making change described above always gives the change using the smallest possible number of coins? How could we prove that?
  
  
• The chapter discusses a couple of general approaches to proving the correctness of greedy algorithms. One approach is known as establishing that the algorithm stays ahead and the other is the use of exchange arguments. The proof we saw in chapter one, that shows that the loop in the G-S algorithm terminates was a kind of 'stays ahead' argument.
  
  
• 4.1 Interval Scheduling
  
  
  • When trying to figure out a greedy algorithm for the interval scheduling problem of section 4.1 of our text, it's a challenge to figure out what might be a good rule-of-thumb (heuristic) for choosing each interval to be added to the set.
    
    
  • Various things seem like they might be plausible. "Available compatible interval with the earliest start time" seems like it might be good until you realize it could lead you to pick just one long interval, thus preventing you from choosing several shorter ones. Obviously this is 'bad' because the goal is to get as many non-overlapping intervals as possible.
    
    
  • Another idea is to accept the shortest remaining compatible interval. A problem with that is one short interval could overlap two longer ones.
    
    
  • Another idea: select the interval with the 'fewest conflicts' - it overlaps with the smallest possible number of other intervals. That seems fairly good but there's an example in the book that shows it can lead to a sub-optimal result. (figure 4.1c)
    
    
  • Another rule that seems reasonable is "accept next the compatible interval that finishes earliest." Perhaps that seems like a plausible strategy, because you are tending to make the resource free again as soon as possible, so that you can add another interval to its schedule sooner. In fact that particular rule-of-thumb does 'work' - it always gives an optimal result - you get the greatest possible number of compatible intervals.
    
    
  • Sketch of proof that the "earliest finish time" (EFT) algorithm works: Given a non-empty set of requested intervals, our task is to choose a set of non-overlapping intervals of maximum size. Let i₁, i₂, ... , i_k be the intervals chosen by the (EFT) algorithm and let j₁, j₂, ... , j_m be an optimal solution. Note that this implies that k<=m. Assume both lists are ordered by increasing finish time. Clearly i₁ has a finish time no later than the finish time of j₁. We know this is true because the EFT algorithm begins by choosing i₁ to be a request with the earliest finish time. (If there is more than one request having the earliest finish time, then the EFT algorithm chooses i₁ to be one of them.)

    
    --- i1 ---  --- i2 --- ... --- ir-1 ---    --- ir ---  ... -- ik ---
    
    ---- j1 ---    --- j2 --- ...  --- jr-1 --- --- jr ---  ...   ---jk --- ...
    

    Suppose that you know that the first r-1 of the i_t finish, respectively, no later than the first r-1 of the j_t. Suppose there is an i_r and a j_r. Since i_r-1 finishes no later than j_r-1, and j_r is compatible with j_r-1; j_r is compatible with i_r-1 too. Therefore, j_r is among the candidates considered when the EFT algorithm chooses i_r. This shows that i_r must have a finish time no later than j_r. The foregoing considerations amount to a proof by induction that each i_t has a finish time no later than the corresponding j_t for t = 1, 2, ..., k.
    
    The EFT algorithm stopped after choosing i_k. That must have been because there are no intervals that start after i_k finishes.
    
    Therefore there are no intervals that start after j_k finishes. This shows that m, the size of an optimal solution, is the same as k. In other words, we have proved that the EFT algorithm finds a set of non-overlapping intervals of the largest possible size.
    
    
  • We've shown that the algorithm will always work. How efficient can we make it? We can sort N intervals of the form (s_i, f_i) by increasing finish time f_i using an O(Nlog(N)) algorithm. Using an additional O(N) amount of time we can select an optimal set while making a single pass through the list:
    
    (s₁, f₁), (s₂, f₂), (s₃, f₃), ... , (s_N, f_N)
    
    We choose the first interval (s₁, f₁) and start scanning the rest of the list. While scanning, we look for the first interval whose start time s_i is not before the finish time f_i-1) of the previous interval chosen.