Chapter Eight -- Deadlocks -- Lecture Notes

• 8.0 Objectives
  
  
  • Illustrate how deadlock can occur when mutex locks are used.
  • Define the four necessary conditions that characterize deadlock.
  • Identify a deadlock situation in a resource allocation graph.
  • Evaluate the four different approaches for preventing deadlocks.
  • Apply the banker's algorithm for deadlock avoidance.
  • Apply the deadlock detection algorithm.
  • Evaluate approaches for recovering from deadlock.
  
  
• 8.1 System Model
  
  
  • Our model system contains threads and resources.
    
    
  • The set of all resources is partitioned into equivalence classes called resource types. For all intents and purposes, any two elements (instances) of a resource type are identical.
    
    
  • Examples of resources: units of CPU time, printers, drives, units of memory, files, and semaphores.
    
    
  • A running thread may request resources at any time.
    
    
  • A thread uses resources according to this pattern:
    
    
    1. Request. The requesting thread must wait if the resource cannot be granted immediately.
    2. Use. For example, if the resource is a semaphore, the thread can enter its critical section.
    3. Release.
    
    Examples of request and release operations that are system calls are request() and release() of a device; open() and close() of a file; and allocate() and free() of memory. Other examples are wait() and signal() on a semaphore; and acquire() and release() of a mutex lock.
    
    
  • A thread may request multiple resources at the same time. However the request may not exceed the number existing in the system. For example if there are two tape drives and four DVD's in the system, the thread may request up to (but no more than) two tape drives and four DVD's with one request.
    
    
  • DEFINITION: Deadlock occurs in a group of threads when each thread in the group is waiting for an event that can only be caused by another thread in the group.
    
    When threads and/or processes operate according to the system model described here, deadlock can happen.
    
    
  • A deadlock is stable. Once deadlock occurs, none of the deadlocked threads are capable of doing anything to solve the problem. Each just keeps on waiting for some other thread or threads to do something.
    
    
  • Suppose that cars arrive at a four-way stop from all four directions simultaneously. The law says that the car on the right has priority. So each driver should wait for the driver on the right to go first. That illustrates the idea of a deadlock.
  
  
• 8.2 Deadlock in Multithreaded Applications

  

  • As a simple example of how a deadlock can happen, imagine two threads T₁ and T₂, sharing two POSIX mutex locks, first_mutex and second_mutex. Supposed the following things happen in the order given:

      /* first_mutex and second_mutex 
         are created and initialized. */
    T1 waits on first_mutex;   /* T1 holds first_mutex */
    T2 waits on second_mutex;  /* T2 holds second_mutex */
    T1 waits on second_mutex;  /* T1 has blocked */
    T2 waits on first_mutex;   /* T2 has blocked */

    Now both threads are blocked forever - deadlocked.

    

    

    Notice that the deadlock described above happens only if the two threads interleave their actions in a specific way. For example, if T₂ first executes all of the code in function do_work_two(), and then T₁ executes all of the code in function do_work_one(), there is no adverse result. In that case, first T₂ acquires both locks, then does some work, then releases both locks; and after that T₁ acquires both locks, does some work, and releases both locks.
    
    
  • 8.2.1 Livelock
    
    The text depicts a livelock situation similar to the example above of deadlock. In the case of the livelock example, the two threads are not blocked, but they continually attempt to acquire the same two locks, fail, and retry in a manner that can continue indefinitely. (See the example below.)

    
           /* thread_one runs in this function */
    void *do_work_one(void *param)
    {  int done = 0;
       while (!done) 
       {  pthread_mutex_lock(&first_mutex);
          if (pthread_mutex_trylock(&second_mutex)) 
          {
                 /*  Do some work  */
    
             pthread_mutex_unlock(&second_mutex);
             pthread_mutex_unlock(&first_mutex);
             done = 1;
          }
          else pthread_mutex_unlock(&first_mutex);
       }
       pthread_exit(0);
    }
    
           /* thread_two runs in this function */
    void *do_work_two(void *param)
    {  int done = 0;
       while (!done) 
       { pthread_mutex_lock(&second_mutex);
         if (pthread_mutex_trylock(&first_mutex)) 
         {
                 /*  Do some work  */
    
             pthread_mutex_unlock(&first_mutex);
             pthread_mutex_unlock(&second_mutex);
             done = 1;
         }
         else pthread_mutex_unlock(&second_mutex);
       }
       pthread_exit(0);
    }

• 8.3 Deadlock Characterization
  
  
  • 8.3.1 Necessary Conditions
    
    These are necessary conditions. Importantly, that means we can prevent deadlock by eliminating any one (or more) of these conditions.
    
    If there is a deadlock, the following four conditions must exist.
    
    
    1. Mutual exclusion. For at least one resource, only one thread at a time can hold that resource.
       
       
    2. Hold and wait. There must be a thread T₀ that is holding a resource R₀, and waiting to acquire a resource R₁ that is currently held by a thread T₁ .
       
       
    3. No preemption. If a thread does not release a resource voluntarily, nothing can take it away.
       
       
    4. Circular wait. There must be a set of one or more threads {T₀, T₁, ..., T_n} such that T_j is waiting for a resource held by T_j+1 for 0 ≤ j < n, and T_n is waiting for a resource held by T₀.
       
       
  • 8.3.2 Resource-Allocation Graph
    
    It is possible to more accurately describe system characteristics related to deadlock by using a system resource allocation graph (RAG). We explain the structure of a RAG below.
    
    
    • Threads are represented as Nodes -- Circles
      
      
    • Resource Types are represented as Nodes -- Squares
      
      
      • Each instance of a resource type is represented by a dot in the square.
        
        
    • A solid request-edge points from a thread to a resource type
      
      
    • When a request is granted the request edge is instantaneously transformed into an assignment edge extending from one of the instances inside the resource to the thread.

      

      

      

    • The assignment edge disappears when the thread releases the resource.
      
      
    • The request-use-release sequence is reflected in the foregoing description of transitions that occur in the resource allocation graph.
      
      
    • The "if no cycles then no deadlock" rule: If there are no cycles in the RAG of a system, then there is no deadlock in the system.
      
      
    • The CONVERSE of the "if no cycles then no deadlock" rule is FALSE, because some systems can get into a state where there IS a cycle in the RAG, but the system is NOT DEADLOCKED. The text has a diagram that depicts such a situation. It is example 03 above.
      
      
    • However, there is a PARTIAL CONVERSE to the "if no cycles then no deadlock" rule: If there is a cycle in the RAG, AND if every resource type on the cycle has only one instance, then the threads on the cycle are deadlocked.
      
      
    • If any of the resource types on the cycle have more than one instance, it is possible for a cycle to exist in the RAG when there is no deadlock.
      
      
    • On the other hand, it is possible for deadlock to occur in systems that have more than one instance of some resource types! The text has a diagram that depicts such a deadlock situation. It is example 02 above.
  
  
• 8.4 Methods for Handling Deadlock
  
  
  • This section gives capsule descriptions of various methods. The text presents more details in Sections 8.5 through 8.8.
    
    
  • The Three Main Categories
    
    
    1. Use a Protocol to Prevent or Avoid. Do something to assure that deadlock cannot happen.
       
       
       • The idea of deadlock prevention is to make rules about how requests and assignments are done so that one or more of the necessary conditions for deadlock is/are always missing. (People who always follow certain rules to avoid problems are rather like a system performing deadlock prevention.)
         
         
       • Deadlock avoidance works by restricting threads only when the system is about to enter an "unsafe state," from which it could immediately "go out of control" and become deadlocked. The OS can't reasonably perform an avoidance strategy unless it receives special advance information about the resources requests and/or releases each thread may perform during its lifetime. (People who do as they please, only maneuvering to avoid nearby danger, are rather like a system performing deadlock avoidance.)
       
       
    2. Perform Detection and Recovery. Allow deadlock, but fix things afterwards. Utilize two kinds of algorithms:
       
       
       • an algorithm for detecting when deadlock has occurred, and
         
         
       • an algorithm for recovering from deadlock.
       
       
    3. Ignore. Basically, do nothing.
       
       
       • This is the most-used method, because there are significant performance and cost penalties for utilizing prevention, avoidance, or detection & recovery.
         
         
       • For a system in which deadlocks are rare, administrators may find it feasible just to kill and restart "frozen" processes, or to reboot a sluggishly-performing machine.
         
         
       • Even if the OS ignores the problem, programmers can write applications in ways that handle potential deadlocks among the threads of the applications.
    
    
  • Hybrid combinations of methods 1-2 are possible. Some researchers feel there's no single method that works well enough to provide a complete solution.
  
  
• 8.5 Deadlock Prevention
  
  
  • 8.5.1 Mutual Exclusion
    
    If X is a resource and the OS immediately grants every request for access to X, then X can never be part of a circular wait - it can never be involved in a deadlock. For example, it is OK for any number of threads to share a read-only file. Obviously sometimes exclusive access to resources is required. So we can't prevent deadlock simply by deciding to make all resources sharable all the time.
    
    
  • 8.5.2 Hold and Wait
    
    
    • Method: Require each thread to request and be allocated all its resources before it begins execution, or
    • Method: allow a thread to request resources only when it has none.
    • Disadvantage of these methods: resources may be allocated but unused for long periods of time, and/or time may be wasted releasing resources, and then requesting them again right away.
    • Disadvantage of these methods: If a thread waits for more than one resource at a time, there is no guarantee they will all become available at the same time, and therefore starvation can occur. (Remember, in such cases the OS does not have the option of giving the resources to the thread one at a time, because that can cause deadlock.)
    
    
  • 8.5.3 No Preemption
    
    
    • Method: If a thread T requests a resource that is not immediately available, the OS takes away all the resources T is holding. T must then wait for the new resource, plus all the resources that were taken away.
    • Method: Suppose a thread T requests some resources that are held by a thread Q. If Q is waiting for a resource then T takes what it wants from Q and this is added to the request for which Q is waiting. (If no thread waits on a waiting thread then there are no cycles.) If Q is not waiting, then P waits. [The case P=Q may need special treatment.]
    • Advantage of these methods: The OS can harmlessly preempt certain things whose 'state' can be quickly saved and restored, like registers and units of memory.
    • Disadvantage of these methods: Deadlocks commonly involve resources like semaphores and mutex locks. It is not practical for the OS to preempt those kinds of resources.
    • Disadvantage of these methods: Time lost waiting for preempted resources, including the possibility of indefinite postponement when waiting for multiple resources.
    
    
  • 8.5.4 Circular Wait
    
    
    • Method: Impose a total ordering on resource types and forbid requests that go against the order. (Also if a thread needs more than one instance of a resource type, it must make a single request for all of them.) These rules assure that, in chains of waiting threads, the resource numbers are strictly increasing, so no cycles can occur.
    • Advantage of this method: It is more easily implemented than other methods, and therefore it is more practical.
    • Disadvantage of this method: It may lead to longer periods of holding some resources, and thus to decreased availability of resources. Also, it allows starvation to occur in cases where there is more than one instance of some resource types.
  
  
• 8.6 Deadlock Avoidance
  
  The deadlock avoidance schemes presented in the text require the OS to have advance knowledge, for each thread, of the maximum number of resources of each type that the thread may need.
  
  
  • 8.6.1 Safe State
    
    
    • The two avoidance algorithms presented in the text both require the OS to recognize unsafe states - states from which the system could slip, out of control, into deadlock.
      
      
    • Basically an unsafe state is one which will turn into a deadlock if all threads immediately request their remaining possible resource needs.
      
      
    • The state is safe if it is not unsafe. If the system state is safe, then even if all threads max out their resource requests, they will nonetheless be able to finish executing and exit in some order T₀, T₁, T₂, ... , T_n. When each thread exits, it gives up its resources. Freed resources become available to the next thread in the sequence.
      
      
    • If the state of the system is safe, it is not deadlocked. If the state of the system is unsafe, it could also be deadlocked or it might not be deadlocked. These ideas are illustrated by Figure 8.8 below.

      

    • If the state of the system is safe, it is possible for the OS to keep it safe, and therefore it is possible for the OS to avoid deadlock. If the state of the system becomes unsafe, the OS immediately loses the ability to avoid deadlock. Threads can throw an unsafe system into deadlock just by requesting resources.
      
      
    • Examples of safe and unsafe systems:

      thread     max needs  cur alloc  needs left   free      
         T0         10          5           5         3
         T1          4          2           2
         T2          9          2           7
      

      There's one resource type (tape drives, say) with twelve instances, allocated as shown. The threads have the indicated max possible needs and remaining needs. The system shown above is safe. {T₁,T₀, T₂} is a safe sequence - the threads could finish executing in that order, even if all the threads first request all of their remaining needs. However, suppose that, starting from the situation depicted in the table above, T₂ requests and is given one more tape drive. Then the state changes to this one:

      thread     max needs  cur alloc  needs left   free      
         T0         10          5           5         2
         T1          4          2           2
         T2          9          3           6
      

      This system is unsafe. If all the threads request their remaining needs, the system will be deadlocked. It will be possible for the OS to give T₁ its remaining needs, and possible for T₁ to finish and exit. However, after that there will be only 4 tape drives free, and so both T₀ and T₂ are set up to wait forever.
      
      
    • If, instead of giving the tape drive to T₂, the OS had made T₂ wait until after T₁ or T₀ released their tape drives, the OS could have avoided deadlock. That's the idea of how the avoidance algorithms work.
      
      
    • When a system practices deadlock avoidance it uses the following criteria to decide whether to grant resources to a requesting thread T. The request is granted if:
      
      
      1. T is not asking to exceed its declared maximum possible needs,
      2. the resources are currently available (free), and
      3. granting the request will leave the system in a safe state.
      
      Typically if T tries to exceed its max, the OS will terminate T. If condition #1 is true but #2 or #3 fails, the system makes T wait for the resources. (They will be granted later - when available and 'safe.')
      
      
    • PROBLEMS WITH THAT:
      • Each time a thread makes a request for an available resource, if the request does not exceed the 'need' of the thread, the operating system is required to do another safety check, involving potentially examining every thread and resource type. This is extra work that will tend to reduce useful throughput.
      • Threads will sometimes have to wait for resources even though the resources are available. This will reduce resource utilization and throughput.
      • There is also the possibility of starvation.
    
    
  • 8.6.2 Resource-Allocation-Graph Algorithm
    
    
    • We can create an augmented resource allocation graph (augRAG) by adding (dotted) claim edges from threads to resources, representing each request that each thread might make.
      
      
    • If there is just one instance of each resource type, then "unsafe" is equivalent to "cycle in the augRAG." This is a conceptually simple way to characterize safety. Cycle detection algorithms typically require O(N²) work, where N is the number of threads in the system. This method does not work when there are multiple instances of some resource types, because in that case there can be a cycle in the augRAG of a safe system.
      
      
    • Example: See text figures 8.9 and 8.10 (below). If T₂ requests R₂, the avoidance algorithm would make T₂ wait, even though R₂ is available. If the OS assigns R₂ to T₂, then, as figure 8.10 shows, the resulting state would be unsafe. From the state shown in figure 8.10, if T₁ next requests R₂, the system would be deadlocked.

      

      

  • 8.6.3 Banker's Algorithm
    
    
    • The Banker's Algorithm is a deadlock avoidance scheme that works when there are multiple instances of resource types. It is generally less efficient than the cycle-detection scheme.
      
      
    • Before it does anything else a new process must declare the maximum number of instances of each resource type that it may need.
      
      
    • Various data structures are required. See the GLOSSARY here. Let n = the number of threads, and m = the number of resource types.
      
      
      • Available is a vector of length m. When Available[j] equals k, it means that there are k available (free) instances of resource type R_j.
        
        
      • Max is an nXm matrix. When Max[i,j] equals k, it means that thread T_i is allowed to request up to k instances of resource type R_j.
        
        
      • Allocation is also an nXm matrix. When Allocation[i,j] equals k, it means that thread T_i currently holds k instances of resource type R_j.
        
        
      • Need is another nXm matrix. When Need[i,j] equals k, it means that thread T_i is allowed to request at most k additional instances of resource type R_j. A request of more than k would mean T_i exceeded its max request. Need is just the difference (Max-Allocation) between the Max and Allocation matrices.
      
      
    • 8.6.3.1 Safety Algorithm
      
      
      1. Let Work and Finish be vectors of length m and n, respectively. Initialize Work = Available and Finish[i] = false for i = 0, 1, ..., n-1.
         
         
      2. Find an index i such that both
         
         
         1. Finish[i] == false
         2. Need_i ≤ Work
         
         If no such i exists, go to step 4.
         
         
      3. Work = Work + Allocation_i
         Finish[i] = true
         Go to step 2.
         
         
      4. If Finish[i] == true for all i,
         then the system is in a safe state
         else the system is in an unsafe state
         
         
      • There is another write-up of the safety algorithm in the notes here.
        
        
      • The safety check algorithm may require Θ(mn²) work.
      
      
    • 8.6.3.2 Resource-Request Algorithm
      
      The Resource-Request Algorithm is the main part of the Banker's Algorithm. It uses the Safety Algorithm as a sub-program (function call). When a thread T_i makes a request for resource(s), the request is formally designated as a vector of length m, Request_i, where m is the number of resource types. When Request_i[j] equals k, it means that the resource request from thread T_i includes a request for k instances of resource type R_j. The steps of the Resource-Request Algorithm are as follows.
      
      
      1. If Request_i ≤ Need_i, go to step 2. Otherwise, raise error: max request exceeded.
         
         
      2. If Request_i ≤ Available_i, go to step 3. Otherwise T_i must wait, because the resources are not all available.
         
         
      3. Have the system pretend to have allocated the requested resources to thread T_i by modifying the state as follows:
         
         Available = Available - Request_i
         Allocation_i = Allocation_i + Request_i
         Need_i = Need_i - Request_i
         
         
      4. Run the safety algorithm on the resulting resource-allocation state. If that state is safe, then go ahead and give Request_i to T_i. If the new state is not safe, then revert to the former state, and make T_i wait for Request_i.
      
      
    • 8.6.3.3 An Illustrative Example
      
      
      • See the textbook example worked out in complete detail.
  
  
• 8.7 Deadlock Detection
  
  Another alternative: Have the system run a deadlock detection algorithm and have the system run a recovery algorithm after it detects a deadlock.
  
  Costs of such schemes include maintaining extensive information sets, computing time to execute the algorithms, and losses incurred when deadlocked processes have to be terminated or rolled back.
  
  
  • 8.7.1 Single Instance of Each Resource Type
    
    
    • In this case there is a deadlock if and only if there is a cycle in the resource allocation graph. Therefore the OS can detect deadlock by maintaining a RAG that represents the system and doing cycle checks from time to time. There is a more compact graphical representation called a wait-for graph that can be used instead. The algorithm will tend to be more efficient if run on this graph.
      
      

      

  • 8.7.2 Several Instances of A Resource Type
    
    
    • If there are multiple instances of some resource types then one can detect deadlock with an algorithm similar to the safety algorithm. (We can view the safety algorithm as checking to see if there would be a deadlock if all the processes were to max out their requests.) The data structures are as follows.
      
      
      • Available is the same as in the Banker's Algorithm.
        
        
      • Allocation is the same as in the Banker's Algorithm.
        
        
      • Request is an nXm matrix. When Request[i,j] equals k, it means that thread T_i requests k instances of resource type R_j.
        
        
    • The deadlock detection algorithm goes like this:
      
      
      1. Let Work and Finish be vectors of length m and n, respectively. Initialize Work = Available. For i = 0, 1, ..., n-1, if Allocation_i ≠ 0, then Finish[i] = false. Otherwise Finish[i] = true. (Setting some Finish flags to true is just an optimization. We could also set them all to false, and the outcome of the algorithm would not be any different.)
         
         
      2. Find an index i such that both
         
         
         1. Finish[i] == false
         2. Request_i ≤ Work
         
         If no such i exists, go to step 4.
         
         
      3. Work = Work + Allocation_i
         Finish[i] = true
         Go to step 2.
         
         
      4. If Finish[i] == true for all i, 0≤i<n,
         then the system is not in a deadlocked state.
         If Finish[i] == false for some i, 0≤i<n,
         then the system is in a deadlocked state.
         Moreover, if Finish[i] == false, then thread T_i is deadlocked.
         
         
      • There is another write-up of the deadlock detection algorithm in the notes here.
        
        
      • This deadlock detection algorithm may require Θ(mn²) work.
    
    
  • 8.7.3 Detection Algorithm Usage
    
    
    • In the system model we are using, when a thread is given resources, it always receives all the resources for which it asked. Therefore, after an assignment edge R--->T is added to a RAG, there are never any edges in the RAG that leave from T. Therefore, the OS never creates a cycle when it adds assignment edges to a RAG .
      
      From the reasoning above, we can conclude that the addition of a request edge to a RAG is always the last step in the creation of a deadlock. If we check for deadlock every time a process blocks requesting a resource, then we will detect each deadlock as soon as it happens. However, if we use our textbook's deadlock detection algorithm, this would require a lot of processing overhead.
      
      
    • It may suffice to check for deadlock only about as often as deadlock occurs. We also might make use of heuristics, such as low CPU utilization or the presence of processes in the system that have been waiting for resources for an unusually long time.
  
  
• 8.8 Recovery from Deadlock
  
  One possibility is for the system to inform the operator (system administrator) that there is a deadlock, and leave it to the operator to decide what to do.
  
  There is also the possibility of programming automatic recovery methods. The usual two options are to abort some or all of the deadlocked threads, or to preempt resources from one or more deadlocked threads.
  
  
  • 8.8.1 Process and Thread Termination
    
    
    • Here are some details regarding the two strategies of recovering from deadlock by terminating processes and/or threads.
      
      
      • Abort all deadlocked processes. To break a deadlock the OS can just abort all the deadlocked processes, and reclaim all their resources. This surely ends the deadlock. However, much of their unfinished work will go to waste.
        
        
      • Abort one process at a time until deadlock is eliminated. Instead we can select and abort victim processes one at a time, stopping when there are no more deadlocked processes. This raises the question of what criteria to use to select victims. Also, there is the problem of determining when there is no deadlock. There will be the expense of testing for deadlock after each process is aborted.
        
        
      • Disadvantage of aborting processes: Deadlocked processes can be in the midst of changing data. Aborting those processes can cause data to be left in incorrect states.
      • If we abort one process at a time, on what basis should we select the next victim? Ideally, we should minimize cost, but how?
        
        
        • Terminate the lowest priority process?
        • Terminate the youngest process?
        • Terminate the process that has most "life" ahead of it?
        • Terminate the process with the fewest stateful resources?
        • Terminate the process that needs the most additional resources?
        • Terminate the smallest possible number of processes?
    
    
  • 8.8.2 Resource Preemption
    
    Instead of killing deadlocked processes, consider taking resources from some and giving them to others until the deadlock is broken. Here are three issues that must be addressed.
    
    
    1. Selecting a victim. Which resources should we take from which processes? Preempt from processes that have a lot of resources? Preempt processes that have not been running long?
       
       
    2. Rollback. If we don't abort a victim process we will probably need to roll it back and restart it at a point in its execution before it acquired the resources that have been preempted. Problem: How? What kind of information is needed? The easiest thing might be just to roll the process back to its beginning - in effect killing and restarting it.
       
       
    3. Starvation. Starvation is a possibility when we abort or rollback processes and restart them. There is no guarantee that they will not become deadlocked again, and be aborted or rolled back again. We may want to build in a mechanism to "spare" former victims.