Lecture #15 (12 March 2002)

Algorithms: Iteration and Recursion
(Case Study: Searching)

Overall Reading
Brookshear: Review Ch. 4.1-4.2
Read pp. 184-190, Ch. 4.5, parts of pp. 206-211, 248-251, 333.

Outline:

  • Overview
  • A Case Study: Searching a sorted list

  • Iterative Structures ("loops") (pp. 184-190 [Br])
  • Sequential Search

  • Divide and Conquer (recursion) (Ch. 4.5 [Br])
  • Binary Search

  • Efficiency (parts of pp. 206-211 [Br])

  • Implementation (pp. 248-251, 333 [Br])

    • Overview

  • Iterative algorithms
    One way to develop an algorithmic solution to a problem is to recognize a repetitive process which can be used, and to then formulate an algorithm based on using 'loops.'

  • Divide-and-Conquer algorithms
    Other times, we can develop an algorithm for a problem which is based on breaking the problem into one or more subproblems. If we are able to solve those subproblems, we might be able to combine those solutions to solve the original problem.

    How do we solve the subproblems? Use the same algorithm recursively (until you get down to some sufficiently small base case).

    Example: When parsing the arithmetic expression 

      ((Y+4)*8/(5-(X+4)))
    We can imagine that part of the algorithm determined that the division in this example, was the top-level operator for the expression. After making this determination, the rest of the parsing was probably carried out by repeating the same algorithm on the two separate expressions: 
      (Y+4)*8
  (5-(X+4))

  • A Case Study: Searching a sorted list
    Assume that we have a guest list for some event, with names appearing in memory in alphabetical order. How could we describe an algorithm for searching the guest list for a particular person? (Let's call this particular person the "target")

    • Iterative Structures (i.e., "loops") (Ch. 4.4 [Br])

    We have already seen the use of loops as a common programming technique.

    Though the syntax may differ from language to language, the three basic elements of repetitive control include:

  • Initialization - Make preparations before beginning the repetition
  • Test Condition - A condition which is repeatedly used to determine whether or not to continue with another repetition.
  • Activity - The activity which is repeated.
    • Warning: when describing such a loop, it is important to make sure that the elements combine in a way so that the activity eventually results in the termination of the repetition.

    Common styles for specifying such repetitions includes:

  • for (count from 1 to N) do (activity)
  • while (condition) do (activity)
  • repeat (activity) until (condition)
    • See Figures 4.8-4.9 of [Br].

  • Sequential Search 
      Select the first entry in the list as the test entry.

  while ( (target value > test entry) AND
          (there remain entries to be considered) )
    do (Select the next entry in the list as the test entry)


  if (target value = test entry)
    then (Declare the search a success.)
    else (Declare the search a failure.)
    The above algorithm works in general. However it fails to be well-defined in the special case that the original list is empty. We could fix this oversight, and rewrite our algorithm in a more general form as a procedure with two parameters: 
      procedure Search (List, TargetValue)
    if (List empty)
      then
        (Declare search a failure)
      else
        [Select the first entry in List as the test entry;
          while ((TargetValue > test entry) AND 
                 (there remain entries to be considered))
            do (Select the next entry in List as the test entry.)

          if (TargetValue = test entry)
            then (Declare search a success.)
            else (Declare search a failure.)
        ]

    • Divide and Conquer (recursion) (Ch. 4.5 [Br])



  • Binary Search

    Let's revisit the idea of searching for a target in a sorted list. Please turn to page 198 of Brookshear.

    (Did you use a sequential search to find page 198?)

    When searching a sorted list, we can take great advantage of the order. Specifically, if we compare the target to an item near the middle of the list we can reduce the original problem to one of two subproblems: either searching the first half of the list or searching the second half of the list. Note, that we will never have to search both halves. In fact, if we find the target at the middle, we can stop immediately.

    This intuition can be used to define a recursive algorithm for searching, called binary search. Not only will I use this idea to break the original problem into subproblems, but I will use the idea recursively to solve any necessary subproblem. 

      procedure Search (List, TargetValue)

    if (List empty)
      then
        (Declare the search a failure.)
      else
        Select the "middle" entry in List as the test entry;

        Execute one of the following blocks of instructions
          depending on whether TargetValue is equal to, less
          than or grater than the test entry

        Case 1: TargetValue = test entry
           (Declare this search a success.)

        Case 2: TargetValue < test entry
           [
            Apply the procedure Search to see whether TargetValue
            is in the portion of List preceding the test entry

            if (that search is successful)
              then (Declare this search a success.)
              else (Declare this search a failure.)
           ]

        Case 3: TargetValue > test entry
           [
            Apply the procedure Search to see whether TargetValue
            is in the portion of List following the test entry

            if (that search is successful)
              then (Declare this search a success.)
              else (Declare this search a failure.)
           ]

    Here is a Binary Search Demonstration which you can run to see some examples.

    • Efficiency (parts of pp. 206-211 [Br])

    So if you are searching for a name in a phonebook, why don't you use sequential search?
    Because it is not very efficient

    Specifically, if we are performing a sequential search on a list with n items, you can expect that on average you will have to examine half of the items. In the worst case, you may have to examine each item.

  • With 2000 items, you might expect to search 1000 of them
  • With 20000 items, you might expect to search 10000 of them
  • With 200000 items, you might expect to search 100000 of them
  • With n items, you might expect to search n/2 of them

    • How efficient is binary search?

    Every time we consider a test entry, we either find the target, or else we effectively divide the size of the list by a factor of two. For small lists, this may not be such an improvement, but for large lists this improvement becomes dramatic. When using binary search,

  • With 1000 items, you will need to examine at most 10 of them.
  • With 2000 items, you will need to examine at most 11 of them.
  • With 4000 items, you will need to examine at most 12 of them.
  • With 8000 items, you will need to examine at most 13 of them.
  • With 1 million items, you will need to examine at most 20 of them.
  • With 1 billion items, you will need to examine at most 30 of them.
  • With n items, you will need to examine at most log n of them.
    • The memory requirements will likely be more limiting then the time it takes to compare elements during the search.

    Implementation (pp. 248-251, 333 [Br])

    How is recursion implemented on a computer? The same way that general procedure calls are handled!

    This is briefly touched on in pp. 248-251 of Ch. 5.3. An even better picture is given, in a different context, by Figure 7.9 on p. 333 of [Br].

    Informally, I like to consider making a procedure call analogous to having an assistant do some of my work while I sit and relax. Specifically, the process is to

  • Stop what I am doing
  • Explain to my assistant what they need to do (passing them parameters of their task, perhaps)
  • ... idle while waiting ...
  • When the assistant returns, perhaps sending me a result, I resume my own task, in progress.
    • Behind the scenes, the programming language handles all the details by means of a procedure stack

    For Recursion, we note that this same mechanism can be used. That is, there is nothing stopping us from defining a procedure which calls itself to handle a subtask.

    The only key to doing this is to make sure that the recursion is defined in a way so that you will be sure that it terminates properly (just as we were concerned with terminating the repetition of loops).

    Termination generally takes place by defining a "base case" which can be solved without the need of recursion.

  • Let's look at this in the context of the binary search demonstration.
    Search('low' index, 'high' index, target value)

    • comp150 Class Page
    mhg@cs.luc.edu
    Last modified: 11 March 2002