Lecture #28 (25 April 2002)

Limits of Computation

Overall Reading
Decker/Hirshfield: pp. 278-290
Brookshear: (Optionally) Ch. 11.3-11.4

Outline:

  • Overview
  • Church-Turing Thesis (pp. 277-278 [DH])
  • Serial Numbers for Programs (pp. 280-281 [DH])
  • A Universal Turing Machine (pp. 281-282 [DH])
  • Impossible Programs (Mod. 8.3 [DH])
  • The Halting Problem (Mod. 8.3 [DH])

    • Overview

    The goal of this lecture is to show that there are certain computational tasks which cannot possibly solved by Turing Machines.

    This is a very important theorem in Computer Science. The proof is quite involved; we will try to give the general idea.

    The example we will use in this lecture is the following:

    Given a TM program and an initial tape, determine whether or not the TM will come to a halt.
    There is no way to program a Turing Machine which solves this task. This "halting problem" is undecidable.

    (the text gives several other examples of undecidable problems)

    Church-Turing Thesis (pp. 277-278 [DH])

    Note: This is only a conjecture not a proven theorem.

    It states that,
    a Turing Machine can be programmed to compute any function which is "computable" by some other algorithmic process.

    The reason that it is a conjecture and not a true theorem is that we need to first agree on a clean definition for "algorithmic process."

    What is certainly true is that a Turing Machine can compute any function which can be computed by contemporary machines. (not bad for a model from 1936).
    [Granted: using a Turing Machine may not be the most efficient method]

    Serial Numbers for Programs (pp. 280-281 [DH])

    We can easily give each Turing Machine a unique serial number.

    This isn't really a surprise.

  • Early in this class we talked about information stored on a computer being represented as 0's and 1's. Certainly, I can type a TM's rules using ASCII characters. So it's 0's and 1's.
  • Any string of 0's and 1's can be interpreted as a single integer, represented in binary. (perhaps a very big integer, but that's not the point)
    • For the record, the book comes up with a more convenient way to turn a TM into an integer. There are many possible systems.

    All that matters is that a TM can be converted to a number, and that the number can be converted back to the same TM.

    A Universal Turing Machine (pp. 281-282 [DH])

    By now, do you agree that if I gave you a Turing Machine and an input, that you could simulate that machine on that input?

    Hopefully you could. Why? Because it is straightforward; You just follow the rules.

    I bet we could write a program to do this for us!

    Though we won't, it is possible to build what we call a Universal Turing Machine

  • Input: two numbers separate in some way
    (one happens to be a serial number for a TM, the other represents 0's and 1's which can make up the input string for that TM)
  • Behavior: should mimic the given TM on the given input.

    • Impossible Programs (Mod. 8.3 [DH])

    Consider the following potential function "P" which takes integers to strings.
  • If running Mn on input n produces any output
    then P(n) = ""
  • If running TMn on input n produces no output
    then P(n) = "0"

    • Question: Can any Turing Machine computing this function P?
    Answer: No!

    If there were such a Turing Machine, that machine would have to have a serial number. But function P() is not computed by Mk for any such k.

    In particular,
    if Mk produces a non-empty string, then P(k)=""
    if Mk produces the empty string, then P(k)="0"

    Function P s noncomputable by Turing Machines

    And if we believe the Church-Turing thesis, noncomputable by any algorithm process.

    The Halting Problem (Mod. 8.3 [DH])

    Let's look back at function P? It seems that we could have created a Turing Machine. We already agreed that we could create a Universal Turing Machine which executes any machine Mk on any input string n.

    It seems a simple step from there. Given input n, calculate P(n) as follows:

    1. Simulate Mn on input n.
      1. If that simulation results in the empty string,
        then output "0"
      2. If that simulation results in a non-empty string,
        then output ""

    Why can't we build this program? What's the flaw?







    The problem is that we assume that when we run the simulation, we see the output of that simulation.

    What if that simulation never halts? P(n) should equal "0" so presumably our machine must stop the simulation to output "0".

    But how did it know that the simulation was never going to halt? Maybe it was just taking a really long time, but was five seconds away from producing its answer.

    In general, we can not determine whether a Turing Machine which has been running for a long time will run forever or eventually halt.

    comp150 Class Page
    mhg@cs.luc.edu
    Last modified: 25 April 2002