Robot Soul

Hugh Loebner, inventor and main sponsor of the Loebner Prize Contest, has an interpretation of what the Turing Test is meant to test for that differs from mine, yet seems to be shared by most parties interested in that test today: the judge, while communicating via a teletype equivalent with two candidates (Loebner calls them "confederates"), has to determine which of the two is the human and which is the machine. In other words, to win at the Imitation Game, a machine has to specifically imitate a human. Only then, it could be concluded, Turing would have said a machine to be intelligent.

I argue that this would mean that Turing would not consider real-life equivalents to HAL9000, Commander Data, or any Asimov-style robot that doesn't pretend to be human to be intelligent. However, it appears to me that a statement he makes at the end of Section 2 of Computing Machinery and Intelligence indicates that he would do so:

It might be urged that when playing the 'imitation game' the best strategy for the machine may possibly be something other than imitation of the behaviour of a man. This may be, but I think it is unlikely that there is any great effect of this kind. In any case there is no intention to investigate here the theory of the game, and it will be assumed that the best strategy is to try to provide answers that would naturally be given by a man.

I'm interested in interactive characters in general, so my reading of the above is that Turing didn't care what kind of creature the machine imitates, just as long as it would react with English output to English input in a way that's to be be expected from any creature in order for an "intelligent" human to classify it as "intelligent". I mean, a real-life Commander Data equivalent will probably always answer the question "Are you human?" in the negative, but I, for one, would be likely to call such an artifact - if it could do in RL what Data does on TV - "intelligent", anyway. Based on my quote, I speculate that Turing would have done so, too, but it seems that the majority of his readers today insist that the "must imitate a human" rule is to be included for a Turing Test to be "the real" Turing Test. Anyway, that's how Loebner sees it, and since he foots the bill for the LPC, he totally pwns the contest rules.

Which he is just updating, to make the "writerly" interpretation of the term "character" completely irrelevant to next years contest. Ironically, he does so in part by focusing on on the ASCII sense of the term "character". He hasn't published the LPC 2006 rules on the contest homepage yet, but has already posted them to the Robitron, inciting intense discussion. The most controversial new rule is "Communications programs will be supplied by contest management."

I have written the communications programs. This is the way it will work: The confederates will sit in front of one computer, the judges will sit in front of one computer with a split screen having "Left" and "Right" screens. One screen will provide interactions with the bot, the other with the Confederate. Which screen is which will be decided by the flip of a coin. The entrants' computers will be in the hallway with the entrants. They will be able to monitor their programs if their programs write to the screen. The entrants' computers will run their programs only.

And as a way for those bot programs to interface with his comm program, he specifies the following algorithm:

1. Request the name of a directory once at start-up
2. Specify an output character by creating a sub-directory with whose name conforms to the naming convention "time.character.other" Time must be resolved to milliseconds or higher resolution. In perl, one simply uses the command "mkdir name" Other languages will use other commands.
3. a. Capture an input character by reading the names of all sub-directories with the extension ".judge". In perl one uses glob("*.judge")
b. Delete the sub-directory
c. Process the information

In other words, the shall be no end-of-message markers. The bots are supposed to look into a network directory, wait until a complete message was typed, and then respond to it. Towards the programmers, that's way cruel.

His reasoning:

The human confederate must face the same problem. If the human can do it, then the program must be able to do it also (it must imitate a human, remember).

I don't care how your program "knows" when to respond. My guess is that it should respond when it has received sufficient input to "understand" the input utterance of the judge or when it has
sufficient input to "decide" to respond. Perhaps after receipt of a "." (period) if the judge is kind enough to include them at the end of his remark.

Another new rule is that the first reply of all confererates, whether human or machine, must be either "Hello, my name is John and I am a man" or "Hello, my name is Joan and I am a woman". In other words, the machine must be smart enough to recognize an arbitrary message when it sees one, but - at least on its first turn - has to give a canned response!

Those are gnarly rules.

Update 05 Oct 2005:
Hugh Loebner just posted the LPC 2006 contest rules to comp.ai.nat-lang and declared the case a closed one. He also posted his communications program that next year's bots will have to interface with, and you know what? Contrary to his announcements about wanting to exclude "non-verbal cues" from the human-machine communication, the protocaol does allow for the transmission of the "return"-symbol and end-of-sentence markers like "!" and "?". That makes the task somewhat easier. Nevertheless, several professsional programmers on Robitron have complained that following his procedure will needlessly complicate the entrant programms without anybody having any advantage from that. Anybody except for Loebner, that is, who gets bragging rights: "I host my own contest, and I run it on my own code." Dude.