Researchers find AI fails strict Turing test design
Summary and headline written by AI from the source article. How we work
Researchers at the University of Canterbury in New Zealand examined Alan Turing’s original writings on artificial intelligence and conducted a new experiment to assess whether current AI systems meet his criteria.
Diane Proudfoot, Sharon Temtsin, and Christoph Bartneck determined that many experiments labelled “Turing tests” deviate from Turing’s initial concept for evaluating machine intelligence. Turing proposed an “imitation game” where a person tries to distinguish between a human and a machine through conversation.
The team discovered that commonly used rules for the Turing test, such as a five-minute time limit and a 30% success rate for fooling judges, were not part of Turing’s original design. Turing instead envisioned open-ended conversations with no time constraints, allowing interrogators to thoroughly probe a system’s understanding. In their experiment, 36 of 37 participants correctly identified GPT-4-Turbo, an AI language model from OpenAI, as the machine.
The researchers also found that most participants, 89%, needed more than five minutes to reach a decision, with conversations averaging 14 minutes and lasting up to 46 minutes. They suggest these findings indicate that shorter tests may not provide enough time to properly evaluate an AI’s capabilities and genuine understanding.
