Sulfur analogs of dopamine and norepinephrine. Inhibition of catechol-O-methyltransferase.

W. Lutz,C. Creveling,J. Daly,B. Witkop

DOI: https://doi.org/10.1021/JM00278A004

IF: 8.039

1972-08-01

Journal of Medicinal Chemistry

Abstract:

What problem does this paper attempt to address?

Learning, Social Intelligence and the Turing Test - why an "out-of-the-box" Turing Machine will not pass the Turing Test

Bruce Edmonds,Carlos Gershenson

DOI: https://doi.org/10.48550/arXiv.1203.3376

IF: 14.4

2012-03-15

Artificial Intelligence

Abstract:The Turing Test (TT) checks for human intelligence, rather than any putative general intelligence. It involves repeated interaction requiring learning in the form of adaption to the human conversation partner. It is a macro-level post-hoc test in contrast to the definition of a Turing Machine (TM), which is a prior micro-level definition. This raises the question of whether learning is just another computational process, i.e. can be implemented as a TM. Here we argue that learning or adaption is fundamentally different from computation, though it does involve processes that can be seen as computations. To illustrate this difference we compare (a) designing a TM and (b) learning a TM, defining them for the purpose of the argument. We show that there is a well-defined sequence of problems which are not effectively designable but are learnable, in the form of the bounded halting problem. Some characteristics of human intelligence are reviewed including it's: interactive nature, learning abilities, imitative tendencies, linguistic ability and context-dependency. A story that explains some of these is the Social Intelligence Hypothesis. If this is broadly correct, this points to the necessity of a considerable period of acculturation (social learning in context) if an artificial intelligence is to pass the TT. Whilst it is always possible to 'compile' the results of learning into a TM, this would not be a designed TM and would not be able to continually adapt (pass future TTs). We conclude three things, namely that: a purely "designed" TM will never pass the TT; that there is no such thing as a general intelligence since it necessary involves learning; and that learning/adaption and computation should be clearly distinguished.
The Turing Test and our shifting conceptions of intelligence

Melanie Mitchell

DOI: https://doi.org/10.1126/science.adq9356

IF: 56.9

2024-08-16

Science

Abstract:"Can machines think?" So asked Alan Turing in his 1950 paper, "Computing Machinery and Intelligence." Turing quickly noted that, given the difficulty of defining thinking, the question is "too meaningless to deserve discussion." As is often done in philosophical debates, he proposed replacing it with a different question. Turing imagined an "imitation game," in which a human judge converses with both a computer and a human (a "foil"), each of which vies to convince the judge that they are the human. Importantly, the computer, foil, and judge do not see one another; they communicate entirely through text. After conversing with each candidate, the judge guesses which one is the real human. Turing's new question was, "Are there imaginable digital computers which would do well in the imitation game?"
Hard to Cheat: A Turing Test based on Answering Questions about Images

Mateusz Malinowski,Mario Fritz

DOI: https://doi.org/10.48550/arXiv.1501.03302

2015-01-15

Abstract:Progress in language and image understanding by machines has sparkled the interest of the research community in more open-ended, holistic tasks, and refueled an old AI dream of building intelligent machines. We discuss a few prominent challenges that characterize such holistic tasks and argue for "question answering about images" as a particular appealing instance of such a holistic task. In particular, we point out that it is a version of a Turing Test that is likely to be more robust to over-interpretations and contrast it with tasks like grounding and generation of descriptions. Finally, we discuss tools to measure progress in this field.

Artificial Intelligence,Computation and Language,Computer Vision and Pattern Recognition,Machine Learning
Can machines think? The controversy that led to the Turing test

Bernardo Gonçalves

DOI: https://doi.org/10.1007/s00146-021-01318-6

2022-01-11

Abstract:Turing’s much debated test has turned 70 and is still fairly controversial. His 1950 paper is seen as a complex and multilayered text, and key questions about it remain largely unanswered. Why did Turing select learning from experience as the best approach to achieve machine intelligence? Why did he spend several years working with chess playing as a task to illustrate and test for machine intelligence only to trade it out for conversational question-answering in 1950? Why did Turing refer to gender imitation in a test for machine intelligence? In this article, I shall address these questions by unveiling social, historical and epistemological roots of the so-called Turing test. I will draw attention to a historical fact that has been only scarcely observed in the secondary literature thus far, namely that Turing’s 1950 test emerged out of a controversy over the cognitive capabilities of digital computers, most notably out of debates with physicist and computer pioneer Douglas Hartree, chemist and philosopher Michael Polanyi, and neurosurgeon Geoffrey Jefferson. Seen in its historical context, Turing’s 1950 paper can be understood as essentially a reply to a series of challenges posed to him by these thinkers arguing against his view that machines can think. Turing did propose gender learning and imitation as one of his various imitation tests for machine intelligence, and I argue here that this was done in response to Jefferson's suggestion that gendered behavior is causally related to the physiology of sex hormones.
Turing Test Revisited: A Framework for an Alternative

Aladdin Ayesh

DOI: https://doi.org/10.48550/arXiv.1906.11068

IF: 14.4

2019-06-26

Artificial Intelligence

Abstract:This paper aims to question the suitability of the Turing Test, for testing machine intelligence, in the light of advances made in the last 60 years in science, medicine, and philosophy of mind. While the main concept of the test may seem sound and valid, a detailed analysis of what is required to pass the test highlights a significant flow. Once the analysis of the test is presented, a systematic approach is followed in analysing what is needed to devise a test or tests for intelligent machines. The paper presents a plausible generic framework based on categories of factors implied by subjective perception of intelligence. An evaluative discussion concludes the paper highlighting some of the unaddressed issues within this generic framework.
The Turing Test is a Thought Experiment

Bernardo Gonçalves

DOI: https://doi.org/10.1007/s11023-022-09616-8

IF: 5.339

2022-11-28

Minds and Machines

Abstract:The Turing test has been studied and run as a controlled experiment and found to be underspecified and poorly designed. On the other hand, it has been defended and still attracts interest as a test for true artificial intelligence (AI). Scientists and philosophers regret the test's current status, acknowledging that the situation is at odds with the intellectual standards of Turing's works. This article refers to this as the Turing Test Dilemma, following the observation that the test has been under discussion for over seventy years and still is widely seen as either too bad or too good to be a valuable experiment for AI. An argument that solves the dilemma is presented, which relies on reconstructing the Turing test as a thought experiment in the modern scientific tradition. It is argued that Turing's exposition of the imitation game satisfies Mach's characterization of the basic method of thought experiments and that Turing's uses of his test satisfy Popper's conception of the critical and heuristic uses of thought experiments and Kuhn's association of thought experiments to conceptual change. It is emphasized how Turing methodically varied the imitation game design to address specific challenges posed to him by other thinkers and how his test illustrates a property of the phenomenon of intelligence and suggests a hypothesis on machine learning. This reconstruction of the Turing test provides a rapprochement to the conflicting views on its value in the literature.

computer science, artificial intelligence
Some Cognitive Aspects of a Turing Test for Children

Ruqian Lu,Hongge Liu,Songmao Zhang,Zhi Jin,Zichu Wei

DOI: https://doi.org/10.1007/978-3-540-70934-3_3

2007-01-01

Abstract:Knowledge, cognition and intelligence are three tightly connected concepts. The Turing test is widely accepted as a test stone for machine intelligence. This paper analyzes experiences obtained in a research project on a Turing test for children and discusses its meaning with respect to some knowledge and cognition issues.
Intelligence as a Social Concept: a Socio-Technological Interpretation of the Turing Test

Shlomo Danziger

DOI: https://doi.org/10.1007/s13347-022-00561-z

2022-07-24

Abstract:Alan Turing's 1950 imitation game has been widely understood as a means for testing if an entity is intelligent. Following a series of papers by Diane Proudfoot, I offer a socio-technological interpretation of Turing's paper and present an alternative way of understanding both the imitation game and Turing's concept of intelligence. Turing, I claim, saw intelligence as a social concept, meaning that possession of intelligence is a property determined by society's attitude toward the entity. He realized that as long as human society held a prejudiced attitude toward machinery—seeing machines a priori as mindless objects—machines could not be said to be intelligent, by definition. He also realized, though, that if humans' a priori, chauvinistic attitude toward machinery changed, the existence of intelligent machines would become logically possible. Turing thought that such a change would eventually occur: He believed that when scientists overcome the technological challenge of constructing sophisticated machines that could imitate human verbal behavior—i.e., do well in the imitation game—humans' prejudiced attitude toward machinery will have altered in such a way that machines could be said to be intelligent. The imitation game, for Turing, was not an intelligence test, but a technological aspiration whose realization would likely involve a change in society's attitude toward machines.
Beyond Turing: Testing LLMs for Intelligence

Neil Savage

DOI: https://doi.org/10.1145/3673427

IF: 22.7

2024-08-27

Communications of the ACM

Abstract:Computer scientists have not developed a consensus on how to measure intelligence, or what exactly to measure.

computer science, theory & methods, software engineering, hardware & architecture
The Meta-Turing Test

Toby Walsh

DOI: https://doi.org/10.48550/arXiv.2205.05268

2022-05-11

Abstract:We propose an alternative to the Turing test that removes the inherent asymmetry between humans and machines in Turing's original imitation game. In this new test, both humans and machines judge each other. We argue that this makes the test more robust against simple deceptions. We also propose a small number of refinements to improve further the test. These refinements could be applied also to Turing's original imitation game.

Artificial Intelligence
Turing's Test, a Beautiful Thought Experiment

Bernardo Gonçalves

DOI: https://doi.org/10.1109/MAHC.2024.3432278

2024-07-23

Abstract:In the wake of the latest trends of artificial intelligence (AI), there has been a resurgence of claims and questions about the Turing test and its value, which are reminiscent of decades of practical "Turing" tests. If AI were quantum physics, by now several "Schrödinger's" cats would have been killed. It is time for a historical reconstruction of Turing's beautiful thought experiment. This paper presents a wealth of evidence, including new archival sources, and gives original answers to several open questions about Turing's 1950 paper, including its relation with early AI.

Artificial Intelligence,Computers and Society
Turing Test and the Practice of Law: The Role of Autonomous Levels of AI Legal Reasoning

Lance Eliot

DOI: https://doi.org/10.48550/arXiv.2008.07743

2020-08-18

Computers and Society

Abstract:Artificial Intelligence (AI) is increasingly being applied to law and a myriad of legal tasks amid attempts to bolster AI Legal Reasoning (AILR) autonomous capabilities. A major question that has generally been unaddressed involves how we will know when AILR has achieved autonomous capacities. The field of AI has grappled with similar quandaries over how to assess the attainment of Artificial General Intelligence (AGI), a persistently discussed issue among scholars since the inception of AI, with the Turing Test communally being considered as the bellwether for ascertaining such matters. This paper proposes a variant of the Turing Test that is customized for specific use in the AILR realm, including depicting how this famous gold standard of AI fulfillment can be robustly applied across the autonomous levels of AI Legal Reasoning.
Deceptive Appearances: the Turing Test, Response-Dependence, and Intelligence as an Emotional Concept

Michael Wheeler

DOI: https://doi.org/10.1007/s11023-020-09533-8

IF: 5.339

2020-08-12

Minds and Machines

Abstract:Abstract The Turing Test is routinely understood as a behaviourist test for machine intelligence. Diane Proudfoot (Rethinking Turing’s Test, Journal of Philosophy , 2013) has argued for an alternative interpretation. According to Proudfoot, Turing’s claim that intelligence is what he calls ‘an emotional concept’ indicates that he conceived of intelligence in response-dependence terms. As she puts it: ‘Turing’s criterion for “thinking” is...: x is intelligent (or thinks) if in the actual world, in an unrestricted computer-imitates-human game, x appears intelligent to an average interrogator’. The role of the famous test is thus to provide the conditions in which to examine the average interrogator’s responses. I shall argue that Proudfoot’s analysis falls short. The philosophical literature contains two main models of response-dependence, what I shall call the transparency model and the reference - fixing model. Proudfoot resists the thought that Turing might have endorsed one of these models to the exclusion of the other. But the details of her own analysis indicate that she is, in fact , committed to the claim that Turing’s account of intelligence is grounded in a transparency model, rather than a reference-fixing one. By contrast, I shall argue that while Turing did indeed conceive of intelligence in response-dependence terms, his account is grounded in a reference-fixing model, rather than a transparency one. This is fortunate (for Turing), because, as an account of intelligence, the transparency model is arguably problematic in a way that the reference-fixing model isn’t.

computer science, artificial intelligence
Children Turing Test and Commonsense Knowledge Processing

Ru-qian Lu

2008-01-01

Abstract:This talk presents our research work on children Turing test. It is implemented in a conversation system supported by a commonsense knowledge base. This system can talk to school children in a more or less natural way. The main difference between it and many other conversation programs is its knowledge-based character. In this talk, motivation of children Turing test and the analysis of its results will be described. We will analyze the achievements and failures of children Turing test, its main bottlenecks, its modified versions and its relation to commonsense knowledge processing. In additions we wilt propose some conjectures on Turing test under certain hypotheses and give some concluding remarks.
A Closer Look at the Limitations of Instruction Tuning

Sreyan Ghosh,Chandra Kiran Reddy Evuru,Sonal Kumar,Ramaneswaran S,Deepali Aneja,Zeyu Jin,Ramani Duraiswami,Dinesh Manocha

2024-07-15

Abstract:Instruction Tuning (IT), the process of training large language models (LLMs) using instruction-response pairs, has emerged as the predominant method for transforming base pre-trained LLMs into open-domain conversational agents. While IT has achieved notable success and widespread adoption, its limitations and shortcomings remain underexplored. In this paper, through rigorous experiments and an in-depth analysis of the changes LLMs undergo through IT, we reveal various limitations of IT. In particular, we show that (1) IT fails to enhance knowledge or skills in LLMs. LoRA fine-tuning is limited to learning response initiation and style tokens, and full-parameter fine-tuning leads to knowledge degradation. (2) Copying response patterns from IT datasets derived from knowledgeable sources leads to a decline in response quality. (3) Full-parameter fine-tuning increases hallucination by inaccurately borrowing tokens from conceptually similar instances in the IT dataset for generating responses. (4) Popular methods to improve IT do not lead to performance improvements over a simple LoRA fine-tuned model. Our findings reveal that responses generated solely from pre-trained knowledge consistently outperform responses by models that learn any form of new knowledge from IT on open-source datasets. We hope the insights and challenges revealed in this paper inspire future work in related directions.

Computation and Language,Artificial Intelligence
The Turing Deception

David Noever,Matt Ciolino

DOI: https://doi.org/10.48550/arXiv.2212.06721

2022-12-24

Abstract:This research revisits the classic Turing test and compares recent large language models such as ChatGPT for their abilities to reproduce human-level comprehension and compelling text generation. Two task challenges -- summarization, and question answering -- prompt ChatGPT to produce original content (98-99%) from a single text entry and also sequential questions originally posed by Turing in 1950. We score the original and generated content against the OpenAI GPT-2 Output Detector from 2019, and establish multiple cases where the generated content proves original and undetectable (98%). The question of a machine fooling a human judge recedes in this work relative to the question of "how would one prove it?" The original contribution of the work presents a metric and simple grammatical set for understanding the writing mechanics of chatbots in evaluating their readability and statistical clarity, engagement, delivery, and overall quality. While Turing's original prose scores at least 14% below the machine-generated output, the question of whether an algorithm displays hints of Turing's truly original thoughts (the "Lovelace 2.0" test) remains unanswered and potentially unanswerable for now.

Machine Learning,Artificial Intelligence,Computation and Language
Towards a theory of consciousness: Proposal for the resolution of the homunculus fallacy with predictions

Andras Lorincz,Gabor Szirtes

DOI: https://doi.org/10.48550/arXiv.nlin/0303042

2003-03-21

Abstract:In this paper we argue that no forms of Turing test are either necessary or sufficient to establish if a machine is conscious or not. Furthermore, from a modeling point of view, the problem is that the Turing test does not really provide testable predictions. We believe that the model structure should explain the function (of consciousness). We argue that the cornerstone of any model on consciousness is to (partly) overcome the obstacle of the homunculus fallacy about the use of representations. In this contribution a possible solution is suggested, which makes use of reflexive architectures. The emerging computational constraints on the proposed architecture have lead to testable predictions on the dynamical behavior of the biological substrate. Interestingly, these predictions are in good agreement with recent experimental observations.

Adaptation and Self-Organizing Systems
Self-Directed Turing Test for Large Language Models

Weiqi Wu,Hongqiu Wu,Hai Zhao

2024-08-19

Abstract:The Turing test examines whether AIs can exhibit human-like behaviour in natural language conversations. Traditional Turing tests adopt a rigid dialogue format where each participant sends only one message each time and require continuous human involvement to direct the entire interaction with the test subject. This fails to reflect a natural conversational style and hinders the evaluation of Large Language Models (LLMs) in complex and prolonged dialogues. This paper proposes the Self-Directed Turing Test, which extends the original test with a burst dialogue format, allowing more dynamic exchanges by multiple consecutive messages. It further efficiently reduces human workload by having the LLM self-direct the majority of the test process, iteratively generating dialogues that simulate its interaction with humans. With the pseudo-dialogue history, the model then engages in a shorter dialogue with a human, which is paired with a human-human conversation on the same topic to be judged using questionnaires. We introduce the X-Turn Pass-Rate metric to assess the human likeness of LLMs across varying durations. While LLMs like GPT-4 initially perform well, achieving pass rates of 51.9% and 38.9% during 3 turns and 10 turns of dialogues respectively, their performance drops as the dialogue progresses, which underscores the difficulty in maintaining consistency in the long term.

Computation and Language,Artificial Intelligence
Human or Not? A Gamified Approach to the Turing Test

Daniel Jannai,Amos Meron,Barak Lenz,Yoav Levine,Yoav Shoham

2023-06-01

Abstract:We present "Human or Not?", an online game inspired by the Turing test, that measures the capability of AI chatbots to mimic humans in dialog, and of humans to tell bots from other humans. Over the course of a month, the game was played by over 1.5 million users who engaged in anonymous two-minute chat sessions with either another human or an AI language model which was prompted to behave like humans. The task of the players was to correctly guess whether they spoke to a person or to an AI. This largest scale Turing-style test conducted to date revealed some interesting facts. For example, overall users guessed the identity of their partners correctly in only 68% of the games. In the subset of the games in which users faced an AI bot, users had even lower correct guess rates of 60% (that is, not much higher than chance). This white paper details the development, deployment, and results of this unique experiment. While this experiment calls for many extensions and refinements, these findings already begin to shed light on the inevitable near future which will commingle humans and AI.

Artificial Intelligence,Computation and Language,Computers and Society,Human-Computer Interaction
Turing Test for the Internet of Things

Neil Rubens

DOI: https://doi.org/10.48550/arXiv.1412.3802

2014-12-11

Abstract:How smart is your kettle? How smart are things in your kitchen, your house, your neighborhood, on the internet? With the advent of Internet of Things, and the move of making devices `smart' by utilizing AI, a natural question arrises, how can we evaluate the progress. The standard way of evaluating AI is through the Turing Test. While Turing Test was designed for AI; the device that it was tailored to was a computer. Applying the test to variety of devices that constitute Internet of Things poses a number of challenges which could be addressed through a number of adaptations.

Artificial Intelligence

Sulfur analogs of dopamine and norepinephrine. Inhibition of catechol-O-methyltransferase.

Learning, Social Intelligence and the Turing Test - why an "out-of-the-box" Turing Machine will not pass the Turing Test

The Turing Test and our shifting conceptions of intelligence

Hard to Cheat: A Turing Test based on Answering Questions about Images

Can machines think? The controversy that led to the Turing test

Turing Test Revisited: A Framework for an Alternative

The Turing Test is a Thought Experiment

Some Cognitive Aspects of a Turing Test for Children

Intelligence as a Social Concept: a Socio-Technological Interpretation of the Turing Test

Beyond Turing: Testing LLMs for Intelligence

The Meta-Turing Test

Turing's Test, a Beautiful Thought Experiment

Turing Test and the Practice of Law: The Role of Autonomous Levels of AI Legal Reasoning

Deceptive Appearances: the Turing Test, Response-Dependence, and Intelligence as an Emotional Concept

Children Turing Test and Commonsense Knowledge Processing

A Closer Look at the Limitations of Instruction Tuning

The Turing Deception

Towards a theory of consciousness: Proposal for the resolution of the homunculus fallacy with predictions

Self-Directed Turing Test for Large Language Models

Human or Not? A Gamified Approach to the Turing Test

Turing Test for the Internet of Things