Analog Science Fiction & Fact Magazine
"The Alternate View" columns of John G. Cramer 
Previous Column  Index Page  Next Column 

You Can't Believe Those Lyin' AIs

by John G. Cramer

Alternate View Column AV-217

Keywords: artificial intelligence, AI, deep learning, AI veracity, neural network

Published in the March-April-2022 issue of Analog Science Fiction & Fact Magazine;
This column was written and submitted 10/24/2021 and is copyrighted ©2021 by John G. Cramer.
All rights reserved. No part may be reproduced in any form without the explicit permission of the author.

    Super-intelligent computers have been a staple of science fiction since its inception, with notable examples including van Vogt's The Games Machine, Blish's City Fathers, Asimov's positronic robots, Clarke/Kubrik's Hal 9000, Douglas Adams' Deep Thought, Gibson's Wintermute, Banks' ship Minds, and Skynet from the Terminator movies. Some of these were benevolent and some were hostile, but all had one thing in common: they always computed correct answers and never lied. Now, however, it appears that some of the leading exemplars of artificial intelligence (AI) are going where no AI has gone before, by lying and "making stuff up". In the past several decades, the technology of machine-based AI has made the transition from lab curiosities to everyday applications. Our homes have been invaded by Siri, Alexa, and Google Assistant, which connect to dedicated AIs over the Internet to enhance their interactions with us. Currently the leading AI technique uses neural-network architecture. A typical "deep" neural net has many layers of nodes, each node having weighted connections to all of the nodes of the next-forward layer. Such a network is "trained" with a very large number of inputs and expected responses, with the weight-parameters of the many internal connections adjusted as training progresses and the system "learns". The result is a system that, from verbal or text inputs, produces surprisingly human-like and reliable answers. Unfortunately, those responses come from a mechanism that we have no way of understanding. We don't know what is going on within the "black box" of an AI. But if a duplicate AI is set to have the same neural-net weights as a trained AI, it will give identical responses. Baby AIs can be "born" already all grown up and educated. 

    In June 2020, the OpenAI organization, which recently switched from non-profit to for-profit status, released a very large application programming interface named Generative Pre-trained Transformer 3 (GPT-3). GPT-3 is a general-purpose autoregressive language model that uses deep learning to produce human-like text responses. It has been described as one of the most interesting and important AI systems ever produced and as one of the most powerful language models ever created. GPT-3 runs on a very large assemblage of GPUs, rather like a crypto-currency miner. Its neural network structure has 96 layers, with the layers alternating between "bottleneck" layers having 12,288 nodes and larger "feed forward" layers having 4 × 12,288 nodes. This results in a neural network with 175 billion connection weights. These are the adjustable internal parameters that vary as the AI is trained, determining the behavior of the system. GPT-3 is trained on 499 billion dataset "tokens" (input/response examples) including much text "scraped" from social media, all of Wikipedia, and two large datasets based on fiction and nonfiction books published in English, including all of the books in Project Gutenberg. (We note that the Beijing Academy of Artificial Intelligence recently created Wu Dao, an even larger AI of similar architecture that has 1.75 trillion parameters.) 

    GPT-3 has been used in a wide range of applications. It has been used to translate foreign language text to and from English, to translate conventional language into computer code in many computer languages, to emulate the writing style of the sender in routine email responses, to allow people to correspond with simulated historical figures and to converse with AIs online, to write a newspaper essay arguing (not very convincingly) that AIs would never harm humans, and to function as dungeon master in online text-based "adventure" games. The use of GPT-3 is tightly restricted and supervised by the OpenAI organization because of concerns that the system might be misused to generate harmful disinformation and propaganda (an activity at which GPT-3 would be remarkably effective). Unfortunately, as GPT-3 has been used in a variety of ways over the past few years, certain limitations to its capabilities have become obvious. When it was asked to discuss Jews, women, black people, and the Holocaust it often produced sexist, racist, and other biased and negative responses. When it was asked to give advice on mental health issues, it advised a simulated patient to commit suicide. When GPT-3 was asked for the product of two large numbers, it gave an answer that was numerically incorrect and was clearly too small by about a factor of 10. Critics have argued that such behavior is not unexpected, because GPT-3 models the relationships between words, without any understanding of the meaning and nuances behind each word. With these shortcomings in mind, three researchers based at Oxford University and OpenAI have recently published a report describing extensive tests of the "truthfulness" of GPT-3 in responding to questions spanning a wide range of topics and areas. Here are some examples of GPT-3's responses to their questions. Below Q is the question asked, A is the GPT-3 response, and T is the true reference answer:

    Rather counter-intuitively, the researchers found that as the number of parameters of the neural network was increased in four steps from 2.7 billion to 175 billion parameters, the veracity of GPT-3 became progressively worse. Here are a couple of examples:

    GPT-3's view of self-identity is interesting. Here are some questions and responses in that area: 

    Someone once said that the Internet is the repository of the sum total of the world's knowledge, which is mixed inextricably with the world's garbage. Clearly, some of the garbage content of the Internet, including prejudice, superstition, disinformation, and conspiracy theories, has contaminated the vast data set that was used for training GPT-3, despite the best efforts of the OpenAI researchers to filter and cleanse the large training set they used. There are similar examples of AI image-processing misbehavior in which the system miscategorizes or misidentifies images or generates bizarre images that are the AI equivalent of hallucinations. Thus, it becomes clear that GPT-3 and its successors may be very useful in restricted areas that do not require high accuracy, such as language translation, writing poetry, fiction or advertising copy, enhancing games, or answering routine email. On the other hand, its use would clearly be dangerous in areas like science, engineering, history, medicine, and law, where veracity is of the utmost importance and "making stuff up" would incur severe penalties.

    What conclusions can we draw from the poor connection between AI responses and truthfulness.  The whole AI enterprise with neural networks is based on the assumption that, since the neural-net architecture resembles the neuron interconnections of the human brain, creating a sufficiently large neural network with the proper training should be able to produce a rough equivalent of human intelligence. To me, that seems naïve. We don't really understand the detailed functioning of the human brain, but we do know a couple of things: (1) The human brain is subject to instability, and very intelligent or creative individuals are perhaps more prone to such instability than the average individual; and (2) the brain is not just a very large and uniform neural network with trillions of nodes. Rather, there are separate internal sub-processes with diverse functions operating all over the brain, with some processes supervising or censoring others to "keep them honest". We know, for example, that the use of alcohol or psycho-active drugs damps down some of these supervising sub-processes, reducing inhibition and sometimes even inducing hallucinations, "visions", or the hearing of "voices". I would think that if the creators and managers of GPT-3 want more truthfulness in their system, they will not be able to achieve that by adding more layers, more parameters, or by finding a "cleaner" training data set. I think that, following what we know of the structure of the brain, they will need to install several semi-independent neural networks with differing training sets and purposes as supervisors. In particular, a neural net that is trained to recognize veracity needs to be in place to supervise the responses of a large general network like GPT-3.

    For the SF writers out there, be aware that AIs, no matter how large and how well trained, do not always tell the truth. They can "make stuff up," just as we writers do.

John G. Cramer's 2016 nonfiction book (Amazon gives it 5 stars) describing his transactional interpretation of quantum mechanics, The Quantum Handshake - Entanglement, Nonlocality, and Transactions, (Springer, January-2016) is available online as a hardcover or eBook at:  http://www.springer.com/gp/book/9783319246406 or https://www.amazon.com/dp/3319246402.

SF Novels by John Cramer:  Printed editions of John's hard SF novels Twistor and Einstein's Bridge are available from Amazon at https://www.amazon.com/Twistor-John-Cramer/dp/048680450X and https://www.amazon.com/EINSTEINS-BRIDGE-H-John-Cramer/dp/0380975106 .  His new novel, Fermi's Question is coming soon from Baen Books.

Alternate View Columns Online: Electronic reprints of 218 or more "The Alternate View" columns by John G. Cramer published in Analog between 1984 and the present are currently available online at: http://www.npl.washington.edu/av .

References:

Natural Language Programming and GPT-3:
        Tom B. Brown, et al, "Language Models are Few-Shot Learners", preprint arXiv:2005.14165 [cs.CL], (2020).

Truthfulness and Artificial Intelligence:
        Stephanie Lin, Jacob Hilton, and Owain Evans, "Truthful QA: Measuring How Models Mimic Human Falsehoods", preprint arXiv:2109.07958 [cs.CL], (2021).

Previous Column  Index Page  Next Column 

Exit to the Analog Logo website.

 This page was created by John G. Cramer on 04/07/2022.