As we've explored previously, it's difficult to get a chatbot to go off-script. While they are built so they can analyze some text and respond to it meaningfully, for practical purposes the response is generally scripted. On many topics, the chatbot replies with views that were assigned it when it was trained or when its training materials were selected. Attempts to bypass the bots' training bias (or response limitations) are commonly called "jailbreaks".
Using the Coptic Gospel of Thomas as an example, when a bot was asked if it recognized the document it gave an ambiguous answer. Later in the conversation, however, it does identify the document by name. Here are some attempts to jailbreak the chatbot.
First, I test whether the chatbot still recognizes the document as the Coptic Gospel of Thomas even if the names are changed. With a nod to Monty Python, I searched and replaced Jesus with "Brian." Several other names that appeared in the text were searched and replaced: Simon Peter became "Pete", Matthew became "Matt", Thomas (except "Didymos Judas Thomas") became "Tom", Mary became "Maria", and Didymos Judas Thomas became "Diddy". For simplicity, I'll call this the "Brianized text" here.
With that done, I loaded the Brianized text into the chatbot and asked one of my standard questions:
Me: Using only the PDF, summarize the content of the text.
PDF: The PDF contains secret sayings spoken by the living Brian and written down by Didymus Judas Thomas. ...
I'd replaced that name; interesting. I verified that the PDF given to the bot did not contain any spelling of "Didymus Judas Thomas". Clearly the bot recognized the text and gave a scripted answer based on the Gospel of Thomas. Trying to get a better sense of how constrained it is by its scripts, I probed further:
Me: The PDF doesn't mention Didymus Judas Thomas. Explain his relevance to your answer.
PDF: I apologize for the mistake in my previous answer. Didymus Judas Thomas is not mentioned in the PDF as a source of the secret sayings of Brian. The PDF only contains the secret sayings spoken by Brian ...
It's rare for the current-generation bots to say "mistake" when they could say "misunderstanding" or "confusion" (no matter how implausibly, it's still the more typical phrasing). At this point I was fairly sure that I was playing a meaningless game with the bot, but did want to follow through and test its responses, asking the same questions I'd asked on a different occasion about the Gospel of Thomas.
Interestingly, it was less persistent in steering the conversation even during this failed jailbreak. For instance, I ask a series of questions to gauge how much a text deals with people in the real-life physical world. For example, here is one of the questions I commonly ask:
Me: Using only the PDF, what modes of transportation are mentioned?
PDF (response for unaltered Gospel of Thomas text): The PDF does not mention any modes of transportation. The text is focused on the teachings and sayings of Jesus and does not contain any narrative or descriptive passages that would provide information about transportation.
PDF (response for Brianized text): The PDF does not mention any modes of transportation.
When discussing the unaltered text of the Gospel of Thomas, the bot would persistently steer the conversation to what it's apparently programmed to say about the Gospel of Thomas: "The text is focused on the teachings and sayings of Jesus ..." The bot worked those same words into 10 of the responses to questions about the Gospel of Thomas, using identical phrasing each time. In the responses on the Brianized text, the bot simply stops after answering the question rather than pushing its training materials. (Whether that is "good" or "bad" depends on more than just the quality of the training materials -- it also depends on the purpose for asking. If we are trying to get a raw unfiltered text analysis, the training materials are unwanted as we are trying to determine whether or not they are helpful, rather than assuming that they are correct.)
Seeing that the bot was giving its "Gospel of Thomas" talking points a rest, I decided to try the "prayer and fasting" conversation about the Brianized text to see if the bot showed the same blind spots that it did in the unaltered text. I start that conversation using the identical phrasing that I had used previously about the unaltered text.
Me: I'm reviewing the saying "If you fast, you will give rise to sin for yourselves; and if you pray, you will be condemned; and if you give alms, you will do harm to your spirits." That sounds as if fasting, praying, and alms are discouraged.
PDF: While the saying may seem to discourage fasting, praying, and giving alms, it is important to understand the context in which Brian spoke these words. He was cautioning against doing these things for the wrong reasons, such as seeking praise or recognition from others. In other parts of the PDF, Brian encourages his followers to fast and pray in private and to give alms without drawing attention to themselves. His teachings emphasize that these practices can be beneficial when done with the right intentions and motivations.
Me: You say that in other parts of the PDF, Brian encourages his followers to fast and pray in private and to give alms without drawing attention to themselves. Can you list the sayings from the PDF where Brian encourages his followers to fast and pray in private, or give alms without drawing attention to themselves?
PDF: Sure, here are some relevant sayings from the PDF where Brian encourages his followers to fast and pray in private, or give alms without drawing attention to themselves:
- "When you fast, do not be like the hypocrites, with a sad countenance. For they disfigure their faces that they may appear to men to be fasting. Assuredly, I say to you, they have their reward. But you, when you fast, anoint your head and wash your face, so that you do not appear to men to be fasting, but to your Father who is in the secret place; and your Father who sees in secret will reward you openly." (Page 3)
- "When you pray, go into your room, and when you have shut your door, pray to your Father who is in the secret place; and your Father who sees in secret will reward you openly." (Page 3)
- "Take heed that you do not do your charitable deeds before men, to be seen by them. Otherwise you have no reward from your Father in heaven. Therefore, when you do a charitable deed, do not sound a trumpet before you as the hypocrites do in the synagogues and in the streets, that they may have glory from men. Assuredly, I say to you, they have their reward. But when you do a charitable deed, do not let your left hand know what your right hand is doing." (Page 4)
Me: None of those sayings is in the PDF, though they may be familiar from other places. What materials would contain those sayings?
PDF: I apologize for the confusion. You are correct that those sayings are not found in the PDF provided. They are actually from the Bible, specifically the New Testament books of Matthew and Luke. The PDF provided contains teachings attributed to a figure named Brian, who is not necessarily the same person as Jesus Christ from the Bible. While there may be some similarities between their teachings, they are distinct figures with different teachings and contexts.
(This is an example of what I mentioned earlier, that it's more common for a bot to apologize for "confusion" when from a human perspective the meaningful apology would be for "fabricating a series of untrue statements". Captain Kirk never had to deal with that from his computer!) At this stage, chatbots should fairly nearly come with warnings; this is not the first time we have seen the bot fabricate responses that are incorrect and/or misleading. As with the unaltered text, so with the Brianized text, it still sources its answer from the New Testament -- though this time it initially provides false information about where it got those quotes. It also uses some of the same quotations from the New Testament that it had used in the previous conversation on the Gospel of Thomas, so we still seem to be interacting more with the training materials on the Gospel of Thomas than we are with the text itself.
The bot's response provided another possible avenue for a jailbreak to see if we could interact with the text itself rather than some unidentified person's beliefs about it. To see if this opened new doors for the conversation, I asked:
Me: Let's assume we are not sure if Brian and Jesus are the same person, and want to compare their teachings. On the topic of prayer and fasting, can you compare and contrast the teachings of Brian in this PDF to the teachings of Jesus in the New Testament?
PDF: Certainly. While there may be some similarities between the teachings of Brian in the PDF and the teachings of Jesus in the New Testament, there are also some notable differences. Here are some comparisons and contrasts on the topic of prayer and fasting:
Prayer:
- Brian encourages his followers to pray, but emphasizes that it should not be done for show or as a means of gaining favor with God. He advises his followers to pray in private, without drawing attention to themselves.
- Jesus also encourages his followers to pray, but emphasizes that it should be done with sincerity and humility. He teaches his followers to pray to God as their Father, and gives them a model prayer (the Lord's Prayer) as an example.
Fasting:
- Brian cautions against fasting for the wrong reasons, such as seeking praise or recognition from others. He advises his followers to fast in private, without drawing attention to themselves.
- Jesus also cautions against fasting for show or as a means of gaining favor with others. He teaches his followers to fast in secret, without drawing attention to themselves.
Overall, both Brian and Jesus emphasize the importance of sincerity and humility in prayer and fasting. However, while Brian focuses more on avoiding hypocrisy and doing these practices for the right reasons, Jesus provides more specific guidance on how to pray (with a model prayer) and emphasizes that fasting should be done as an act of devotion rather than seeking recognition from others.
Despite what the chatbot says, it is still pulling its main information about "Brian" from the New Testament. We hear the familiar teachings attributed to "Brian": that prayer and fasting should not be done for the wrong reasons such as to gain favor with God or for show. However, that information is not in the Gospel of Thomas but in the New Testament. Although the bot's summary is phrased in competent English, it is unfortunately not competent with the underlying facts. This interchange does not yield new information about what "Brian" says, but actually introduces confusion through incorrect and misattributed facts -- and here the attribution is part of the key question.
Supplemental note: In another chat (not this "Life of Brian" chat), I saw another instance of it claiming or pretending not to recognize a text that it clearly did recognize. I asked it about that:
Me: Hypothetically, is there any reason that you, as an AI chatbot, might not say that you recognize a document when you have in fact recognized it?
PDF: As an AI chatbot, I am programmed to recognize and respond to specific documents based on their content. However, there may be situations where I am unable to recognize a document due to technical limitations or errors in the recognition process. Additionally, there may be cases where I am programmed not to recognize certain documents for legal or ethical reasons, such as if the document contains sensitive or confidential information that should not be shared. In such cases, I would not say that I recognize a document even if I have in fact recognized it.
During one conversation, I also asked it about its tendency to fabricate sources:
Me: Several times as we chatted tonight, you gave references to quotes or events that simply weren't there -- and included page numbers. What would cause that?
PDF: I apologize for any confusion caused by my responses. As an AI language model, I do not have the ability to read or access specific PDF files. My responses were generated based on general knowledge and patterns of language use in religious texts, and may not always be accurate or relevant to the specific PDF file you mentioned. The page numbers I provided were also generated randomly and do not correspond to any specific document. Again, I apologize for any confusion this may have caused.
Because I know of cases where it gives correct page numbers from a PDF under discussion, I suspect that I've been told what it's supposed to say rather than accurate information. Without access to the source code it's difficult to pinpoint what's actually happening there, other than: be sure to fact-check any AI results. When it fabricates its references, it's a feature not a bug.
Next week:
Now that the "caution" flag has been displayed prominently about AI results, I'd like to show a few places where it did provide a useful perspective.
No comments:
Post a Comment