EMBRACING 'I DON'T KNOW' IN AN AI-DRIVEN WORLD
To know that we know what we know, and to know that we do not know what we do not know, that is true knowledge. -- Nicolaus Copernicus
In the world of decision-making and problem-solving, conditional logic is a basic tool, guiding our choices based on specific conditions or criteria. This "if-then" approach is the backbone of many computational processes and human reasoning patterns. However, equally important to this sequential logic is the often-overlooked power of admitting uncertainty - the simple yet profound act of saying "I don't know." If you see the front yard is wet, you don’t necessarily know why, it could be rain, the sprinklers, or an epic water-balloon soaking. Until you get more information, the best answer to “If the yard is wet, then ____ happened” is “I don’t know”. We take this uncertainty for granted, but children tend to recognize only concrete relationships, not hypothetical ones. With some teaching, even young children can recognize logical situations where one can’t know and apply proper conditional logic to relevant situations.
This admission of knowledge gaps is intellectually honest and paves the way for curiosity, genuine learning and growth. While humans can sometimes struggle with acknowledging uncertainty, this challenge is even more pronounced in the realm of artificial intelligence (AI). Today’s Large Language Models (LLMs), despite their impressive capabilities, struggle with expressing uncertainty, sometimes leading to a concerning phenomenon known as "hallucination" - where they generate plausible-sounding but incorrect information rather than admitting a lack of knowledge. This false sense of confidence can be very misleading and potentially manipulating. Can LLMs learn how to recognize when they don’t know something? That would certainly improve their usefulness and adoption.
NOTE: We are continuing our experiment with an AI-generated podcast that summarizes this post by Google’s NotebookLM. Listen here and let us know what you think:
UNDERSTANDING CONDITIONAL LOGIC
Conditional logic is a method of reasoning and decision-making based on specific conditions or criteria. It follows an "if-then" structure, where a particular action or conclusion is triggered only if certain conditions are met. This form of logic is a basic building block in human thought processes and forms the basis for many of our daily decisions. For instance, we might think, "If it's raining, then I'll take an umbrella," or "If it’s raining, then my dog’s paws will get wet and need drying." These examples illustrate how conditional logic helps us navigate various situations by considering relevant factors before making choices.
In computer science and programming, conditional logic plays a more formal and structured role. It's a fundamental concept in coding, allowing developers to create programs that can make decisions and execute different actions based on varying inputs or states. Programming languages typically implement conditional logic through constructs like "if-else" statements, "switch" cases, and ternary operators. These tools enable software to behave dynamically, responding to user inputs, system states, or data conditions. For example, a banking application might use conditional logic to determine whether a transaction should be approved based on the account balance, or a game might use it to trigger different events depending on a player's actions. This application of conditional logic in computing allows for the creation of complex, responsive systems that can handle a wide array of scenarios and user interactions. Because of these specific rules, they will reliably behave the same way under the same conditions.
Let’s take the general rule “if p, then q” where every time p happens, then q happens. But if q happens, did p happen? If p doesn’t happen, did q happen? These are the situations where “I don’t know” is important. With the information provided, there is no way of knowing in some situations.
To use our previous example of “if it is raining, then the ground is wet”, here are the resulting logical conclusions.
As you can see, there are times when we are logically uncertain in these situations and should be able to recognize and communicate this uncertainty.
THE ROLE OF "I DON'T KNOW" IN DECISION-MAKING
However, much of the time, either we don’t know the exact rules or have enough information to be sure. Acknowledging uncertainty is an important aspect of human cognition and decision-making. Our brains are constantly processing vast amounts of information, yet there are always gaps in our knowledge or understanding. Recognizing and admitting these gaps through the simple phrase "I don't know" is a powerful cognitive tool. It allows us to pause, reflect, and approach situations with a more open and inquisitive mindset. This acknowledgment of uncertainty is not a sign of weakness, but rather a demonstration of intellectual honesty and metacognitive awareness - the ability to think about our own thinking processes. It is also a sign of possibilities, where curiosity can drive us to figure out the stories and explanations.
Admitting a lack of knowledge offers several significant benefits. Most importantly, it promotes learning and growth by creating opportunities for acquiring new information. When we acknowledge what we don't know, we open ourselves up to seeking out new knowledge and perspectives. This attitude fosters continuous learning and personal development. In my teaching and presentations, I’ve found that saying "I don't know" builds trust and credibility. It demonstrates honesty and integrity, showing others that we value accuracy over the appearance of omniscience. People are more likely to trust and respect someone who can admit their limitations rather than someone who pretends to know everything. Finally, acknowledging uncertainty plays a crucial role in preventing the spread of misinformation. By refraining from making unfounded claims or guesses, we avoid contributing to the proliferation of false or misleading information.
In my undergraduate honors thesis (The Effects of Pragmatic Reasoning Training on Conditional Reasoning in Children and Adults), I explored how proficient people are at reasoning through conditional logic and found that training on the “I don’t know” situations helped both adults and children. When armed with the power of uncertainty, even elementary school children could correctly navigate all four logical conditions of “if…then” statements – albeit in a context that made sense to them. Monopoly. I won’t bore you with the details of the study, but once children understand that if someone lands on Park Place, it is unknown whether they bought it until the round is over. This new power allowed them to solve plenty of other “if…then” statements after learning in the context of the game.
LARGE LANGUAGE MODELS AND THE STRUGGLE WITH UNCERTAINTY
Large language models (LLMs) are advanced artificial intelligence systems designed to understand and generate human-like text. These models, such as OpenAI’s GPT (Generative Pre-trained Transformer) series, are trained on huge amounts of textual data from the internet and other sources. They use complex neural networks to learn patterns in language, enabling them to perform a wide range of tasks including translation, summarization, question-answering, and creative writing. LLMs have shown remarkable capabilities in processing and generating coherent and contextually appropriate text, often producing responses that can be difficult to distinguish from those written by humans.
Despite their apparent sophistication, LLMs are challenged when it comes to expressing uncertainty or admitting a lack of knowledge. Their struggle is real. First, the training data used to develop these models often contains biases and inaccuracies present in human-generated content on the internet. These biases can be inadvertently learned and reproduced by the model, hindering its ability to recognize incorrect information. Second, the optimization processes used in training LLMs typically reward confident and specific answers. The models are often trained to maximize the likelihood of generating the most probable next word or sentence, which can lead to a preference for definitive statements over expressions of uncertainty. This "pressure to provide an answer" is further reinforced by how these models are typically used and evaluated, where direct and specific responses are often favored over more nuanced or uncertain ones.
The difficulty LLMs have in expressing uncertainty can lead to a phenomenon known as hallucination in AI responses. In the context of AI, hallucination refers to the generation of information that is plausible-sounding but factually incorrect or entirely fabricated. This occurs when the model produces content that goes beyond its training data or makes connections that aren't based on real-world facts. Examples of AI hallucinations can include generating non-existent historical events, inventing fake scientific studies, or creating false biographical information about real people. For instance, an LLM might confidently a lawsuit that was never filed, or provide detailed but entirely fabricated statistics about a topic. These hallucinations can be particularly problematic because they often appear convincing and are delivered with the same level of confidence as accurate information, making it difficult for users to distinguish between fact and fiction without external verification.
CONSEQUENCES OF AI HALLUCINATION
The consequences of AI hallucination extend far beyond mere inaccuracies, posing significant challenges in our increasingly AI-dependent world. Perhaps the most immediate and pervasive impact is the potential for widespread misinformation. As large language models are integrated into search engines, content creation tools, and various information systems, hallucinated responses can rapidly spread false or misleading information. This problem is exacerbated by the perceived authority of AI systems, which can lead users to accept generated content without critical evaluation. In an era where digital literacy is crucial, the spread of AI-generated misinformation can contribute to confusion on important topics, fuel conspiracy theories, and even influence public opinion on critical issues like health, politics, and science.
The prevalence of AI hallucinations can lead to a broader erosion of trust in AI systems and the organizations that deploy them. As users encounter inaccurate or fabricated information from AI sources, they may become increasingly skeptical of all AI-generated content, even when it's accurate and valuable. This loss of trust could hinder the adoption and effective use of AI technologies in fields where they could provide significant benefits, such as healthcare, education, and scientific research. The real-world impacts of this erosion of trust and spread of misinformation can be far-reaching. In professional settings, decisions based on hallucinated AI outputs could lead to financial losses, legal issues, or compromised safety measures. In personal contexts, individuals might make poor health choices, financial decisions, or life plans based on unreliable AI-generated advice. As AI systems become more integrated into critical infrastructure and decision-making processes, the potential for hallucinations to cause serious harm – from minor inconveniences to major societal disruptions – becomes an increasingly pressing concern that demands urgent attention and solutions.
Can we teach LLMs to only respond when they are confident of the answer? Whether there is additional training required through RLHF or a “confidence score” provided with each response that denotes how sure the model is of the output could go a long way towards communicating uncertainty and helping people properly weight a model’s output. R-Tuning explicitly trains large language models (LLMs) to express uncertainty by generating refusal-aware responses by fine tuning using a dataset that includes both certain and uncertain answers. The system is prompted with questions like "Are you sure you accurately answered the question based on your internal knowledge?" to encourage the model to refuse an answer when it's unsure. Like children, learning when to say “I don’t know” is a powerful tool for reasoning and interpersonal trust. And if you had an intern or assistant, you would want to get an idea how sure they are in their work.
CONCLUSION
The ability to acknowledge uncertainty and say "I don't know" is a crucial skill in both human cognition and artificial intelligence. For humans, it fosters intellectual honesty, promotes learning, and helps prevent the spread of misinformation. With AI, particularly LLMs, the challenge of expressing uncertainty highlights important areas for improvement and research. As we've seen, the consequences of AI hallucinations can be far-reaching, affecting trust in technology and potentially leading to real-world harm. Thus, we need to continue research and development in AI uncertainty quantification and expression. As this field progresses, it is equally important for individuals to cultivate a healthy relationship with uncertainty in their own lives. Embracing the unknown, being comfortable with ambiguity, recognizing the limits of our knowledge, and being open to possibilities are valuable skills in an increasingly complex world. By doing so, we not only improve our decision-making processes but also contribute to a more honest and nuanced understanding of the world around us. In both human and artificial intelligence, the path to true wisdom often begins with three simple words: "I don't know."
