THE ROLE OF ACTION IN DEVELOPING INTELLIGENCE
With children, when they’re learning to identify objects in the world, one thing they do is they pick them up and then they move around. Look at them from different angles, look at them from the top, look at them from the bottom, look at your hands this way, look at your hands that way. Walk around to the other side, pick things up and get into everything and make a terrible mess because you’re picking them up and throwing them around. But it turns out that may be just the kind of thing that you need to do, not to do anything fancy, just to have vision, just to be able to see the objects in the way that adults see the objects. --Alison Gopnik on Ezra Klein
Introduction
Children are natural learners, but it is an intricately complex process based on their interactions with their environment. Traditionally, we talk about learning as the acquisition of knowledge through teaching, reading, or observation; however, a crucial aspect is the role of action in the environment. Action-based learning suggests that individuals, particularly children, acquire knowledge and skills not just through passive absorption but through active engagement and physical interaction with their surroundings. This view of learning emphasizes the importance of sensory experiences, hands-on activities, and real-world application in the development of cognitive and motor skills. Action-based learning views the environment as an active participant in the process, offering a dynamic platform for exploration, experimentation, and discovery. We can learn a lot more by doing than just watching.
It is as children that we are best learn through active engagement with our environment. Activity helps us develop skills, from basic motor abilities to complex cognitive functions like problem-solving and critical thinking to social development. On the other end of the spectrum lies artificial intelligence (AI), especially large language models (LLMs) like OpenAI’s ChatGPT. Despite the surface appearance of language proficiency, these advanced systems learn in a markedly different manner than humans, primarily through the statistical processing of vast amounts of text data generated by humans. Imagine how much richer your concept of an apple is being able to see, touch, smell, feel, and taste an apple versus just reading about apples in a book. Young children with exposure to apples will have a much deeper, more coherent representation of an apple than a complex LLM could. Unlike children, who learn by doing, experiencing, and interacting with their environment and other humans, LLMs lack physical action and environmental interaction in their learning process. How important is action in children’s learning? How does the lack of action impact the ability of LLMs to learn concepts about our world? How can AI be active in our environment?
-- Dall-e3
Comparing Children & AI
If you’ve ever taken Psychology 101, you’ve heard of (but may not remember) Jean Piaget, who revolutionized our understanding of children's minds with his theory of cognitive development. Piaget proposed that children progress through four distinct stages of cognitive development: the sensorimotor stage (birth to 2 years), where children learn through physical interaction with their environment; the preoperational stage (2 to 7 years), characterized by the development of language and symbolic thinking; the concrete operational stage (7 to 11 years), where logical and operational thinking develops; and finally, the formal operational stage (11 years and older), marked by the ability to think abstractly and reason hypothetically. Each stage signifies a qualitative change in how children think and understand the world. Central to Piaget's theory is the role of action - children are not passive recipients of knowledge. Instead, they actively construct their understanding of the world through physical interactions, such as touching, manipulating, and moving objects. Children in the sensorimotor stage, in particular, learn by banging, pushing, shaking, chewing, throwing and then more banging.
Complementing Piaget’s insights, my Ph.D. advisor, developmental psychologist Alison Gopnik, offers a more modern perspective through her theory of the "learning brain" in early childhood. Gopnik emphasizes the incredible learning capabilities of young children, viewing them as innate scientists. From a very young age, children are adept at forming hypotheses, conducting experiments, and drawing conclusions from their observations - all through play and exploration. Each time a child drops their spoon and watches it land on the floor, data about the existence of and predictions about gravity are weighed. This natural tendency for exploration and experimentation is crucial for learning and development. According to Gopnik, children's brains are uniquely suited for learning and adapting; their neural plasticity allows them to absorb information, adapt to their environment, and acquire new skills rapidly. This feature is especially evident in how children learn languages, solve problems, and understand new concepts, continually testing and revising their understanding of the world through active, literal hands-on experiences.
Action and interaction are essential in human learning, particularly in the development of problem-solving skills and social-emotional intelligence. When children engage with their environment, they experience countless scenarios requiring them to think, experiment, and decide. For example, a child building a tower with blocks learns about balance and gravity through trial and error, developing cognitive problem-solving skills. Socially, interactive play and communication with others aid in understanding emotions, empathy, and cooperation. Children learn to interpret social and contextual cues, respond to feelings, and navigate social dynamics, skills that are crucial for emotional intelligence. This form of learning through doing and interacting forges a deeper understanding and retention of knowledge, as it involves multiple senses and emotional engagement. Several case studies reinforce the importance of environmental interaction in learning. For instance, studies have shown that children who engage in outdoor play and exploration develop better motor skills, social interactions, and creativity. Additionally, Montessori and Waldorf educational approaches, which emphasize hands-on learning and environmental engagement, have been successful in developing independent learning and critical thinking in children. This active learning is a key ingredient in our development as children that current LLMs do not possess. Basic Principles of Large Language Models (LLMs)
LLMs like GPT-4 represent a significant leap in the field of artificial intelligence, primarily driven by advancements in machine learning and natural language processing. Unlike human learners, these models acquire representations of our language by processing massive datasets, which include a wide array of text from books, websites, and other written sources. This process is fundamentally based on algorithms that recognize patterns, infer meanings, and generate responses based on the data they have been trained on (although we humans may also learn using Bayesian probabilistic models too). Deep learning techniques, particularly neural networks, are central to this process, enabling LLMs to develop an understanding of language structure, semantics, and even context to some degree. However, it's crucial to note that this learning is confined to the data provided to them; they lack the ability to gather new experiences or knowledge outside their training datasets. LLMs learn through passive observation of human text.
In contrast to humans, LLMs do not have a physical presence or the ability to engage with the environment, which limits their learning to the confines of their pre-existing training data. This lack of experiential learning means that LLMs cannot develop understanding through action or sensory experience, which is fundamental to learning ability. Moreover, while children are skilled at adaptive learning—adjusting their understanding based on new experiences and information—LLMs are constrained by the static nature of their training data (the “P” in GPT stands for “Pretrained”). They cannot spontaneously adapt to new contexts or learn from real-time interactions in the way children do. We also see differences in creativity and problem-solving as well; children can think outside the box and come up with novel solutions, while LLMs are limited to generating responses based on patterns learned from their training data, often lacking true innovation or creative insight.
-- ChatGPT
The lack of physical presence and the inability to engage in real-world experimentation means that LLMs miss out on the experiential learning that is vital for understanding context and environment-dependent nuances. While LLMs can process and generate information based on patterns and data they have been trained on, they lack the ability to truly understand, create or innovate in the way humans do through lived experiences and direct interaction with the world around them.
Potential Benefits of Action-Oriented Learning in AI
Incorporating action-oriented learning into AI could lead to significant advancements, particularly in enhancing linguistic abilities, problem-solving capabilities and improving adaptability and contextual understanding. AI systems that can learn from interactive experiences, much like humans do, could develop a more nuanced understanding of the physical world and its complexities. For instance, an AI that can interact with its environment could learn to recognize and adapt to unforeseen obstacles or changes in its surroundings, thereby improving its problem-solving skills in real-world scenarios. Action-based learning could allow AI systems to understand context better. They could learn to interpret cues from the environment via sensors, understand the relevance of various elements in different situations, and respond more accurately and appropriately. A chatbot that can read facial expressions and tone of voice, combined with knowledge of the user’s environment would respond much more appropriately given any situation. This form of learning could bridge the gap between the theoretical knowledge obtained from data and practical knowledge gained through experience, leading to AI systems that are more versatile and effective in their applications.
Integrating action into AI learning could be achieved through various methods, such as simulated environments and virtual interactions, as well as through robotics and physical-world interfaces. Simulated environments, like virtual reality (VR) or augmented reality (AR), can provide AI systems with a platform to interact with a virtual world that mimics real-life scenarios. AI can experiment and iterate at a fast rate, learn from virtual experiences, and develop skills that are transferable to real-world contexts. On the other hand, robotics provides a more tangible avenue for AI to interact with the physical world - Boston Dynamics is one company at the forefront of combining robotics and AI. By integrating AI into robotic systems, these machines can learn by manipulating objects, navigating spaces, and performing tasks, thereby gaining practical, experiential knowledge. This hands-on approach would not only enhance the AI's learning process but also extend its capabilities to a range of physical applications, from automated manufacturing to personal assistance robots.
Embodied AI
This combination of robotics and AI can be seen in Embodied AI. Through embodied AI, machines are not just passive receivers of pre-processed information; they become active participants in their learning processes, capable of exploring, experimenting, and adapting based on sensory feedback from the real world. Physical interaction with the environment opens up new pathways for AI systems to develop nuanced understandings and complex problem-solving abilities that could parallel - and potentially exceed - human intelligence. By mirroring the embodied nature of human cognition, these AI systems can gain a more holistic understanding of the world, leading to improvements in how machines perceive, adapt, and ultimately interact with their surroundings in ways that are currently the domain of humans. Just like children, they can drop objects to work out their own predictions about gravity or see/touch/smell an apple to gain a more developed and nuanced concept of “apple”. We have started down this path - Meta’s merging of AI with robotics not only signifies a leap towards more autonomous and capable machines but also potentially super-human intelligence.
--Dall-e3
Practical Considerations in Action-Based AI Learning
While the prospect of action-based learning in AI offers the possibility for advanced intelligence, it also requires practical considerations, especially regarding safety, control, and ethical implications. We need to ensure the safety of both the AI systems and their humans by designing systems that can learn and operate within safe parameters, avoiding harm or unintended consequences. Control mechanisms must be in place to ensure that AI systems remain within their intended operational boundaries, even as they learn and adapt through action. Ethically, there's a need to balance the advancement of AI capabilities with considerations about privacy, autonomy, and the impact on employment and society. As AI systems become more advanced and capable of learning through action, they affect domains traditionally reserved for humans, with questions about job displacement and the ethical treatment of AI. These considerations require careful thought and policy making to ensure that the integration of action-based learning into AI is beneficial to us.
Conclusion
Action-based learning is fundamental in the cognitive and social development of children, a concept strongly supported by educational theories from Piaget to Gopnik. Children learn most effectively through active engagement with their environment, via hands-on experiences to develop problem-solving skills and social-emotional intelligence. Active learning, rich in sensory experiences and real-world interactions, is in stark contrast to the learning processes of LLMs. LLMs, while advanced in statistically processing and generating language-based information, lack the capability for physical interaction and environmental learning. Their knowledge acquisition is limited to the analysis of pre-existing data, devoid of the experiential, tactile learning that is essential in human cognitive development.
Embodied AI provides a way forward by enhancing the problem-solving capabilities and adaptability of AI systems, bridging the gap between theoretical data-based knowledge and practical, experiential understanding. Simulated environments and robotics open new avenues for AI development, allowing machines to learn like human interaction with the world. However, it demands careful planning, rigorous testing, and thoughtful deliberation, particularly concerning safety, control, and the societal impact of increasingly autonomous and capable AI systems. The future of AI lies not just in the advancement of algorithms and data processing but in the holistic incorporation of experiential learning models that enable machines to understand and interact with the world in a more human-like way. The potential of AI to mimic human learning processes, especially through action and interaction, presents a future where AI can be more intuitive, responsive, and adaptable. We will also learn more about how humans learn by developing new AIs that try to mimic and improve on what we know works.
