Tani, Jun-Exploring Robotic Minds _ Actions, Symbols, And Consciousness as Self-Organizing Dynamic Phenomena-Oxford University Press (2016)

  i Exploring Robotic Minds ii OXFORD SERIES ON COGNITIVE MODELS AND ARCHITECTURES Series Editor Frank E. Ritter Series Board Rich Carlson Gary Cottrell Robert L. Goldstone Eva Hudlicka William G. Kennedy Pat Langley Robert St. Amant Integrated Models of Cognitive Systems Edited by Wayne D. Gray In Order to Learn: How the Sequence of Topics Influences Learning Edited by Frank E. Ritter, Joseph Nerb, Erno Lehtinen, and Timothy O’Shea How Can the Human Mind Occur in the Physical Universe? By John R. Anderson Principles of Synthetic Intelligence PSI: An Architecture of Motivated Cognition By Joscha Bach The Multitasking Mind By David D. Salvucci and Niels A. Taatgen How to Build a Brain: A Neural Architecture for Biological Cognition By Chris Eliasmith Minding Norms: Mechanisms and Dynamics of Social Order in Agent Societies Edited by Rosaria Conte, Giulia Andrighetto, and Marco Campennì Social Emotions in Nature and Artifact Edited by Jonathan Gratch and Stacy Marsella Anatomy of the Mind: Exploring Psychological Mechanisms and Processes with the Clarion Cognitive Architecture By Ron Sun Exploring Robotic Minds: Actions, Symbols, and Consciousness as Self-​Organizing Dynamic Phenomena By Jun Tani   iii Exploring Robotic Minds Actions, Symbols, and Consciousness as Self-​Organizing Dynamic Phenomena Jun Tani 1 iv 1 Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide. Oxford is a registered trade mark of Oxford University Press in the UK and certain other countries. Published in the United States of America by Oxford University Press 198 Madison Avenue, New York, NY 10016, United States of America. © Oxford University Press 2017 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, by license, or under terms agreed with the appropriate reproduction rights organization. Inquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above. You must not circulate this work in any other form and you must impose this same condition on any acquirer. Library of Congress Cataloging-​i n-​Publication Data Names: Tani, Jun, 1958– author. Title: Exploring robotic minds : actions, symbols, and consciousness as self-organizing dynamic phenomena / Jun Tani. Description: Oxford; New York: Oxford University Press, [2017] | Series: Cognitive models and architectures | Includes bibliographical references and index. Identifiers: LCCN 2016014889 (print) | LCCN 2016023997 (ebook) | ISBN 9780190281069 (hardcover : alk. paper) | ISBN 9780190281076 (UPDF) Subjects: LCSH: Artificial intelligence. | Robotics. | Cognitive neuroscience. Classification: LCC Q335 .T3645 2017 (print) | LCC Q335 (ebook) | DDC 629.8/9263—dc23 LC record available at https://lccn.loc.gov/2016014889 9 8 7 6 5 4 3 2 1 Printed by Sheridan Books, Inc., United States of America   v Contents Foreword by Frank E. Ritter  ix Preface  xiii Part I  On the Mind  1. Where Do We Begin with Mind?  3 2. Cognitivism  9 .1 Composition and Recursion in Symbol Systems  2 9 2.2 Some Cognitive Models  13 2.3 The Symbol Grounding Problem  16 2.4 Context  18 2.5 Summary  19 3. Phenomenology  21 3.1 Direct Experience  22 3.2 The Subjective Mind and Objective World  23 3.3 Time Perception: How Can the Flow of Subjective Experiences Be Objectified?  26 3.4 Being-​in-​the-​World  29 3.5 Embodiment of Mind  32 3.6 Stream of Consciousness and Free Will  37 3.7 Summary  41 v 7 Summary  136 Part II  Emergent Minds: Findings from Robotics Experiments  6.2 A New Understanding of Action Generation and Recognition in the Brain  55 4.4 Modeling the Brain at Different Levels  109 5.4 Deciding Among Conflicting Evidence  75 4.1 Robots with Subjective Views  141 6.3 Summary  172 .3 How Can Intention Arise Spontaneously and Become an Object of Conscious Awareness?  69 4.1 Development of Compositionality: The Symbol Grounding Problem  152 7.3 Behavior-​Based Robotics  103 5. Predictive Learning About the World from Actional Consequences  151 7.2 Gibsonian and Neo-​Gibsonian Approaches  93 5.5 Summary  77 5.6 Neurorobotics from the Dynamical Systems Perspective  125 5. New Proposals  141 6. Introducing the Brain and Brain Science  43 4.1 Hierarchical Brain Mechanisms for Visual Recognition and Action Generation  44 4.3 The Subjective Mind and the Objective World as an Inseparable Entity  148 7.2 Predictive Dynamics and Self-​Consciousness  161 7.2 Engineering Subjective Views into Neurodynamic Models  143 6.1 Dynamical Systems  83 5. Dynamical Systems Approach for Modeling Embodied Cognition  81 5.vi vi Contents 4.5 Neural Network Models  112 5. 2 Phenomenology  247 11.2 Robotics Experiments on Developmental Training of Complex Actions  209 9.3 Summary  216 10.3 Objective Science and Subjective Experience  251 11. and Postdiction  230 10.5 Summary  196 9.1 A Mirror Neuron Model: RNNPB  8 177 8.1 A Dynamic Account of Spontaneous Behaviors  1 219 10.4 Binding Language and Action  190 8.  vii Contents vii 8.4 Future Directions  255 11. Mirroring Action Generation and Recognition with Articulating Sensory–​Motor Flow  175 . Free Will for Action and Conscious Awareness  219 0.5 Summary  262 Glossary for Abbreviations  269 References  271 Index  289 .1 1 Compositionality in the Cognitive Mind  243 11.3 Imitating Others by Reading Their Mental States  182 8. Conclusions  243 1.2 Embedding Multiple Behaviors in Distributed Representation  180 8.1 Self-​Organization of Functional Hierarchy in Multiple Timescales  203 9. Consciousness. Development of Functional Hierarchy for Action  199 9.2 Free Will.3 Summary  239 11. viii . ix . and hybrid architectures. Tani’s work has explored some of the deep issues in embodied cog- nition. It is work that is in the spirit of Newell and Simon’s (1975) theory of empirical exploration of computer science topics and their work on generation of behavior. Ritter This book describes the background and results from Jun Tani’s work of over a decade of building robots that think and learn through interaction with the world. so I am pleased to see it in the Oxford Series on Cognitive Models and Architectures. consciousness and free will. but can instead be composable neuro-​dynamic structures arising through iterative learning of perceptual experience with the physical world. At the same time. subsymbolic. and how more complex behavior can be created or how it arises through more simple aspects. These les- sons include insights about the role of interaction with the environment. about how interaction with the environment happens. this work extends the physical symbol hypoth- esis in a very useful way by suggesting by example that the symbols of human cognition need not be discrete symbols manually fed into com- puters (which we have often done in symbolic cognitive architectures).  ix Foreword Frank E. and also takes Newell and Simon’s and Feynman’s motto of understanding through generation of behavior seri- ously. It has numerous useful and deep lessons for modelers developing and using symbolic. and lessons about how to build neural net architectures to drive behavior in robots. what this means for representation and learning. and for modeling and understanding interaction with an external world. a mental trace of what lower levels should do or are doing. The second part of the book also presents several systems used to explore these ideas. a type of self-​reflexive mental model. This work argues that behavior is not always programmed or extant in a system. attractors. His work provides another way of representing and generating behav- ior. This review also reminds us of areas that current symbolic models have been uninformed by—​I don’t think that these topics have been so much ignored as much as put on a list for later work. and. Using these concepts in existing architectures and models will provide new insights . imitation). or explanations of what they have done based on predictions of the agent’s own behavior. which Tani already uses it for. this could be a useful additional way to measure fit of a model to behavior. and the role of time in thinking). but that it can or often should arise in systems attempting to achieve homeostasis—​ that there are positions of stability in a mental representation (including modeling others. but they probably exist in them. These components are reviewed in the first part of the book. Perhaps more importantly. The book also notes that state space attractors can be a useful concept in understanding cognition. The simple ideas of evolution of knowledge. includ- ing some of the philosophical foundations in this area (including the symbol grounding problem. I  would add. These aspects are becoming more timely. It argues for a role of hierarchy in modeling cognition. The review chapters make this book particularly useful as an advanced textbook. This new way to examine behavior in architectures has provided insights already about learning and interaction and consciousness. Lessons from this book could and should change how we see all kinds of cognitive architectures. as Tani’s work shows they can be. and that differences in knowledge between the levels can give rise to effects that might be seen to be a type of con- sciousness. feedback. Many of these concepts have not yet been noticed in symbolic architectures. This way emphasizes the dynamic behavior of systems rather than the data structures used in more traditional approaches. and further concepts provide food for thought for all systems that generate behavior. in the second half of the book (Chapters 6 to 11) Tani describes lessons from his own work. These results sug- gest that more models should model homeostasis and include more goals and knowledge about how to achieve it. phenomenology.x x Foreword The book starts with a review of the foundations of this work.   xi Foreword xi into how compositional thoughts and actions can be generated without facing the notorious problems of the symbol grounding problem or. but that the mismatch could lead to a kind of postdiction in which intention becomes consciously aware after action. In his work about layers of representation. the mind–​body problem. Not only that higher levels could follow and not lead lower levels. . An interpre- tation of the higher levels trying to follow or predict the lower levels provides a potential computational description and explanation of some forms of consciousness and free will. and indeed should look for it. but also follow them. ulti- mately. their environments. We might see this elsewhere as other architectures. and their interaction with the environment become more complex. I hope you find the book as useful and suggestive of new areas of work and new aspects of behavior to consider for including in architectures as I have. he has seen that higher levels might not just lead the lower levels. I  found these concepts particu- larly intriguing. adjust- ing their own settings based on the lower levels’ behavior. xii . This core idea can provide a scaf- fold to account for the various fundamental aspects of the mind and cognition. and vice versa. We avoid such problems because the cogito embedded 1. cogito denotes a subject of cognizing or thinking. Allowing entangling interactions between the top-​down and bottom-​up processes means that the skills we need to generate complex actions. Cogito is from a Latin philosophical proposition by Rene Descartes “Cogito ergo sum.” which has been translated as “I think.  xiii Preface The mind is ever elusive. The essential proposal of the book is that the mind is comprised of emergent phenomena. which appear via intricate and often conflictive interactions between the top-​down subjective view for proactively acting on the external world and the bottom-​up recognition of the resultant perceptual reality. This book attempts to show a clear pic- ture of how the mind might work. knowledge. therefore I am. based on tangible experimental data I have obtained over the last two decades during my work to construct the minds of robots. and imagining its underlying mechanisms remains a constant challenge. The crucial argument here is that this cogito is free from the prob- lems inherent in Cartesian dualism. such as that of interaction and how a nonmaterial mind can cause anything in a material body and world.” Here. and concepts for representing the world and the linguistic competency we need to express our experiences can naturally develop—​and the cogito1 that allows this “compositional” yet fluid think- ing and action appears to be embedded in dynamic neural structures. xiii . Each of these statements—​my propos- als on the workings of the mind—​will be examined systematically by reviewing multidisciplinary discussions. I would like to commend and thank all members of my former labora- tory at RIKEN as well as of the current one in the Korean Advanced Institute of Science and Technology (KAIST) who. nonlinear dynamics. Heidegger’s philosophy. over the years. I have debts of gratitude to many people. Therefore. Gibsonian psychology. strange attractor from chaos theory. Takashi Ikegami has been one of the most inspiring. I  hope that a general audience and undergraduate students with a specific interest in this subject will enjoy reading on to the more technical aspects of the book that describe the neurorobotics experiments. cognitive sci- ence and cognitive robotics. and more. First of all. Actually. phenomenology. It is further proposed that even the nontrivial prob- lem of consciousness (what David Chalmers has called the hard problem of consciousness) and free will can become accessible by considering that consciousness is also an emergent phenomenon of matter arising inevita- bly from such conflictive interactions. mirror neurons. the other side pulls back elastically so that a point of compromise can be found in conflictive situations through iterative dynamics. largely from the fields of neuro- science. the cogito can interact physically with the external world: As one side pushes forward a little. rather than nonmatter composed of a discrete symbol system or logic. psychology. the book aims for a unique way of understanding the mind from rather an unordinary but inspiring combi- nation of ingredients such as humanoid robots. The matter here is alive and vivid in never-​ending trials by the cogito to comprehend an ever-​changing reality in an open-​ended world. and neural network modeling—​before exploring the subjects specifically in relation to the emergent phenomena which I believe constitute the mind. have contributed to the research described in this book. His . I thank Jeffrey White for plenty of insightful advice on this manuscript in regard to its contents. The book has been written with a multidisciplinary audience in mind. deep learning neural nets. as well as for editing in English and examining every page.xiv xiv Preface in the continuous state space of dynamic neural systems is also matter. I  am lucky to have many research friends with whom I can have in-​depth discussions about shared interests. nonlinear dynamics. By providing a brief introduction to each topic. neu- roscience and brain science. Each of the chapters start by presenting general concepts or tutorials on each discipline—​cognitive science. phenomenology. I must thank Masao Ito and Shun-​ichi Amari at RIKEN Brain Science Institute for their thoughtful advice to my research in general. Karl Friston has provided me thoughtful advice in the research of our shared interests on many occasions. I could not have completed this book without their patient and loving support. Toshitada Doi. I am grateful to Frank Ritter as the Oxford series editor on cognitive models and architectures who kindly provided me advice and suggestions from micro details to macro levels of this manuscript during its development. I  express my gratitude for Miki Sagara who prepared many figures. Yougo Tani. Ichiro Tsuda provided me deep thoughts about possible roles of chaos in the brain. Finally. and my mother.” . titled “Neuro-​Robotics Experiments with Large Scale Brain Networks. Tomoko. I wish to thank my Oxford University Press editor Joan Bossert for her cordial support and encouragement from the beginning. this work was partially supported by RIKEN BSI Research Fund (2010-2011) and the 2012 KAIST Settlement and Research of New Instructors Fund. Some additional resources such as robot videos can be found at https://​sites.google. I  admit that many of my research projects described in this book have been inspired by thoughtful discussions with him. The book could not have been completed in the pres- ent form without his input. He kindly read my early draft and sent it to Oxford University Press. Harumi. Michael Arbib offered insight into the concept of action primitives and mirror neuron model- ing. my biggest thanks go to my wife. I have been inspired by frequent discussions about developmental robot- ics with Minoru Asada and Yasuo Kuniyoshi. who pro- fessionally photographed the book’s cover image. Kentaro. And.  xv Preface xv stroke of genius and creative insights on the topics of life and the mind are irreplaceable. I would like to express my gratitude and appreciation to Masahiro Fujita.com/​site/​tanioupbook/ ​home. and Mario Tokoro of Sony Corporation who kindly provided me with the chance to start my neurorobotics studies more than two decades ago in an elevator hall in a Sony building. my son. Finally. The late Joseph Goguen and late Francisco Varela generously offered me much advice about the links between neurodynamics and phenomenology. who ignited my interest in science and engineering before he passed away in my childhood. This book is dedicated to the memory of my father. xvi .   1 Part I On the Mind . 2 . sometimes some novel images. my everyday thinking or action follows routines. my utterances are generated smoothly. when reaching for my mug of coffee on the table. and symbols. rules. How is this possible? Although it seems that many of our thoughts and actions are generated either consciously or unconsciously by utilizing knowl- edge or concepts in terms of images. or acts can be created.  3 1 Where Do We Begin with Mind? How do our minds work? Sometimes I notice that I act without much consciousness. or social conventions. However. thoughts. putting on a jacket. my actions as well as thoughts tend to shift freely from getting out the milk to looking out the window to thinking about whether to stay in for lunch today. When I’m doing something like making a cup of cof- fee. for example. I wonder how they are actually stored in our memories and how they can be manipu- lated in our minds. a somewhat phil- osophical question arises: How can I believe that this world really exists 3 . Is this spontaneous switching generated by my will? If so. habituation. I automatically combine words in the correct order and seldom consciously manipulate grammar when speaking. Nevertheless. I suddenly become conscious of my actions. How does this conscious- ness arise at such moments? In everyday conversation. How are they generated? Finally. like I fail to grasp the mug properly or the road to the station is closed due to roadwork. how is such will initiated in my mind in the first place? Mostly. or walking to the station for my daily com- mute. if something unexpected happens. however. synapse modulations. consciousness. it is and has always been the job of philosophers. It originated in a ship yard about 10 km away from the plant and inside the plant yard it was connected to a complex of looping networks equipped with various functional components such as automatic control valves. and tanks. I  noticed a strange coherence of sounds occurring across the yard. like those just described. Although the scientific liter- ature contains an abundance of detailed information about such bio- logical phenomena in the brain. and existence until something wonderful happened by chance to start me thinking about these things seriously. although the initial thunderous noise had faded. presumably because the pressure surge had propagated and was being reflected at various terminal ends in the piping network. immediately after valve shutdown. It is hard. it is still difficult to find satisfactory explanations about how the mind actually works. Several seconds later I  heard the same sound arising from various locations around the plant yard. One of the greatest of philosophers. asserted that “The mind is the part of the soul by which it knows and understands” (Aristotle. After some minutes. pumps. One day I traveled to a chemical plant site in an isolated area in northern Japan to examine a hydraulic system consisting of piping networks. Trans. But understanding the mind is not only the remit of scien- tists. with a diameter of more than 1. We know that the phenomena of the mind. The pipeline I saw there was huge. and more. too. originate in the brain: We often hear scientists saying that our minds are the prod- ucts of “entangled” activities of neurons firing. Twenty-​five years ago. Aristotle. I was a chemical plant engineer with no such thoughts about the brain. the loud knocking heard in a pipe caused by an abrupt pressure surge upstream of the valve. to link such metaphysical arguments to the actual biological reality of the brain. I was terrified by the thundering noise of the “water hammer” phenomenon.5 m and a total length of around 20 km. surge accumulators. I was conducting an emergency shutdown test of one of the huge main valves downstream in the pipeline when. This is because each piece of detailed knowledge about the biological brain cannot as yet be connected together well enough to produce a comprehensive picture of the whole. neuronal chemical reactions.4 4 On the Mind without my subjectively thinking about it? Does my subjective mind subsume the reality of the world or is it the other way around? The mind is one of the most curious and miraculous things. 1907). I heard “a pair” . and said in a synthesized voice. This coherence appeared and disappeared almost capri- ciously.” “This is a blackboard. In the next moment. a refrigerator means the smell of refreshing cool air when I open the door to get a beer on a long hot summer day. Surely the robot didn’t understand the meaning of a refrigerator or a chair in such a way. I had another epiphany several months later when.” While we all stood amazed at seeing the robot correctly naming the objects around us. Sometimes these patterns seemed to form in a combinatory way. I asked myself how the robot could know what a refrigera- tor meant.  5 Where Do We Begin with Mind? 5 of water hammers at different places. seeming to respond to each other periodically. The researchers there showed us a sophisticated mobile robot that could navigate around a room guided by a map preprogrammed into the robot’s computer. the plots showed some oscillatory patterns of pressure hikes appearing at certain points and tending to transform to other oscillatory patterns within several minutes. I  had the chance to visit a robotics research labora- tory. At that point I jumped on a bicycle to search for more water hammers around the plant yard even though it was already dusk. one of the most advanced of its kind in Japan. stopped in front of some objects. The meanings of these items to me. would materialize as the result of my own experiences with them. where its consciousness arose. To me. and I thought then that this might explain the spontaneous nature of the mind. In the next moment I started to think about build- ing my own robot. plotting the time history of the internal pressure at various points in the piping network. one that could have a subjective mind. Hearing this mysterious ensemble of roaring pipes in the darkness. “This is a refrigerator. arising again in other locations. Surely the meanings of various things in the world around us would be formed in our brains through the accumulation of our everyday experiences inter- acting with them.” and “This is a couch. I went back to the plant control room to examine the operation records. together with my fellow engineers. As I thought. however. I stopped and reflected to myself that this was not actually a mystery at all but complex transient phe- nomena involving physical systems. experience . as these items were nothing more to it than landmarks on a regis- tered computational map. I felt as if I was exploring inside a huge brain. During the demonstration the robot maneuvered around the room. with a set of patterns appearing in different combina- tions with other sets. however. such as the smell of cool air from the refrigerator or the feeling of my body sinking back into a soft chair as I sit down to drink my beer. neuroscience. but rather I  sus- pected that a good understanding. * * * This book asks how natural or artificial systems can host cognitive minds that are characterized by higher order cognitive capabilities such as compositionality on the one hand and also by autonomy in generating spontaneous interactions with the outer world either con- sciously or unconsciously. neural network modeling.6 6 On the Mind feelings. No single discipline could fully explain what the mind is or how it works. imagine things. It is like examining the emergence of nontrivial patterns of water hammer phenomena under the specific operational conditions applied in complex pipeline networks. Sometime later I went back to school. It was then it came to me that building robots while taking a mul- tidisciplinary approach could well produce a picture of the mind. . I also had some vague notion that a subjective mind should involve dynamic phenomena fluttering between the conscious and unconscious. under specific conditions and for various “cognitive” tasks. just as with the water hammers that had captured my imagination a few months earlier. and think about the world by interacting in it. if attainable. I sim- ply didn’t believe that one day a super genius like Einstein would come along and show us a complete picture of the mind. and philosophy. and the way the problems were approached by each discipline seemed too narrow to exchange ideas and views with other disciplines. where I studied many subjects related to the mind and cognition. robotics. The book draws answers from examination of synthetic neurorobotics experiments conducted by the author. The aim of synthetic neurorobotics studies is to examine experimentally the emergence of nontrivial mindlike phe- nomena through dynamic interactions. The underlying motivation of this study differs from that of conventional intelligent robotics studies that aim to design or program functions to generate intelligent actions. The current book presents the outcome of two decades of research under this motivation. including cognitive science. relational understanding between multiple disciplines. Each discipline seemed to have its own specific way of understanding the mind. would come from a mutual. enabling new findings and concepts in one domain to be explainable using differ- ent expressions in other disciplines. My intuition is that the key to unlocking all of the mysteries of the mind. which results in the modification of top-​down intention in order to minimize gaps or errors between our prior expectations and actual outcomes. is hidden in this as yet unex- plored phenomenon of circular causality and the structure within which it occurs. The synthetic robotics approach described in this book seeks to answer this fundamental question through the examination of actual experimental results from the viewpoints of various disciplines.  7 Where Do We Begin with Mind? 7 The synthetic neurorobotics studies described in this book have two foci. namely “Part I On the Mind” from ­c hapter  1 to c­ hapter  5 and “Part II Emergent Minds:  Findings from Robotics Experiments” from ­c hapter 6 to ­c hapter 11. One is to make use of dynamical systems perspectives to under- stand various intricate mechanisms characterizing cognitive minds. Our studies emphasize top-​down inten- tionality on the one hand. views. The dynamical systems approach has been known to be effective in articu- lating mechanisms underlying the development of various functional structures by applying the principles of self-​organization from physics (Nicolis & Prigogine. the . Moreover. Haken. including our experi- ences of consciousness as well as free will.” concepts. or linguistic thoughts. This book is organized into two parts. In Part I. and thoughts consolidated into structures through past experience are proactively projected onto the objective world. may develop by means of self-​organization in internal neurodynamic systems via the consolidative learning of experience. 1977. by which our own subjective images. This naturally brings us to the distinction between the subjective mind and the objective world. Structures and functions to mechanize higher order cognition. The crucial focus here is on the circular causality that emerges as the result of iterative interactions between the two processes of the top-​down subjective intention of act- ing on the objective world and the bottom-​up recognition of the objec- tive world with modification of the intention. such as for compositional manipu- lations of “symbols. guiding and accompany- ing our actions. 1983). The other focus of these neuro- robotics studies is on the embodiment of cognitive processes crucial to understanding the circular causality arising between body and environ- ment as aspects of mind extend beyond the brain. close examination of this structure might help us address the fundamental philosophical problem brought to the fore in mind/​body dualism: how the subjective mind and the objective world are related. Our studies also emphasize bottom-​up recognition of the perceptual reality on the other hand. including cognitive science. and robot- ics. psychology. this book traces my own journey in exploration of the fundamental nature of the mind. These in-​depth reviews will provide general readers with a good introduction to relevant disciplines and should help them to appreci- ate the many conflicting arguments about the mind and brain active therein. In the end. . and through analysis of their results comes out with some answers to fundamental questions about the nature of the mind. brain science. and in retracing this journey I hope to deliver an intuitively accessible account of how the mind works. neural network modeling.8 8 On the Mind book reviews how problems with cognitive minds have been explored in different research fields. phenomenol- ogy. Part II starts with new proposals for tackling these problems through neurorobotics experiments. However. Indeed. 2. the principle asserts that the meaning of a complex expression is determined by the mean- ings of its constituent expressions and the rules used to combine them (sentences are composed from sequences of words). its cen- tral notion that the whole can be decomposed into reusable parts (or primitives) is applicable to other faculties.  Composition and Recursion in Symbol Systems The essence of cognitivism is represented well by the principle of com- positionality (i. Let us begin our discussion of cognitivism by looking at the core ideas of cognitive science..e. such as action generation. and this approach has become successful as the speed and memory capacity of computers has grown exponen- tially.  9 2 Cognitivism One of the main forces having advanced the study of the mind over the last 50 years is cognitivism. but specifically that as expounded by Gareth Evans (1982) in regard to language. Michael Arbib (1981) in his motor schemata theory. which was 9 . the meaning of the whole is a function of the mean- ing of the parts). Cognitivism regards the mind as an exter- nally observable object that can be best articulated with symbol systems in computational metaphors. According to Evans.1. comprising arbitrary shapes of tokens (Harnad. FLN is thought to generate internal representations by utiliz- ing syntactic rules and mapping them to a sensory–​motor interface via the phonological system as well as to the conceptual–​intentional inter- face via the semantic system. Because these objects to be manipulated—​either by computers or in mental processes—​are symbols without any physical dimensions such as weight.” and the computational mechanics of the combinatorial operations of operands. 2002)  proposed that the human brain might host two distinct cognitive competencies:  the so-​called faculty of language in a narrow sense (FLN) and the faculty of language in a broad sense (FLB). length. is provided with recursive functionality for the tokens’ operations.10 10 On the Mind published not long before Evans’ work on language. or force. When such a symbol system. goal-​d irected actions can be decomposed into sequences of behav- ior primitives. and the computational mechanisms for recursion that allow for an infinite range of expressions from a finite set of elements. involves only recursion and is regarded as a uniquely human aspect of language. a conceptual-​intentional system. Chomsky. & Fitch. behavior primitives are sets of commonly used behavior pattern segments or motor programs that are put together to form streams of continuous sensory-​motor flow. FLB com- prises a sensory-​motor system. The chimps became able to count up to around five objects correctly. speed. Cognitive scientists have found a good analogy between the compositionality of mental pro- cesses. famous for his revolutionary ideas on generative grammar in linguistics. FLN. Here. proposed that com- plex. their manipulation processes are considered to be cost free in terms of time and energy con- sumption. 1992). Chimps have become able to count the number of objects on a table by indicating a corresponding panel representing the correct number of objects on the table by association. on the other hand. but one or two errors creep in for more than five . Chomsky and colleagues admit that some animals other than humans can exhibit certain recursion-​like behaviors with training. In both cases we have con- crete objects—​ symbols—​ and distinct procedures for manipulating them in our brains. Noam Chomsky. has advocated that recursion is a uniquely human cognitive competency. it achieves compositionality with an infinite range of expressions. like combining the meanings of words into those of sentences or combining the images of behavior primitives into those of goal-​d irected actions “at the back of our mind. Chomsky and colleagues (Hauser. Chomsky and colleagues’ crucial argument is that the core aspect of recursion is not a matter of what has been learned or developed over a lifetime but what has been implemented as an innate function in the faculty of language in a narrow sense (FLN). Similarly. some individuals could nest only two different sizes of cups whereas others could pair three by employing a subassembly strategy. which in turn can be “nested” or “seriated” into larger cups. In counting numbers. Another example of recursion-​like behavior in animals is cup nesting. and in sentence generation the recursive substitution of one of the context-​ free grammar rules for each variable could generate sentences of infinite length after starting with the symbol “S” (see Figure 2. What then is the core mechanism of FLN? It seems to be a recursive call of logical rules. 3. However. although some animals can learn to perform recursion-​like behaviors. Chomsky and colleagues thus speculated that the human brain might be uniquely endowed with the FLN component that enables infinite recur- sion in the generation of various cognitive behaviors including language. In their view. When observing chimps and bonobos cup nesting. the more inaccurate at counting the chimps become.1 for an illustra- tive example). in the recursive structure of sentences. … .  11 Cognitivism 11 objects: The more objects to count. the number of nestings never reliably went beyond three. infinity as the “add one” rule is called at each recursion. clauses nest inside of other clauses. Similar limitations in cup nesting performance have been observed in parrots (Pepperberg & Shive. the logical rule of “add one to the currently memorized number” is recursively called:  Starting with the currently memorized number set to 0. These observations of animals’ object counting and nesting cup behaviors suggest that. what is to be learned or developed are the interfaces from this core aspect of recursion . Cup nesting can be performed infinitely when the logical rule of “put the next smallest cup in the current nesting cup” is recursively called. that is. Johnson-​P ynn and colleagues (1999) found that performance differed by species as well as among individuals. 2. nesting a small cup in a medium size cup as a subassembly and then nesting them in a large cup. 2004). the depth of recursion is quite limited particu- larly when contrasted with humans in whom almost an infinite depth of recursion is possible as long as time and physical conditions allow. it is increased to 1. 2001) and the degu. a small rat-​size rodent (Tokimoto & Okanoya. a task in which each cup varies in size so that the small- est cup fits into the second smallest. are symbols actually manipu- lated recursively somewhere in our heads when counting numbers or generating/​recognizing sentences? If there are fewer than six objects on a table. depending on the nature of the substituting rules (e. a limitation in working memory size in FLB in remembering currently generated word sequences) or of physical time constraints that hamper perform- ing infinite recursions in FLN.1. Chomsky and colleagues. if there are more than six objects. we may start to count them one by one on our fingers. see this not as a problem of FLN itself but as a problem of external constraints (e.  On the left is a context-​f ree grammar (CFG) consisting of a set of rules and on the right is an example sentence that can be generated by recursive substitutions of the rules with the starting symbol “S” allocated to the top of the parsing tree. but by automatically and subconsciously .. however. repeated substitutions of R2: NP→A NP). Such a view is contentious though. ability to the sensory–​motor systems or semantic systems in the faculty of language in a broad sense (FLB). the number would be grasped analogically from visual pat- terns. In our everyday conversations we generally talk without much concern for spoken grammar:  Our colloquialisms seem to be generated not by consciously combining individual words following grammatical rules. it is not realistic to assume that we humans perform infinite recursions in everyday life. Second.12 12 On the Mind Sentence generation Context-free grammar S R1: S → NP VP R1 R2: NP → (A NP)/N NP VP R3: VP → V NP R2 R3 A NP V NP R4: A → Small R4 R2 R6 R2 R5: N → dogs/cats Small N like N R6: V → like R5 R5 dogs cats Figure 2.g.g.. They assert that the unique exis- tence of this core recursive aspect of FLN is an innate component that positions human cognitive capability at the top of the hierarchy of living systems. even those with infinite length. Note that the same CFG can generate different sentences. First. We can neither count infinitely nor generate/​ recognize infinite-​ length sen- tences. we some- times find ourselves consciously dealing with grammar in our search for appropriate word sequences. The General Problem Solver (GPS) (Newell & Simon. For example. 1972. which has made a significant impact on the subsequent direction of artificial intelligence research. with its strong conviction that the core aspect of cognition should reside in symbol representation and a manip- ulation framework. when needing to write complex embed- ded sentences such as those often seen in formal documents. the notion of there being infinite levels of recursion in FLN might apply only rarely to human cognition. In everyday life. Cognitivism embraces the former possibility. if we are to assume that symbols play a central role in cognition. Simon is such a typical cognitive model. It requires some level of manipulation of internal knowledge about the world. this is merely to describe the meaning of a symbol by way of other symbols.  Some Cognitive Models This section looks at some cognitive models that have been developed to solve general cognitive tasks by utilizing the aforementioned symbol- ist framework. 1990) that was developed by Allen Newell and Herbert A. How is such process- ing done? One possibility might be to use the core recursive component of calling logical rules in FLN under the limitation of finite levels of recursions.” However.2. Newell. and I’m not sure how my everyday experience with apples could be represented in this form. Thus. a typical artificial intelligence system may repre- sent an “apple” with its features “color-​is-​R ED” and “shape-​is-​SPHERE. yet does not involve infinite complexity. Many cognitive behaviors in everyday life do still of course require some level of manipulation that involves composition or recursion of information. . how would symbols comprising arbitrary shapes of tokens convey the richness of meaning and context we see in the real world? For example. But. However. generating goal-​d irected action plans by com- bining behavior primitives into sequences cannot be accounted for by the simple involuntary action of mapping sensory inputs to motor out- puts. it seems unlikely that an infinite range of expressions would be used. 2.  13 Cognitivism 13 combining phrases. Another possibility might be to assume subrecursive func- tions embedded in analogical processes rather than logical operations in FLB that can mimic recursive operations for finite levels. In the Table 2. the initial state.” “has ball.” “not hungry” “hungry” .” door to middle “at door” room.” “on floor.” “has bananas.” “middle “at door” room” room” Rule 3 “walk from door “at door.” “on middle room.” “chair at door”].1. and the transition rules are defined. Let us consider the so-​called monkey– ​banana problem in which the goal of the monkey is to become not hungry by eating a banana. By considering that the goal is [“not hungry”] and the start state is [“at door.” “hungry. a list of “add” states and a list of “delete” states.” “chair at middle “chair at door. In solving a problem. the problem space in terms of the goal to be achieved.1. as is shown later. 1983)  and Soar (Laird et al.14 14 On the Mind Numerous systems such as Act-​R (Anderson. Therefore.” “at chair” room. Each transition rule is specified by an action operator associated with a list of precondition states.  Example Rules in GPS Rule # Action Precondition Add Delete Rule 1 “climb on chair” “chair at middle “at bananas. this precondition state of [“has bananas”] becomes the subgoal to be achieved in the next step.” “on “at middle room. although it has a crucial prob- lem.” “has bananas” “empty handed” “empty handed” Rule 5 “eat bananas” “has bananas” “empty handed. The GPS provides a core set of operations that can be used to solve cognitive problems in various task domains. the corresponding “add” states and “delete” states are added to and deleted from the precondition states.” “at middle room” “at door” to middle room” “on floor” Rule 4 “grasp bananas” “at bananas. By following a means-​end-​analysis approach.” floor” “on floor” Rule 2 “push chair from “chair at door. the goal to be achieved is divided into subgoals and GPS attempts to solve each of those. 1987) use this rule-​based approach. it can be seen that the goal state [“not hungry”] can be achieved by apply- ing an action of “eat bananas” in Rule 5 if the precondition state of [“has bananas”] is satisfied.. A  rule actually specifies a possible state transition from the precondition state to the consequent state after applying the action. The rules defined for GPS can be as shown in Table 2. After an action is applied.   15 Cognitivism 15 same manner. chunking. At the same time. who has led the development of Soar for more than two decades. Actually. this chunked action sequence is recalled rather than deliberating over and synthesizing it again. this action sequence is memorized as a chunk (a learned rule) in long-​term memory. however. the monkey may learn an action sequence of “grasp bananas” and “eat bananas” as an effective chunk for solving a current “hungry” problem. Later sections of this book explore how chunks can be structured out of continuous sensory–​motor flow experiences. The idea of chunking has attracted significant attention in cognitive psychology. Chunking is involved in the conversion of an experience of an action sequence into long-​term memory. in the case of the monkey–​banana problem. . I myself had been largely influenced by this idea after I learned about it in an artificial intelligence course given by John Laird. which cognitive models built on symbolist frameworks inevi- tably encounter. and chess. The architecture of GPS is quite general in the sense that it has been applied to a variety of different task domains including proving theo- rems in logic or geometry. which can be achieved again by applying another action of [“climb on chair”]. 1987)  developed a new cognitive model. Soar. I could not arrive at full agreement with the treatment of chunking in Soar because the basic elements to be chun- ked are symbols rather than continuous patterns even at the lowest per- ceptual level. I speculated that the mechanism of chunking should be considered at the level of continuous perceptual flow rather than symbol sequences in which each symbol already stands as an isolated segment within the flow. When the same subgoal appears again. word puzzles. and the goal state can be achieved from the start state by applying the resulting action sequence. Repetitions of backward transition from a particular subgoal to its sub-​subgoal by searching for an adequate action enabling the transi- tion can result in generation of a chain of actions. the subgoal [“has bananas”] can be achieved by applying an action of [“grasp bananas”] with the precondition of [“at bananas”]. For example.. and may retain this chunk because “hungry” may appear as a problem again in the future. the next section introduces the so-​called symbol grounding problem. Of particular interest is its primary learning mechanism. When a particular action sequence is found to be effective to achieve a par- ticular subgoal. by further extending GPS. however. Allen Newell and his colleagues (Laird et  al. First. Harnad argued that meaning. as an abstract model of cognitive systems. a hybrid sys- tem consisting of a symbol system in the upper level and a nonsymbolic pattern processing system in the lower level. This idea of a hybrid system reminds me also of Cartesian dualism. A typical mobile robot. This concept of a hybrid system has similarities to that of FLN and FLB advocated by Chomsky and colleagues in the sense that it assumes a core aspect of human cognition in terms of logical symbol systems.3. According to Descartes the mind is a thinking thing that is nonmaterial whereas the body is nonthinking matter. our minds depend on our physical condition and the freshness of the mind affects the swiftness of our every move. reviewing my own work on the subject (Tani.  The Symbol Grounding Problem The symbol grounding problem as conceptualized by Steven Harnad (1990) is based on his assertion that the meanings of symbols should originate from a nonsymbolic substrate like sensory-​– ​motor patterns and as such.16 16 On the Mind 2. and vice versa. however. he proposed. as such representation is now grounded in the world. Descartes showed some concern about this “problem of interactionism.” asking how a nonmaterial mind can cause anything in a material body. The nonsymbolic pattern processing system functions as the interface between sensory–​motor reality and abstract symbolic representation by categorizing continuous sensory–​motor patterns into sets of discrete symbols. Obviously. and the body to sensory–​motor processes that are defined in physical space. in the hybrid system would no longer be parasitic on its symbol representation but would become intrinsic to the whole system operation. and the two are distinct. symbols are grounded bottom up. 1998). seem to consider—​ rather optimistically I think—​that some “nice” interfaces would enable interactions between the two opposite poles of nonmatter and matter. or semantics. To give shape to this thought. The nonmaterial mind may correspond to FLN or symbol systems that are defined in a nonphysical discrete space. The crucial question here is how these two completely distinct existences that do not share the same metric space can interact with each other. Cognitive scientists in modern times. which is equipped with simple range sensors. may travel around an office environment while taking the range reading that . which can support up to an infinite range of expressions. and peripheries as the interface to a sensory–​motor or semantic system that may not be involved in composition or recursion in depth. Let’s consider the problem by examining a problem in robot naviga- tion as an example. Redrawn from Tani (1998).  17 Cognitivism 17 provides an estimate of geometrical shapes in the surrounding environ- ment at each time step. .2. This internal map consists of nodes representing position states of the robot associated with encountered landmark types and of arcs rep- resenting transitions between them associated with actions such as turn- ing to right/​left and going straight.2. which consists of a finite number of discrete states and their state transition rules. The upper level constructs a chain representation of landmark types by observing sequential outputs of the categorizer while the robot explores the envi- ronment. a corner. or a room entrance. it becomes able to predict the next sensation of landmarks on its travels by looking at the next state transition in the FSM.  Landmark-​based navigation of a robot using hybrid-​t ype architecture consisting of a finite state machine and a categorizer. The continuous flow of the range image pattern is categorized into one of several predefined landmark types such as a straight corridor. C Straight T C Right C T C FSM “T-Branch” Straight categorizer Right robot and its environment t sensory pattern Figure 2. An illustrative description is shown in Figure 2. a T-​branch. When the actual percep- tion of the landmark type matches the prediction. It is noted that the rule representation in GPS can be con- verted into this FSM representation by considering that each rule descrip- tion in GPS can be expanded into two adjacent nodes connected by an ark in FSM. Once the robot acquires the internal map of its environment. the robot proceeds to the prediction of the next landmark to be encountered. This representation takes exactly the same form as a symbolic representation known as a finite state machine (FSM). In a larger sense. Context. Context originally means discourse that surrounds a lan- guage unit and that helps to determine its interpretation. they inevitably encounter inconsistencies between the two pathways of top-​down expectation and bottom-​up reality.18 18 On the Mind Problems occur when this matching process fails. This problem cannot be resolved simply by implementing certain interfaces between the two systems because the two simply do not share the same metric space enabling smooth. too. Although details of his mathematical formulas are not introduced here. Wittgenstein once said: “Whereof one cannot speak. The robot becomes lost because the operation of the FSM halts upon receiving an illegiti- mate symbol/​ landmark type. and direct interactions. Not only in philosophy. is an example. halting the system’s operations. Spencer-​ Brown (1969) highlighted a paradox in his attempts to explicitly specify context in his formulation of the calculus of indica- tions. 2. there is always some- thing that cannot be expressed explicitly. If one side pushes forward a little. or background. Context Another concern is how well symbol systems can represent the real- ity of the world. This cooperation entails iterative interactions between the two sides through which opti- mal matching between them is sought dynamically. It is considered that both levels are dually responsible for any inconsistency and that they should resolve any conflict through cooperative processes.” meaning that language as a formal symbol system for fully expressing philosophical ideas has its limitations. his statement could be interpreted to mean that indexing the . but in everyday life. the other side should pull back elastically so that a point of compromise can be found through iterative dynamic interac- tions. it also means the surroundings that specify the meaning or exis- tence of an event. This is my concern about the symbol grounding problem. The problem is how such inconsistencies can be treated internally without causing a fatal error. thereof one must be silent. When systems involve bottom-​up and top-​down pathways. dense. The problem here is that the symbol systems defined in a discrete space appear to be too solid to afford such dynamic interactions with the sensory–​motor system.4. We live and act in the world surrounded or supported by context. namely that a bottle-​like shape was seen immediately after you opened the refrigerator door. and incomplete for us at best. mood. my next questions would be how can symbols compris- ing arbitrary shapes of tokens interact with sensory–​motor reality and how can they access matters involving context. Also. which means that the bottle is chilled. we can com- bine the acquired rules to create new images. yet you can still reach for the bottle of beer to drink it! Although FLN may have the capability for infinite regression. every observation entails a symbol. uncertain. Summary We humans definitely have internal images about our surrounding world. Accounting for this aspect. it is hard to believe that our minds actually engage in such infinite computations. utterances. This situation can be disambiguated by specifying its immediate background (context). and thoughts. Because indexing the background requires further indexing of the back- ground of the background. My question. as evidenced by the fact that we can acquire language skills involving grammar. It is presumably hard to differentiate between doing something . or tacit knowl- edge that are considered to be difficult to deal with by formal symbol systems? It is also difficult to represent the state of consciousness with them.  19 Cognitivism 19 current situation requires the indexing of its background or context. Let’s imagine you see a bottle-​like shape. cognitivists tend to assume that symbols exist to be manipulated in our heads. which is always implicit. is what is the reality of those symbols we suppose to be in our heads? Is symbol representation and manipulation an operational principle in the cogni- tive mind? If so. Spencer-​Brown wrote that. where the cross operation denotes indexing of the background. an unwritten cross. though. Further background information that the refrigerator was opened after you went back to your apartment after a long day at work would mean that what you see now is a bottle of chilled beer waiting to be drunk. in this aspect. There is no logical way to terminate this regression. How can a formal symbol system represent such a situation? 2.5. We can extract regularities from our experiences and observa- tions both consciously and unconsciously. the operation of indexing situations ends up as an infinite regression. this book proposes an abrupt transition from the aforementioned conventional symbolist framework. Such dynamic activ- ity of matter. . The objective of phenomenology is not only to investigate the problem of minds but also to search for how the problems themselves can be constituted from the introspective view. If we attempt to model or reconstruct mind. it should be essential to reconstruct not only rational thinking aspects but also the feelings that accompany our daily experiences such as consciousness as the vivid feeling of qualia characterizing various sensations. what would be a viable solution? Indeed. A  crucial argument would be that such cognitive minds could be naturally situ- ated to the physical world because these two share the same metric space for interaction. But if sym- bol systems cannot deal with such matters. The main pro- posal is to consider that what we have in our brains as “symbol” is not just arbitrary shape of token but dynamic activity of physical matter embedded in continuous spatio-​temporal space. Readers will find that the disciplinary of phenomenology is quite sympathetic to the aforementioned dynamic system view. The next chapter addresses this very problem from the standpoint of a different discipline. that of phenomenology. might enable compositional but vivid and contextual thinking and imaging in our brains. adequately developed.20 20 On the Mind consciously and unconsciously in the processes of merely manipulating symbols by following logic. through direct perception or pure experience. As this chapter shows. some of which are quite analogous to Husserl’s phenomenology. he also pro- vided numerous essential philosophical ideas about the mind. influenced by Buddhist meditation. the question of how the world can be con- stituted in our subjective reflection might be analogous to the question of how the knowledge of the world can be represented in cognitive sci- ence studies. Kitaro Nishida (1990) developed his original thinking. through which the analysis of the natural world is based purely on the conscious experiences of individuals. In Japan. which turned out to include ideas with some affinity to those of Husserl and James.  21 3 Phenomenology Phenomenology originated in Europe at the beginning of the 20th cen- tury with Edmund Husserl’s study of so-​called phenomenological reduc- tion. which has 21 . Here. Husserl’s study subsequently evolved and was extended by the existentialism of Martin Heidegger and the embodiment of Maurice Merleau-​Ponty and others. by suspending our ordinal assumption that the world exists as a physical fact from the outset. who was born 17 years earlier than Husserl in the United States. Phenomenology asks us to contemplate how the world can exist for us and how such a belief can be constituted from our experiences. Although James is best known as the founder of modern psychology. however. Phenomenology. focuses more on phenomena themselves. We should also not forget to mention William James. the discipline also explores the being of cogito (how cognition arises) in the higher level by examining how it can be developed purely through the accumulation of perceptual experiences.3) . Thus. Kitaro Nishida introduced a simi- lar idea in terms of pure experience. Husserl considered that an examination of such direct experience could serve as a starting point to explore phenomena. pure experience is identical with direct experience (Nishida. 3. It is said that Mach drew the picture to represent what he sees with his left eye while closing his right one. the tip of his nose appears to the right of the frame with his eye socket curving upwards. but also to the judgment of what the color or sound might be. It is said that Husserl noticed the importance of direct experience when coming across Mach’s perspective (Figure 3. a notable Japanese philosopher. the moment of seeing a color or hearing a sound is prior not only to the thought that the color or sound is the activity of an external object or that one is sensing it. However. (Nishida.  Direct Experience Let us begin by examining what direct experience means in phenomenol- ogy. For Nishida. this should represent the direct experience that we then reconstruct in our minds. phenom- enology asks how cognition is constituted from direct perception. a line of questioning deeply related to the later discussions on how robotic agents can develop views or recognition of the world from their own sensory–​motor experiences. For example. a rose exists in our subjectivity as a conscious phenomenon of a par- ticular smell or a particular visual shape. This discipline then focuses purely on phenom- ena and questions the existence of the world from such a viewpoint. 1998). pure experience is not describable by language but is transcended: When one directly experiences one’s own state of consciousness. 1990.1. writing that: For example. In this regard. Although we usually do not notice this sort of perspective. p. but not by our knowledge of its objective existence. 1990. Around the same time. there is not yet a subject or an object… .1) (T. p.3).22 22 On the Mind not yet been articulated either by conception or language. Tani. From this perspective. the former utterance is considered to express pure experience in which subject and object are not yet separated by any articulation in the cogito.  The Subjective Mind and Objective World We might ask. In this interpreta- tion.  23 Phenomenology 23 Figure 3. explains this by analyzing the example utterance. who is known for his studies on Nishida’s philosophy.” the explication of “I” as the subject conveys a subtle expression of subjective experience at the moment of hearing.2. Source: Wikimedia Commons.” If it is said instead as “I hear the temple bell ringing. “The temple bell is ringing.1. Here. however. Is our experience of perception the same as that of . what exactly does this phrase “there is not yet a subject or an object” mean? Shizuteru Ueda (1994). how much the phenomena of experience depend on direct perception. 3. This analysis is analogous to what Husserl recognized from Mach’s perspective.  Ernst Mach’s drawing. At this point. a square could “appear” with unequal angles in various real situations. as an example. when it should have equal right angles in the ideal: in such a case. which emphasizes subjective reflection and representation of the world.  Husserl’s ideas on the structural relationship between “appearance” and “that which appears” in perceiving a square. we should forget about the actual existence of this square in the physical world because this object should. in Husserl’s sense. he uses the example of perceiving a square. In other words. despite them having slightly unequal angles. a parallelogram or trapezoid is the “appear- ance” and the square is “that which appears” as the result of percep- tion.2. we usually perceive them to be squares with equal right angles. In looking at squarelike shapes in everyday life. Let’s look then at how this issue of the subjective and the objective has been addressed by different phenomenological ideas. as shown in Figure 3. exist only through idealization. and on the other we have cognitivism.24 24 On the Mind infants in the sense that any knowledge or conception in the cogito does not affect them at all? In answer. In Husserl’s (2002) analysis of the structural relationship between what he calls appearance and that which appears in perceiving an object. they exist regardless of their actual being. When things are constituted in our minds. But how did these conflicting poles of the subjective mind and the objective world appear? Perhaps they existed as one entity originally and later split off from each other. This approach that puts aside correspondence to actual being is called that which appears Square is perceived (appearance1) (appearance2) (appearance3) Parallelogram Trapezoid Parallelogram Figure 3. Whether things exist or not is just a subjective matter rather than an objective one. we have sensationalism on one side. which emphasizes direct experiences from the objective world. .2. Although understanding such constitu- tional aspects of the polarity (i. He regards the body as ambiguous. phenomenology does not seek to take that direction and instead attempts to explore how the apparent polarity of. Incidentally. being positioned between the subjective mental world and the objective phys- ical world. on the other hand. the cogito and perception. Nishida (1990) also considers that the subject and object should be one unified existence rather than taken originally as independent phenomena. and mind and material. or suspension of belief. For example. interesting assumptions have been made about there being some sort of immanence enabling self-​development of such structures.  25 Phenomenology 25 epoché. the phenomena of experience cannot be accounted for only by direct experience at the level of perception. as explained in detail later. He. However. And through his approach to the problem of being he turned out to be successful in showing how subjectivity and objectivity can appear.. readers might speculate that the level of cogito and the level of perception are treated as separate entities in phenomenology. This intentional process of constituting representation from direct experience actually entails consciousness. how the polarity developed) contin- ues to be a subject of debate in phenomenology. could have appeared from a single unified entity in the beginning. subjectivity and objectivity. Merleau-​Ponty (1968) pro- fesses that this iteration of unification and division would take place in the medium of our bodies. it can be said that the phenomena of experiences stand on the duality of these two levels. but it must also be accounted for by conscious representation at the level of cogito. from the preceding text. Husserl considers how the cogito level of dealing with temporal structure submerged in a stream of experience could emerge from the direct perceptual level. . Therefore. as he considers that the two poles of the subjective mind and the objective material actually meet and intermin- gle with each other there. argues that the unified existence could have internal contradictions that lead to bifurcation or the division of the unity into the subject and object that we usually grasp. Husserl considers that direct experi- ence has intentionality toward representation. for example. devoted himself to exploring a more fundamental problem of being by working on what it means to be human rather than splitting the problem into that of subject and object. Heidegger. He suggests that the phenomenological entity simply continues to develop by repeat- ing these unification and division processes. Ultimately.e. however. every experience is implicit and yet must be articulated. This idea seems born from his thinking on the structural relationship between “appearance” and “that which appears” mentioned earlier in this chapter.26 26 On the Mind What follows examines the philosophical arguments concern- ing the subjective mind and the objective world in more depth. 1964). along with related discussions that include time perception as propounded by Husserl. as is detailed later. At the preem- pirical level. It should be noted that “time” discussed here is not physical time having dimensions of seconds. His famous explanatory example is about hearing a continuous melody such as “do-​re-​mi. the continuous flow of experience becomes articulated into consciously accessible events by its develop- ment though these phenomenological levels. being-​in-​the-​world set forth by Heidegger. Let’s begin by looking closely at each of these. Husserl presumed that time consists of two levels: so-​called preempirical time at a deep level and objective time at a surface level. the world should consist of objects that the subject can con- sciously meditate on or describe. and hours but rather time perceived subjectively without objective measures. However. According to him. Analyzing how a continuous flow of experience can be articulated or segmented into describable objects or events brought him to the problem of time perception.” When we hear the “re” note. Husserl asks how we perceive temporal structure in our experiences (Husserl. minutes. embodiment by Merleau-​Ponty.3. In considering the problem. starting with Husserl’s conception of the problem of time perception. and the stream of consciousness by James. but there is some sort of passive intention toward the flow of experience which he refers to as retention and protention. we would still perceive a lingering impression of . The problem of time perception is a core issue in this book because both humans and robots that generate and recognize actions have to manage continuous flows of perception by articulating them (via segmentation and chunking). 3.  Time Perception: How Can the Flow of Subjective Experiences Be Objectified? To Husserl. he noticed that our direct experiences do not originate with forms of such consciously represent- able objects but arise from a continuity of experience in time that exists as pure experience. primary impression. There is a fundamental .” By means of con- sciously unifying immediate pastness in a recall with presentness in the next recall in the retention train.  27 Phenomenology 27 “do” and at the same time we would anticipate hearing the next note of “mi. slip into the distant past but still be retrieved through conscious memory. The present appearance of “re” is called the primary impression. Yet.” The former refers to retention and the latter protention. that is. in the sense that the subject of this utterance is not yet consciously reflected. in terms of retention and protention: Retention of “do” and protention of “mi” are included in the primary impression of hearing “re. a sense of objective time emerges as a natural consequence of organizing each appearance into one consistent linear sequence.” This would be true especially when we hear “do-​re-​mi” as the chunk of a familiar melody rather than as a sequence consisting of independent notes.” and so on in order back to “do. These three terms of retention. and the immediate future. experienced just as an impression. and we can also recall hearing the same “so” that retains the appearance of “fa. however. considered that the subjective experience of nowness is extended to include the fringes of the experi- enced sense of both the past and the future. Having now understood Husserl’s notion of nowness in terms of retention and protention. Nowness as experienced in this situation might be taken to correspond with the present point of hearing “re” with no duration and nothing beyond that. respectively. we can recall hearing the final “la” that also retains the appearance of “so” by means of retention.” In this situation. as described next. The situation is similar to that of the utterance “The temple bell is ringing” mentioned ear- lier. as Francisco Varela (1999) once asked in the context of neurophenomenology? Conscious memory of the past actually appears at the level of objective time. and protention are used to des- ignate the experienced sense of the immediate past. the question arises: Where is nowness bounded? Husserl seems to think that the immediate past does not belong to a representational conscious mem- ory but merely to an impression. Let’s consider the problem of nowness in the “do-​ re-​ mi” example. let’s consider remembering hearing the slightly longer sequence of notes in “do-​re-​mi-​fa-​so-​la. how could the immediate past. In other words. Husserl. the present. This time. They are a part of automatic processes and as such cannot be monitored consciously. objective time is constituted when the original experience of continuous flow (in this case the melody) is artic- ulated into a sequence of objectified events (the notes) by means of con- sciously recalling and unifying each appearance. primary impression. The former is a present. In his later years. an absolutely given dynamic which is nonreducible. his motiva- tion toward a logically manipulable ideal representation of the world via reflection seems to me problematic in that it has exactly the same problem as the symbol grounding problem in cognitive models. the latter may lack the pureness or vividness of the original experience. the original pure experience is objectified and simulataneously the subjectivity or ego of this objecti- fying process emerges. Husserl proposed two types of intentionality. each appearance has its own duration. and objective time arises from there. Dreyfus (Dreyfus & Dreyfus. argues that the main computational scheme . In the process of interweaving these two intentionalities (double intentional- ity) into the unitary flow of consciousness. Therefore. However. but only after the original experience is retained. Here. yet may fit well with Husserl’s goal that pure experience can be ultimately represented as logical forms dealing with discrete objects and events. This alternating flow and stagnation is primordial. who is well known for his criticism of arti- ficial intelligence research. neither retention nor protention has yet appeared—​only flow exists. Husserl introduced an analysis at an even deeper level.28 28 On the Mind difference between an impression that will sink into the horizon in preempirical time and the same past which is represented in objective time. longitudinal intentionality affords an immanence of time structures (from preempirical time to objective time) by means of conscious recall of retained events in the retention train. whereas the latter can be constituted as consciously represented or manipulable objects. the absolute flow level. this flow is not homogeneous. Husserl’s persistent drive to reduce the ideas and knowl- edge of man to direct experiences is admirable. char- acterizing the uniqueness of the absolute flow of consciousness and set- ting it apart from consciousness as developed elsewhere. this intentionality might be considered to be reten- tion of retention itself (a reflective awareness of this experience). Consequently. Tohru Tani (1998) interpreted this aspect by saying that consciousness flows as well as stagnates. However. and protention in preempirical time. each heading in a dif- ferent direction:  transversal intentionality refers to integration of the living-​present experience by means of retention. The passive intentional acts of retention and protention that dimensionalize experience along the continuum of temporality in the next level originate from this primordial stage of consciousness. living experience of passing away. 1988). In sum. sub- jectivity versus objectivity. It is said that Heidegger noticed a phil- osophical problem concerning the cogito and consciousness. Although a disciple of Husserl. while avoiding tackling directly the problems of cogito versus perception. Heidegger subsequently departed from Husserl’s phenomenology. 3. Heidegger just could not accept the unconditional prior existence of the cogito. a unique undoubtable being. . Actually Husserl (1970) had already toyed with an idea similar to the frame system.4. which introduced domain specificity into the logical descrip- tions of objects and the world. Instead. then something or someone must be doing the doubting.  29 Phenomenology 29 based on logical inferences and categorical representation of knowl- edge in modern cognitive science or artificial intelligence originated from the ideas of Husserl. therefore the very fact that he doubted proved his exist- ence (Williams. Heidegger as a disciple of Husserl actually took an alternative route to escape this pre- dicament. changing the direction of philosophy dramatically by introducing his thinking of existentialism (Dreyfus. once he became inspired by his own thoughts on the subjective constitution of the world. Husserl. It is important to note that Heidegger sought not to obtain an objective understanding of the problem but rather to undertake a hermeneutic analysis of it. and mental versus material.  Being-​in-​the-​World Heidegger is considered by many to be one of the greatest philosophers of modern times. as we discover next. Nor could he accept an ideal and logical representation of the world that the cogito supposedly constitutes. He concluded that if he doubted. Descartes considered that the cogito. taking on this thought. should be taken as the initial point of any philosophical thoughts after everything in the world is discarded for its doubtfulness of being. he raised the more fundamental question of asking what it means to be human. 2014). However. 1991). a notable invention of Marvin Minsky. a problem that was considered by Descartes as well as Husserl and yet fully over- come by neither. presented his idea that the world and objects should exist in terms of conscious rep- resentations in the cogito and that such conscious representations ought to be ideal ones. but he finally admitted defeat in the face of infinite possibilities of situations or domains. and the carpenter building my house implies the way of presently being for each of these entities as situated among others: None exist independently but are uni- fied in their existence via the preunderstanding of how each interacts in the constitution of a situation characterized by purposeful activity. but cannot articulate it precisely when asked to do so. he asks what it means that a hammer exists.” Put another way. For now. the hammer becomes transparent to him: The hammer and . Heidegger asserts that the being of equipment is mostly “trans- parent. Despite this difficulty. beginning with this same vague preunderstanding. let’s examine Heidegger’s famous notion of being-​in-​the-​ world (Heidegger. It is said that we take being as granted. the existence of pieces of equipment is not noticed much in our daily usage of them. 1962) by looking at his interpretation of the ways of being in relation to equipment. It is not suffi- cient to answer that it exists as a thing made from cast iron and wood because such an answer merely describes its objective features. the hammer being used by the carpenter. as discussed later. we experience an unending interpreta- tive loop. the meaning of being of a hammer must be approached by way of its employment in daily activities. Because we can understand the whole in terms of its parts and the parts only through their relationship to the whole. Rather.” Such an account of nails being hit with a hammer. However. something like “The carpenter building my house is hitting nails with it. or biases. when we read a new piece of text. When a carpenter continues to hit a nail. Being and Time. In his classic text. hermeneutics possesses an inherent difficulty because preunderstanding (bias) originates from intuitions in a context-​ dependent way and there is a potential danger of being caught up in a loop of interpretation. namely emergent phenomena. of them that are adequately modified during the process of understanding. It is his thoughts on understanding by hermeneutic cycling that form the essential philosophical background to the central theme of this book. Heidegger holds that there are some fundamental problems. For example. a preunderstanding of the author’s intention would help our understanding of the content as we go along. Heidegger focuses on the purposeful exercise of naive capacities as extended by equipment and tools.30 30 On the Mind Hermeneutics is an approach that attempts to deepen the under- standing of targets while having prior estimates. the so-​called hermeneutic circle. For example. like what it means to be human. he attempts to elucidate the meaning of being via her- meneutic cycling. which can only be understood in this way.   31 Phenomenology 31 the nail are absorbed in a connected structure. ultimately recognizing and taking responsibility for his or her existence in such a way. to transform the situation in which one is “thrown” (the clearing of being as inherited. Death is to be regarded as the absolutely certain impossibility of being further related to any other kind of being. such as consciousness. Heidegger recognizes that ordinary man has rare opportunities to reflect on the meaning of his own way of being. occupied as he is with the daily routines of life. Heidegger tells us that this way of being can be changed to authentic being when man thinks positively about the possibility of his death. the unified structure breaks down and the independence of each entity becomes noticeable. and he regards such a state of being as inau- thentic. through cycles of self-​corrective analysis. vaguely anticipating the future and mostly forgetting the past. In this process. we must focus on Heidegger’s brilliant notion that the present is born via the dynamic interplay between a unique agent’s projected future possibilities and its past. he cannot become individuated. when he fails to hit the nail correctly. one reclaims one’s self from the undifferentiated flow of idle chatter and everyday routine.” merely repeating established routines and defending established conventions regardless . Man in this case lives his daily life only in the immediate present. Although man can live in his neighboring community occupied with “idle talk” and trivia. “idle chatter. the purposeful activity that is house building. However. and when confronted in this way it prompts the authentic being to head toward its own absolute impossibility. Death is an absolutely special event because it is the ultimately individuating condition that cannot be shared with others. like “why did ‘I’ fail?” and “what’s wrong with the hammer and the nail?”. which could occur at any moment and not necessarily so very far into the future. In the breakdown of the once-​unified structure. This is authenticity. it is considered that the herme- neutic approach can provide an immanent understanding of metaphys- ical existence. In this way. Their relative meanings become interpretable only in such a breakdown. as one is born into it) into that ideal self-​situation that characterizes one’s unique potential. and rather invests his or her time in distractions. The authentic agent has the courage to spend the time of his or her life in becoming an agent of change. the separated entities of the subject and the object become apparent with self-​questioning. Here. However. The inauthentic agent hides from this potential. Heidegger estab- lishes the subjective sense of temporality (rather than objective time) as the ground of authentic being. Heidegger. shows how the differentiable aspects of past. 3. For example. Although the body in terms of flesh can be regarded as material. should be taken to lie. there are other aspects of mind to review. Thus. our cheeks turn red when we get angry and tears start to fall when we feel sad. and future rise from the mortal condition.5. and from which any inquiry into the nature of being.  Embodiment of Mind In the philosophy of embodiment developed by Merleau-​Ponty. as a part of our being.32 32 On the Mind of suboptimal and even grossly immoral results. Husserl considers that temporality appears as the result of subjective reflection to articulate direct experiences of sensory streams as consciously manipulable object or event sequences. In response. As mentioned before. Merleau-​ Ponty’s notion of embodiment has been recognized as a notion of ambi- guity. and with this ambiguity he successfully avoided tackling Cartesian dualism directly. in light of which anything at all comes to matter. including the role of the body in mediating interactions between the mind and the material world. which is regarded as nonmaterial by Descartes. whereas the inauthentic being tries to nullify this subjective sense of time by ignoring his or her mortality and retreating into the blind habit and routine that is characteristic of “fallenness. and material world while these effectively exist in different spaces. we actually often experience the body as an aspect of mind. includ- ing any derivative understanding of time as sequence. is ultimately drawn. From this background. Next. Temporality is the dynamic structure of being. Descartes thought that the world consists of two extremes—​the subjective mind of nonmaterial and the objective things of materials—​and this invited a problem of interaction. we see that his notion of temporality is drastically different from that of Husserl’s. material body. present. for example.” Now. The problem is how to account for the causal interaction among nonma- terial mind. on the other hand. Merleau-​Ponty devel- oped his thoughts on embodiment. we can easily find the influence of Heidegger’s being-​in-​the-​world. Merleau-​Ponty proposes that we consider . asking at which pole the body. He had difficulty pointing to or moving a part of his body when asked to do so unless he deliberated over his movement from an objective view ahead of time. he could per- form concrete movements in natural situations in daily life very easily. and our familiar horizons (Merleau-​Ponty. however. our situation.” which is the implicit negation of what runs counter to the natural momentum that throws us into our tasks. It can be summarized then that the analysis of the blind man with his stick indicates the possibility of extension of the familiar horizon associated with daily use of the stick. These examples might help us to understand how the horizon of subjective possibility is con- stituted via daily interactions between the body and the world. our cares. had lesions in vision-​related cortical areas. he could pick up a cup to have a drink or make a fire by striking a match without problems. it becomes a part of his body when he scans his surroundings when walking by touching its tip to things. but could blow his nose with a handkerchief. . Merleau-​Ponty explained the phenomena in terms of “refusal of deficiency. Although he had a problem in recognizing objects visually. For example.  33 Phenomenology 33 the body to be an ambiguous existence belonging to neither of these two extremes. When he was asked to point to his nose. it also recalls the possibility that the range of self can be extended or shrunk through the use of tools and artifacts. In short. Although this is an interesting example showing the possi- bility of body extension. Schneider. he presented an analysis of a blind man with a stick (Merleau-​Ponty. thereby enriching our understanding of being in the world. Merleau-​Ponty examined various means of interaction between mind and body. 1962). The stick becomes an object when the blind man grasps it in order to guide his movements. he had difficulty doing so. his analysis of the phenomenon of phantom limbs might indicate the complete opposite of the blind man’s case. Along the same line of the thought. It is said that people who have had a limb amputated often still experience pain in the amputated limb that no longer exists. So. Merleau-​Ponty addressed the problem of body schema—​the integrated image of the body—​by con- ducting an analysis of a patient with neurological blindness. like tactile scanning with the finger. His patient. he could see the shapes and outlines of objects but needed a reasoning process to identify them. In another example. At the same time. one of refusal of the sudden shrinking of this once familiar horizon. 1962). “see- ing” the cup or the match. whereas the anal- ysis of phantom limbs indicates another possibility. our body or body part is not an object that we move in an objective space. a neurological condition in which sensation in one modality unconsciously evokes perception in another. His approach is to see perception as ongo- ing structuring processes of the whole. In concrete movements. He also refutes the notion of separating perception from action. that we move in a bodily space. feel textures on hearing particular sounds. He writes that painters often feel as if the objects in their own paintings gaze back at them. or expe- rience strong tastes on hearing certain words. In shaking hands. has been reported in a variety of forms. or Gestalt. He explains that the hand touching something reverses into an object that is being touched because the hand itself is tangible flesh. Some synesthetes perceive colors upon seeing certain shapes or letterforms. vision is analogous to exploring objects in the dark by tactile palpa- tion. as Heidegger discussed in terms of Dasein. by mak- ing a direct reference to the world and its objects. which appears in the communicative exchanges between the different modalities of sensation. Visual palpation by looking inevitably accompanies a sense of being seen at the same time. the body as a subject. Merleau-​Ponty refutes ordinary scientific views of modu- larity to understand the reality of perception by reducing it into the sum of each modality of sensation. perception of objects in the world is achieved in the iterative interactions between multiple modalities of sensation by reentrant mechanisms established in the coupling of us and the world. Merleau-​Ponty says that a see-​er reverses into a visible object because of the thickness of its flesh. Synesthesia. Merleau-​Ponty came to the conclusion that such concrete movements situated in everyday life are fundamental to the consideration of body schema. The body communicates with them through a skill or tacit knowledge. Merleau-​Ponty’s (1962) analysis of synesthesia is also worth introduc- ing here. wherein the body comprehends its world and objects without explicitly representing or objectifying them. Thus. These movements performed by our living body are organized in famil- iar situations in the world. which would account for how we humans engage in the world. rather than simply being perceptual side effects. Rather. Indeed. There are silent exchanges . we feel that we are touching another’s hand and simultaneously that our extended hand is being touched.34 34 On the Mind but had difficulty performing abstract movements without context and without an objective view. Merleau-​Ponty speculates that these sensory slips should have some clear meaning behind them. This direct reference implies the fundamental structure of being-​in-​the-​world. Analogously. it is our living body. At this moment of touching. pp. 1968. involving touching. as a three-​dimensional object. such that the touch is formed in the midst of the world and as it were in the things” (Merleau-​Ponty.] [A see-​er is a visible object. Although the subject of touching and the object of being touched are opposite in meaning. and things tangible. There is flux in the reciprocal network that is body and world. Imagine that your right hand touches your left hand while it is palpating something. it is expressed as: [This circle is a rectangle. the subjective world of touching transforms into the objective world of being touched. a column. the two-​d imensional world is extended to a three-​d imensional one in which a circle and a rectangle turn out to be just different views of the column. “the touching subject passes over to the rank of the touched. he would say that this circular column could be a rectangular one and this rectangular column could be a circular one (Figure 3. let’s imagine a situation in which a person who has language to describe only two-​dimensional objects happens to encounter a novel object. is a rhetorical method to locate words by crossing over.] X [An object of being touched is a subject of touching. Merleau-​Ponty wrote that. Because flesh is tactile as well as visible. it can touch as well as be touched and can see as well as be seen.] Thus.] X [This rectangle is a circle. or emerge. in which flesh of the same tangibility as well as the same thickness can be given to both the subject of touching or seeing and the object of being touched or being seen. [A subject of touching is an object of being touched. The conflict between the two is resolved by means of creating an additional dimension that supports their identity in a deeper level. This dimension of embodiment . Although the concept might become a little difficult from here onward. When this is written in the form of chiasm. as Tohru Tani (1998) sug- gests. seeing. combining subjective experience and objec- tive existence.133–​134). vision. Let’s take another example. originating from the Greek letter χ (chi). they are rendered identical when Merleau-​Ponty’s concept of chiasm is applied. Chiasm.] X [A visible object is a see-​er. descends into the things. in this sense.3).] Merleau-​Ponty suggests that embodiment as an additional dimen- sion emerges. Let’s consider then what could be created.  35 Phenomenology 35 between the see-​ers and the objects. By exploring the object from different viewpoints such as from the top or side. in the fol- lowing cases. But my seeing body subtends this visible body. the touched takes hold of the touching. as a three-​ dimensional object. . he would have imagined a flux. a column. p. When the flux intertwines the two.  A person who has language to describe only two-​d imensional objects happens to encounter a novel object. … My body as a visible thing is contained within the full spectacle. but rather allowed them to move dynamically between the two. There is reciprocal insertion and intertwining of one in the other (Merleau-​ Ponty. the seeing is not without visible existence. By positioning the poles to face each other. in exploring ambiguity between the two poles of subjectivity and objectivity. from one pole to the other and an intertwin- ing of the two in the course of resolving the apparent conflicts between them in the medium of embodiment. 1968. there is a circle of the visible and the seeing.36 36 On the Mind Figure 3. can facilitate the space for iterative exchanges between the two poles of subject and object: There is a circle of the touched and the touching. a flow.3. Merleau-​Ponty. and all the visibles with it. did not anchor his thoughts in the midst of these two extremes. the subject and the object become an inseparable being reciprocally inserted into each other with the world arising in the gap.143). As we go. 225) as follows: 1. 4. By examining James’ stream of consciousness. we’ll find some connection between James’ thinking and Husserl’s concept of time perception. feel- ings. O’Regan & Noë.  37 Phenomenology 37 Recently. The first characteristic means that the various states comprising the stream are ultimately subjective matters that the subjects feel they . referred as the behavior-​based approach (Brooks. 1990)  in artificial intelligence and robotics started under this trend. we can move closer toward answering how our will might be free. 3. and desires that flow while they constantly change. Actually. as is repeatedly encountered in later chapters. and welcomes or rejects—​chooses from among them. James defines his notion of the stream of consciousness as the inner coherence or unity of conscious states as they proceed from here to the next. a new movement. Thompson & Rosch. p.  Stream of Consciousness and Free Will We experience our conscious states of mind as thoughts. we’ll see a certain affinity between this notion and that of Merleau-​Ponty’s in his attempt to show the imma- nent dynamics of our inner phenomena. Clark. especially that of the level of absolute flow. 2000. Each personal consciousness is sensibly continuous. Ritter et al. 1991. 3. 1998. in a word—​all the while. Also. the thoughts on embodiment have been revived and have provided significant influences in cognitive science in terms of the rising “embodied minds” paradigm in the philosophy of mind and cognitive sciences (Varela. 2.. images. Within each personal consciousness states are always changing. 2001). Let’s move on now to examination of the concept of the stream of consciousness put forward by the pioneering American psychologist and philosopher William James (1892) more than a half century before Merleau-​Ponty. Every “state” tends to be part of a personal consciousness. He explains the four essential characteristics of this stream in his monumental Principles of Psychology (1918. It is interested in some parts of its object to the exclusion of others.6. what strikes is the pace of its parts. James considers that the stream comprises successions of substantive parts of stable “perchings” and transitive “flights. its states are constantly chang- ing autonomously as various thoughts and images are generated. p. the transitive parts generate successive transitions from one substantive part to another in temporal association. This statement appears to conflict with the concept of time per- ception at the objective time level put forward by Husserl. we can find a structural similarity to what Tohru Tani (1998) interpreted as consciousness flowing as well as stagnating when referring to Husserl’s flow of absolute consciousness. the subjects can keep them private in their states of mind. On the other hand. Because the thoughts and images are so faint. immediately after hearing someone say something. one of the most important of James’ claims. James says that conscious- ness is not like chopped up bits or jointed segments but rather flows like a river. The transitive parts are the fringes of stable images and relate to each other. they are lost if we attempt to catch them. The third observation suggests that the private states of consciousness constantly change but only continuously so. asserts that although the stream pre- serves the inner coherence as one stream.” Conscious states of thoughts and images appear more stably in the substantive parts. James considers these fringes to be more essential than stable images and that the actual stream of consciousness is generated by means of tensional dynamics between stable images related to each other by their fringes. and the dura- tion of each substantive part can be quite different but only in terms of subjective feeling of time. In other words. Although it is said that the transitive parts function to connect and relate various thoughts and images. James writes that: When we take a general view of the wonderful stream of our consciousness. 243). a relevant image is about to pop into the mind but is not yet quite fully formed. it seems to be an alternation of flights and perchings (James. Like a bird’s life. 1918. This alternation between the two parts takes place only intermittently. The second characteristic. because he considered that objective time comprises sequences of discrete objects .38 38 On the Mind experience by themselves. how are they actually felt phenom- enally? James describes them as a subtle feeling like when. where information flows like the free water of consciousness around these images. Here. the essential question concerning free will is that if we suppose that everything proceeds deterministically by following the laws of physics. Then. by itself. from the notion of the sensible continuity of the stream of consciousness we can see another essential consequence of James’ thought. wherein “the cause of the will is not the will itself. He . We can consider then that the notion of the stream of consciousness evolved from James’ notion of present existence characterized by con- tinuous flow to Husserl’s notion of recall or reconstruction with trains of segmented objects. He considers that will is not in fact free at all because voluntary actions. multiple alternative possibilities are imag- ined with the help of some degree of randomness in the first stage and then one possibility is chosen to be enacted through deterministic evalu- ation of the alternatives in the second stage. However. have necessary causes. what is left that enables our will to be free? According to Thomas Hobbes. 1884). as mentioned before.  39 Phenomenology 39 and events. rather than being random and uncaused. 1841. in the so-​called two-​stage model (James. Or. but something else which is not disposed of it” (Molesworth. that the continuous generation of the next state of mind from the current one endows a feeling that each state in the stream belongs to a single enduring self. James’ idea is analogous to the absolute flow level. Alongside this discussion. that con- sciousness brings forth some part of a whole as its object of attention. 376). The experi- ence of selfhood—​the feeling of myself from the past to the present as belonging to the same self—​might arise from the sensible continuity of the conscious state. in terms of the course of actions or images. Finally.” and James’ observations of this aspect of the stream of consciousness lead to his conception of free will. how can these possible alternatives. p. a materialist philosopher. “voluntary” actions are compatible with strict logical and physical determinism. In this model. Heidegger (1962) attends to this under the heading of “attunement. I suspect that James limited his observation of the stream of consciousness to the level of pure experience and did not pro- ceed to observation of the higher level such as Husserl’s objective time. James proposed a possible model for free will that combines random- ness and deterministic characteristics. be generated? James considers that all possibilities are learned by way of experience. Free will is the capability of an agent to choose freely. a course of action from among multiple alter- natives. the fourth observation professes that our consciousness attends to a particular part of experiences in the stream. However. proposed as an end. . and finally one of the alternatives is selected for actual enactment. has left an image of itself in the memory. Multiple streams of action images can be generated with spontaneous variations of transitions from among the images embedded in the memory. 487). or involuntary way. “when a particular movement. He considers further that the iterative expe- riences of different movements result in connections and relations among various images of movements in memory. Learning from various experiences forms a memory that has a relational structure among substantial images associated with actions. One of those streams of action images is selected for actual generation. Then. We believe the brain to be an organ whose internal equilibrium is always in a state of change—​the change affecting every part.40 40 On the Mind says.4 for an illustration of his ideas).4. These “accidental generations with spon- taneous variations” might be better understood by recalling how James’ stream of consciousness is constituted. and deliberately willed” (James. He writes: Consider once again the analogy of the brain. When the memory holds complex relations or connections between images of past experiences. images can be regenerated with spontaneous variations into streams of consciousness (see Figure 3. then the movement can be desired again. The stream is generated by transi- tions of thoughts and images embedded in the substantial part. James considers that all of these things are mechanized by dynamics in the brain. p. having once occurred in a random. The pulses of change are Stable image Transitive relation Experiences Learning Memory Multiple streams of Actual action an image generated selected with spontaneous variations Figure 3. multiple alternatives can be imagined as accidental generations with spontaneous variations from the memory that has been consolidated. reflex.  An interpretative illustration of James’s thought accounting for how possible alternatives can be generated. 1918. as to what the object of a refrigerator can actually mean to a robot when it names it a “refrigerator.” The refrigerator should be judged not from its characteristic physical features but from the ways in which it is used. 1892). although the figures are always rearranging themselves. discussed in the introduction to this book. his thinking has been heavily criticized by Dreyfus and other modern philosophers. His thinking is compatible with today’s cutting-​edge views outlined in studies on neurodynamic modeling. Summary Skeptics about the symbolist framework for representing the world as put forward by traditional cognitive science and outlined in ­chapter 2 has led us in the present chapter to look to phenomenology for alter- native views. considering our usage of things. Focusing on the ways of being in everyday life. as seen in later chapters. whilst others simply come and pass (James. It was Heidegger who dramatically redirected phenomenology by returning to the problem of being. a process that entails consciousness. Their representations are constituted by means of the intentionality of direct experiences.7. regardless of their corresponding existences in the physical world. They claim that the inclination to ideality with logical formalism has turned out to provide a foundation for the symbol- ist framework envisioned by current cognitive science. His thinking lies behind my early question. Heidegger explains through his notion of being-​in-​the-​ world that things can exist on account of the relational structure between them. As in a kaleidoscope revolving at a uniform rate. Phenomenology begins with an analysis of direct experiences that are not yet articu- lated by any ideas or thoughts.  41 Phenomenology 41 doubtless more violent in one place than in another. for example. … So in the brain the perpetual rearrangement must result in some forms of tension lingering relatively long. Let’s take stock of what we’ve covered. It is amazing that more than 100 years ago James already had devel- oped such a dynamic view of brain processes. Husserl considers that objects and the world can exist because they can be meditated on. their rhythm more rapid at this time than at that. 3. Although Husserl thinks that such representations are intended to be idealistic so as to be logically trac- table. . for example. who was influenced by Heidegger. The next chapter examines neuroscience approaches for exploring the underlying mechanisms of the cognitive minds in biological brains. when a carpenter mishits a nail in hammering. as usage becomes habit and habit proceeds smoothly. no longer delivering anticipated success. the authentic individual engages in serious reflection of these past habits. do these thoughts deliberated by those philosophers suggest any- thing useful for building minds. Thompson & Rosch. the subject and the object constitute an inseparable being.42 42 On the Mind such as for taking a chilled beer from it. Merleau-​Ponty. when seeing is regarded as being seen and touching as being touched. examined bodies as ambiguous beings that are neither subject nor object. His dynamic stream of conscious is generated by spontaneous variations of images from past experiences consolidated in memory. By means of such iterative processes. the hammer. In a similar way. . On Merleau-​ Ponty’s account. he notices that himself. Recently. such as by Varela and his colleagues (Varela. We finished this chapter by reviewing how William James explained the inner phenomena of consciousness and free will. though? Indeed. his thoughts on embodiment have been revived and have provided significant influences in cognitive science in terms of the ris- ing “embodied minds” paradigm. and thus lives proactively for his or her “own most” future alongside and with others with whom these habits and conventions are shared. when habits and conventions break down. Heidegger also says that such being is not noticed particularly in daily life as we are submerged in rela- tional structures. By the way. reciprocally inserted into each other in the course of resolving the apparent conflicts between them in the medium of embodiment. his ideas are still inspiring work in systems neuroscience. We become consciously aware of the individual being of the subject and the object only in the very moment of the breakdown in the purposeful relations between them. at the least we should keep in mind that action and perception interact in a complicated man- ner and that our minds should emerge via such nontrivial dynamic pro- cess. transforms them. More than a century later. 1991). and the nail are independent beings. these different modalities of sensation intertwine and their reentrance through embodiment is iterated. we saw that a phenomenological understanding of the mind has come from introspection and its expression through language. albeit in many cases the evidence is still conflicting. is clearly an essential approach. We understand the words used intuitively or deliberatively by matching them with our own experiences and images. which make use of modern technolo- gies to help us understand how we think by understanding how the brain works. recognizing visual objects. that of subjective reflection. It attempts to explain biological mechanisms for various cogni- tive functions such as generating actions. takes is quite different from that of cognitive science and phenomenology because it rests on objective observation of biological phenomena in the brain. or brain science. However.  43 4 Introducing the Brain and Brain Science In the previous chapter. What we do have is some evidence of what is hap- pening in the brain. readers should note that brain science is still in a relatively early stage of development and we have no confirmative accounts even for basic mechanisms. This approach to understanding the mind. and is especially valuable when coupled with the vast knowledge that has been accumulated through other scientific approaches. 43 . such as neuroscience. or recognizing and generating speech. The approach that neuroscience. Visual recognition is probably the most examined brain function. First.  Hierarchical Brain Mechanisms for Visual Recognition and Action Generation This section explores how visual recognition and action generation can be achieved in brains by reviewing accumulated evidence. In the proc- ess. 4. further guiding insights may be generated.1. The visual stim- ulus enters the retina first. we introduce some ideas informed by our robotics experiments on how intentions for actions originate in (human and other animal. One important characteristic assumed in the visual cortex as . A  spe- cial focus will be put on how those processes work with hierarchical organization in brains. 4. organic not artificial) brains. At the end of this chapter. anesthetized animals. because insights into this structure help to guide us in approaching outstanding questions in cognitive science. let us look at the visual recognition process. and then continues on to the primary visual cortex (V1). and research into the nature of the mind relative to the function of the brain will advance. and then covers recent views that argue that these two processes are effec- tively inseparable. The next section starts with a review of the current state of the art in brain science with a focus on the processes of visual recognition and action generation. as well as how the direct experience of sensory–​motor flow can be objectified. held against guiding insights into the phenomenology of the human condition. such as those left by James and Merleau-​Ponty. essential for creating autonomous robots. because related neuronal processes can be investigated relatively easily in electrophysiological experiments with nonmoving. the chapter provides a conventional explanation of each independently. proceeds to the lateral geniculate nucleus in the thalamus.1  Visual Recognition Through Hierarchy and Modularity First. such as how compositional manipulations of sensory-​ motor patterns can be achieved. while adding yet more experimental evidence in the confirmation or disputation of these guiding insights.1.44 44 On the Mind What we have to do is build up the most likely construct for a theory of the brain by carefully examining and linking together all the pieces of evidence we have thus far accumulated. . neurons in the column get fired) and other columns become silent. Because movements in the background scene are related to own body movements in many cases. Figure 4. the signal propagates to V2 where columns undertake slightly more complex tasks such as perceiving different orientations of line segments by detecting the end terminals of the line segments.  45 Introducing the Brain and Brain Science 45 VIP LIP LIP: lateral intraparietal area wh er VIP: ventral intraparietal area e MST/MT MST: medial superior temporal area V2 V1 MT: middle temporal area V4 TEO. passing through V4. well as in other sensory cortices is its hierarchical and modular pro- cessing. V1 is thought to be responsible for lower end processing such as edge detection by using so-​called columnar organization.1. The ventral branch is called the what pathway owing to its main involvement in object identification and the latter is called the where pathway due to its involvement in informa- tion processing related to position and movement. the MST consequently detects . After V1.  Visual cortex of the macaque monkey showing the “what” and “where” pathways schematically. only the best matching column for the edge orientation is activated (i. the visual processing pathway branches into two: The ventral path- way reaches areas TEO and TE in the inferotemporal cortex. Taking the case of the where pathway first.1 shows the visual cortex of a macaque monkey in which the visual stimulus from the retina through the thalamus enters V1 located in the posterior part of the cortex. and the dorsal pathway reaches areas LIP and VIP in the parietal cortex. whereas the MST detects background scenes with a larger recep- tive field. that is.e. The orientation of the perceived edge in the local receptive field is detected in a winner-​take-​all manner. passing through the middle temporal area (MT) and medial superior temporal area (MST). After V2. which uses specific neuronal connectivity between local regions. it is said that the MT detects direction of object motion with a relatively small receptive field. The cortical columns for edge detection in V1 are arrayed for continuously changing orienta- tion. TE: inferior temporal areas TEO TE what Figure 4. In the case of the what pathway. Columnar representations were found in the TE for a set of complex object features. proprioception. As discussed later. many neurons in the parietal cortex integrate visual inputs with another modality of sen- sation (i. cells in V4 respond to specific contours or simple object features.46 46 On the Mind Figure 4. Redrawn from (Tanaka. inspiring observations were made by Keiji Tanaka (1993) when conducting single-​u nit recording1 in par- tially anesthetised monkeys while showing the animals a set of artifi- cially created complex object features. Cells in the TEO respond to both simple and complex object features. somatosensory.. and cells in the TE respond only to complex object features. 1993). it has been found that some VIP neurons in macaque monkeys respond when the experimenter strokes the animal’s face. LIP neurons are involved in processing saccadic eye movements. Single-​u nit recording is a method of measuring the electro-​ physiological responses of a single neuron using a microelectrode system. 1. This information is then sent to areas such as the VIP and LIP in the parietal cortex. enabling the visual localization of objects. In terms of the visual processing that occurs in the inferotemporal cortex. .2. For example. or auditory).  Cell responses to complex object features in area TE in the inferotemporal cortex.2).e. Cells in the VIP are multisensory neu- rons that often respond to both a visual stimulus and somatosensory stimulus. Columnar modular representation in the TE for complex visual objects. wherein most of the cells in the same column reacted to similar complex object features (Figure 4. and the same neurons fire when the experimenter shakes the mon- key’s hand in front of its face. self-​movements. 3. This observation suggests that TE columns rep- resent a set of complex object features discretely like visual alphabets. in the sense that a set of visual parts registered in a previous level of the hierarchy are spatially combined in its next level.3. In V4 with its larger receptive field. we must exercise some caution in interpreting actual brain mechanisms from the data available . as illustrated in Figure 4. combinations of the object feature are detected as a complex object feature. It can be summarized then that visual perception of objects might be compositional in the what pathway. but allow a range of modulation of complex object feature within the column. For example. Finally. It seems that columns in each visual cortical area represent primitive features at each stage of visual processing. in the TE. Furthermore. each primitive feature represented in a column might be parameterized for minor modulation by local cell firing patterns. connected edge seg- ments for continuously changing orientations are detected as a single contour curvature.  Schematic illustration of visual perception in the what pathway. in a particular column that encodes starlike shapes.  47 Introducing the Brain and Brain Science 47 TE V4 V2 V1 Figure 4.1. edges are detected at each narrow local recep- tive field from the raw retinotopic image. and in V2 the edge segments are detected. geometric combinations of con- tour curvatures in a larger again receptive field are detected as a simple object features (some could be complex object features). Then in the TEO. different cells may react to similar starlike shapes that have a differ- ent number of spines.2  Counter Arguments As mentioned in the beginning of this section. In the first stage in V1. 4. Although the aforementioned compositional mechanisms for visual recognition were considered utilizing explicit representations of the visual parts stored in the local columns and hierarchical manipu- lation of those from the lower level to the higher. Kourtzi and colleagues (2003) provided corroborative evidence that early visual areas V1 and V2 respond to global rather than simple local features. Contrary to this method. whereas the bottom-​up signal conveys the prediction error signal from the lower level. . top-​down signals seem to be equally as important as the bottom-​up ones in understanding the hierarchy of vision. which modulates the higher level activity. however. in effect putting the local features to which the cells respond into their full perceptual global context.48 48 On the Mind to us thus far. There is accumulating evidence that neuronal response in the local receptive field in early vision can be modulated contextually by means of lateral interactions with areas outside of the receptive field as well as through top-​down feedback from higher levels. the real mechanism may not be so simply mechanical but also highly contextual. there was a close correlation between the responses of V1 neu- rons and the perceptual saliency of contours. Li and colleagues (2006) showed that. They argue that the visual recognition of complex objects is achieved via such interaction between these two pathways rather than merely through the bottom-​up one. They concluded that contours can be perceived even in V1 by using the contextual infor- mation available at this same level and the higher level. This insight is deeply important to the neurorobotic experiments to come. Rajesh Rao and Dona Ballard (1999) proposed so-​called predictive coding as a model for hierarchical visual processing in which the top-​down signal conveys prediction from the higher level activity to the lower one. they showed that the same visual contours elicited significantly weaker neuronal responses when they were not the objects of attention. Although contours are thought to be perceivable only after V4 in the classical theory. Interestingly. In the elec- trophysiological experiments of the visual cortex. These experimental results were obtainable because of the use of awake animals rather than anesthetized ones during the recording. It was argued that context modulation in the early visual cor- tex has a highly sophisticated nature. in monkeys performing a contour detec- tion task. animals are usually anesthetized so as to avoid contamination of purely bottom-​up percep- tual signals with unnecessary top-​down signals from the higher order cognitive brain regions such as the prefrontal cortex. Some researchers think that the prefrontal cortex may play a fur- ther higher functional role in action generation. Then. the idea is that detailed motor patterns along with the motor program are generated in M1. Figure 4. They mention that visual cortex cells could have multiple response dimensions.4 shows the main brain areas assumed to be involved in action generation in the cortex. M1 sends the motor pattern signals via the pons and cerebel- lum to the spinal cord. 4. It is generally held that the SMA is involved in organizing action programs for volun- tary action sequences. To sum up.3  Action Generation Through Hierarchy Understanding the brain mechanisms behind action generation is essen- tial to our attempts at understanding how the mind works because actions tie the subjective mind to the objective world. This finding obviously conflicts with the classical view that cells with similar response properties are clustered together in columns. Because these areas have dense projections to the primary motor cortex (M1). Yen and colleagues (2007) made simultaneous recordings of multiple early visual cortex cells in cats while showing the animals movies containing scenes from daily life. whereas the PMC is involved in organizing action programs for sensory guided action sequences. the presumption of strict hierarchical and modular proc- essing in visual recognition might have to be reconsidered given accumu- lated evidence obtained as experimental setups become more realistic. As a seminal study for the primary motor cortex.1.  49 Introducing the Brain and Brain Science 49 The modularity of feature representation in the columnar organiza- tion is also questionable. What they found was that there is a substantially large heterogeneity in the responses of adja- cent cells in the same columns. It is generally thought that complex actions can be generated by moving through mul- tiple stages of processing in different local areas in the brain in a similar way to how visual perception is achieved. and we will return to this view later. The next subsection begins this process concerning action generation in the brain. The supplementary motor area (SMA) and the premotor cortex (PMC) are considered to sit at the top of the action generation hierar- chy. Georgopoulos and col- leagues (1982) found evidence in electrophysiological experiments in . sitting as it does above the SMA or PMC. which then sends out detailed motor commands to the corresponding muscles to finally initiate physical movement. 4  Voluntary Sequential Movements in the Supplementary Motor Area Considerable evidence suggests that hierarchical relations exist between the SMA and M1. the mere . it is just that they seem unable to regulate their actions at will. and parietal cortex. The people act well. These patients tend to gen- erate actions completely bypassing their consciousness. their hand reaches out to it and they comb their hair compulsively. which encodes a set of basic movement patterns including the one for combing hair.  The main cortical areas involved in action generation include the primary motor cortex (M1). 4. In the following.1. monkeys that the direction of hand movement or reaching behavior is encoded by a population of neural activities in M1. It is essential to note that skilled behaviors involved in combing their hair are completely intact. premotor cortex (PMC).4. supplementary motor area (SMA). when they see a comb. we review possible relationships between SMA and M1 and between PMC and M1. The prefrontal cortex and inferior parietal cortex also play important roles.50 50 On the Mind PMC M1 SMA Parietal cortex Prefrontal cortex Inferior parietal cortex Figure 4. By way of explanation. For example. it is thought that the SMA might regulate the generation of skilled behaviors by placing inhibitory controls over M1. One well-​k nown example involves patients with alien hand syndrome due to lesions in the SMA. So if this inhibitory control is attenuated by lesions in the SMA. pushing. Figure  4. The first interesting finding was that 54 out of 206 recorded cells showed sequence-​specific activities. In this way. In the unit recording in the SMA during the regeneration phase.5a shows raster plots of one of these 54 cells. the three primitive movements were connected in serial order with a specific time interval at each transition of movement. in this case an SMA cell. within a fraction of a second. 1998. three types of task-​related cells were found. and turning a handle. Shima & Tanji. 2000).  51 Introducing the Brain and Brain Science 51 perception of a comb could automatically trigger the movement pattern for the combing of hair stored in M1. that the cell is responsible for preparing the action program for the specific sequential movement. Shima & Tanji. It is interesting to note that it took a few seconds for the SMA cell to be fully activated before onset of the sequential movements and that the acti- vation was diminished immediately after onset of the first movement. This activity might then lead to the activation of other SMA cells that can induce specific transitions from one movement to another during run time by activating partic- ular M1 cells. The M1 cell started to become active immediately before the onset of the specific movement and became fully activated during the actual movement itself. This is contrasted with the situation observed in the M1 cell shown in the raster plot in Figure 4. the task can be regarded as memory driven rather than sen- sory reactive.5b. These cells might encode whole sequences as abstract action programs with slowly changing activation profiles during the preparatory period. After the training. Here. Tanji and Shima’s results imply that some portion of SMA cells play an essential role in the generation of compositional actions by sequen- tially combining primitive movements. we can assume . 1994. which was activated only before the sequence Turn-​Pull-​Push (lower) was initiated. therefore. not before other sequences such as Turn-​Push-​Pull (upper) were initiated. In each sequence. In these studies. monkeys were trained to be able to regenerate a set of specific sequential movements involving a combination of three primitive movements—​pulling. as well as SMA cells that encode corresponding move- ments with rapidly changing activation profiles. Neurophysiological evidence for the encoding of voluntary sequential movement in the SMA was obtained in pioneering studies conducted by Tanji’s group (Tanji & Shima. It is assumed. The preparatory period of this M1 cell was quite short. the monkeys were required to regenerate each learned sequen- tial movement from memory without any sensory cues being given. a certain spatiotemporal structure that affords hierarchical organiza- tion of sequential movements. Shima and Tanji (2000) reported further important findings from more detailed recording in a similar task protocol. not for other sequences such as the Turn-​Push-​Pull sequence shown in the top panel. Adopted from Tanji and Shima (1994) with permission. 1s Mean firing (b) sec. but two or three different sequences out of four trained sequences. (a) An SMA cell activated only in the preparatory period for initiating the Turn-​Pull-​Push sequence shown in the bottom panel. 1s Figure 4. Although evidence acquired by various brain measuring techniques supports the notion that hierarchical organization of voluntary sequen- tial movements occurs in the SMA for abstract sequence processing and in M1 for detailed movement patterns. Turn Pull Push Raster plots sec. this view is not yet set in stone. Some cells were found to play multiple func- tional roles: Some SMA cells encoded not only a single specific motor sequence. Lu and Ashe (2005) recorded M1 cell activity .52 52 On the Mind (a) SMA Turn Push Pull Raster plots M1 [SEQ4] Turn Pull Push Mean firing sec.  Raster plots of showing cell firing in multiple trials in the upper part and the mean firing rate across the multiple trials in the supplementary motor area (SMA) and primary motor cortex (M1) during trained sequential movements. This suggests an interesting neuroscientific result that a set of primitive sequences is represented by distributed activation of some SMA cells rather than each sequence being represented by some specific cells exclusively and uniquely. In later work. Valuable challenges have arisen against the idea of the SMA encod- ing abstract sequences.5. (b) An M1 cell encoding the single Push movement. each arm movement was either downward. They trained monkeys to generate sequential movements under two different condi- tions: the internal motivation condition in which the monkeys remem- bered sequential movements and reproduced them from memory. It was found that the neural activity of some M1 cells imme- diately before onset of the sequential movements “anticipated” the com- ing sequences. It is known that there are so-​called bimodal neu- rons in the PMC that respond to both specific visual stimuli and to one’s own movement patterns.1. Unit recording in both the SMA and PMC during these two task conditions revealed a distinct difference in the functional roles of these two regions. whereas the PMC is considered to generate actions in a more externally driven manner by making use of immediate sensory infor- mation. upward.. PMC neurons were more active when the task was visually guided and SMA neurons were more active when the sequence was self-​determined from memorized sequential movements. and the external sensory driven condition in which the monkeys generated sequential movements guided by given visual cues. this percentage is much higher than that observed in the SMA by Tanji and Shima. These bimodal neurons in the PMC associ- ated with visual movement are said to receive “what” information from the inferotemporal cortex and “where” information from the parietal .5  Sensory-​Guided Actions in the Premotor Cortex The SMA is considered by most to be responsible for organizing com- plex actions such as sequential movements based on internal motiva- tion. In the task. artificially created by microinjection of chemicals. degraded only the generation of sequences not each move- ment. Are the sequence-​related activities of M1 cells merely epiphenomena that reflect the activity of SMA cells upstream or do they actually function to initiate corresponding motor sequences? Lu and Ashe dispelled any doubt about the answer by demonstrating that a lesion among the M1 cells. 4. During both the premovement and movement periods. 1991). or toward the right. Surprisingly. at least in the monkeys and cells involved in Lu and Ashe’s experiment. Mushiake in Tanji’s group showed clear neurophysiological evidence for this dissociation (Mushiake et  al. and that 40% of the recorded M1 cells could do this. toward the left.  53 Introducing the Brain and Brain Science 53 during sequential arm movements in monkeys. It seems then that M1 cells primarily encode sequences rather than each movement. The next section explores an alternative view accounting for action generation mechanisms. in some aspects. and chest. Some evidence was also presented indicating that many neurons in the motor cortices are actually bimodal neurons that participate not only in motor action generation but also in sensory perception. these bimodal neurons seem to enable the PMC to orga- nize sensory-​g uided complex actions. They stimulated motor-​related cortical regions with an elec- tric current and recorded the corresponding movement trajectories of the limbs. However. Some stimuli generated movements involved in reaching to specific parts of the monkey’s own body including the ipsilateral arm. and the SMA and PMC are together responsible for the more macroscopic manipulation of these primitives. for example. whereby M1 seems to encode primitive movements. It was also found that many of those neurons were bimodal neurons exhibiting responses also to sensory stimulus. the PMC.54 54 On the Mind cortex. which has recently emerged from obser- vation of bimodal neurons that seem to integrate these two processes of action generation and recognition. whereas others generated movements involving reaching toward external spaces. . some textbookish evidence has been introduced to account for the hierarchical organization of motor generation. They found some topologically pre- served mapping from sites over a large area including M1 and PMC to the generated reaching postures. Thus. rather than there being separate subdivisions such as M1. At the same time. their experimental results conflict with the conven- tional ideas that M1 encodes simple motor patterns such as directional movements or reaching actions as shown by Georgopoulos and col- leagues. mouth. but reached toward the upper space when the ventral and anterior sites were stimulated. The hand reached toward the lower space when the dorsal sites in the region were stimulated. Graziano and colleagues have adopted a different view from the conventional one in that they believe that functional specifi- cation is topologically parameterized as a large single map. Given these results. Graziano and colleagues (2002) in their local stimulation experi- ments on the monkey cortex demonstrated related findings. and the SMA that are responsible for differentiable aspects of motor-​related func- tions in a more piecemeal fashion. So far. some counter evidence was introduced that M1 cells function to sequence primitives as if no explicit differences might exist between M1 and the PMC. as well as a particular sequence of visual perception of my hand approaching the mug with a specific expectation related to the moment of touching it.  55 Introducing the Brain and Brain Science 55 4. a subject should be able to anticipate the perceptual outcomes for his or her own intended actions if similar actions are repeated under similar conditions. In their ecological approach. learning an action is not just about learning a motor command sequence. In one way. intended behaviors performed by bodies acting on environments necessarily result in changes in proprioception. in the case of voluntary action. tactile. top and bottom. They once wrote in their seminal book (2000) that infants are active learners who perceptually engage their environments and extract information from them. For example. Putting two together. Eminent neuroscientist Walter Freeman (2000) argues that action generation can be regarded as a pro- active process by supposing this sort of action–​perception cycle. Indeed. as this area appears to be the exact place where the top-​down perceptual image for action intention originating in the frontal area meets the perceptual reality originating bottom-​up from the various peripheral sensory areas. the developmental psychologists Eleanor Gibson and Anne Pick have emphasized the role of perception in action generation. rather than as the more passive. Upon keeping minds of these arguments. the parietal cortex may play an essential role in mediating between the two. Indeed. However. Rather.2. visual. conventional perception–​action cycle whereby motor behaviors are generated in response to perception. when I reach for my mug of coffee. we may think that a motor behavior is generated in response to a particular sensory input. actions might be represented in terms of an expec- tation of the resultant perceptual sequences caused by those intended actions. It then examines in detail so-​called mirror neurons that are thought to be essential to pair generation and . this chapter starts by exam- ining the functional roles of the parietal cortex.  A New Understanding of Action Generation and Recognition in the Brain This book has alluded a number of times to the fact that perception of sensory inputs and generation of motor outputs might best be regarded as two sides of the same coin. it might be represented by a particular sequence of proprioception for my hand to make the preshape for grasping. it involves learning possible perceptual structures extracted during intentional interactions with the environment. and auditory perceptions. Thus located. Goodale and Milner tested D. When she was asked to name some household items. Later. and orientation because her ventral what pathway includ- ing the inferotemporal cortex was damaged.1  The Parietal Cortex: Where Action Intention and Perceptual Outcome Meet The previous section (4. She could. by allowing a close interaction between motor components and sensory components. It is said that the finding of mir- ror neurons drastically changed our understanding of the brain mecha- nisms related to action generation and recognition. could not recog- nize three-​d imensional objects visually using information about their category. by examining some evidence from neuroscience that bear on the nature of free will. is very similar to that of Merleau-​Ponty’s patient Schneider (see c­ hapter 3). That the parietal cortex involves the generation of skilled behaviors by integrating vision-​related and motor-​related processes is a notion supported by the findings of electrophysiological experiments. In this sense then. David Milner. Finally. F. the parietal cortex appears to be involved in how to manipulate visual objects. D. she misnamed them. 4. she could do it smoothly. size. and colleagues (1991) conducted a series of investigations on patient D.” that are thought to be initiated farthest upstream in the actional brain net- works. however.56 56 On the Mind to perceptual recognition of actions. or “will. the case of D.  F. Mel Goodale.  who had visual agnosia.2. but not in the dorsal how pathway. Today. a severe disorder of visual recognition. Thus.1) discussed the what and where pathways in visual processes. F. However. This implies that D. when she was asked to pick up a pen from the table. espe- cially those concerning bimodal neurons in the parietal cortex of the .’s ability to perceive the three-​d imensional orientation of objects. F. was found to have bilateral lesions in the ventral what pathway. This was possible because her dorsal pathway including the parietal cor- tex was intact. F. many researchers refer to the where path- way that stretches from V1 to the parietal cortex as the how pathway because recent evidence suggests that it is related more to behavior gen- eration that makes use of multimodal sensory information than merely to spatial visual perception. calling a cup an ashtray or a fork a knife. in the parietal cortex. gener- ate visually guided behaviors without conscious perception of objects. the chapter rounds out by looking at neural correlates for intentions. Skilled object manip- ulation behaviors such as pushing a switch should require an association between the visual information about the object itself and the motor outputs required for acting on it and so. 2000. can result in various deficits in skilled behavior needed for tool use. they might knock on the table with their fist or when asked to mime picking up tiny grains of rice. an increasing number of recent studies suggest that the parietal cortex might participate in predicting perceptual inputs associated with behaviors by acquiring some type of internal model (Sirigu et  al. Hideo Sakata and colleagues (1995) identified populations of neurons that fire both when pushing a switch and when visually fixating on it. In particular. 1996. such as that caused by cere- bral hemorrhage due to stroke or trauma. and surrounding workspace). individuals have difficulty particularly with miming:  When asked to mime using a knife. They found that specific cells in the parietal cortex encode temporal estimates of the direction in which the cursor is moving. individuals cannot understand how to use tools: If they are given a comb. 2003. and through these models various mental images about pos- sible interactions with the external world can be composed... Mulliken and colleagues (2008) found direct evidence for the existence of the predictive model in the parietal cortex in their unit recording experiment involving monkeys performing a joystick task to control a cursor. they might try to brush their teeth with it. objects. be mechanized in the parietal cortex? Such skills would seem to require not only motor pattern generation but also proactive represen- tation of the perceptual image associated with the motor act. These clinical observations suggest that the parietal cortex might store some forms of knowledge. tools. Although the parietal cortex is conventionally seen as being responsible for inte- grating input from multiple sensory modalities.. Eskandar & Assad. or “models. Bor & Seth.g. for instance. they move their hand toward the imagined grains but with it wide open.. but can be obtained by forward prediction. Damage to the parietal cortex in humans. estimates that cannot be obtained directly from either of the current sensory inputs or motor outputs to the joystick. In the disorder ideational apraxia. 2008. In ideomotor apraxia.” about the external world (e. Desmurget & Grafton. . Mulliken et al. some popula- tions of parietal cortex neurons should participate in this association by accessing both modalities of information.  57 Introducing the Brain and Brain Science 57 monkey during visually guided object manipulation. How can the skills or knowledge for object manipulation. by extension. or tool usage. 2012). Ehrsson et  al. 1999. speculate that the predictive model in the parietal cortex may predict the perceptual outcome sequence as corresponding not to motor commands at each moment but to macroscopic states of “inten- tion” for actions that might be sent from the higher-​order cognition pro- cessing area such as the prefrontal cortex (Figure 4. More recently. Some researchers have considered that a predictive model referred to as the forward model and assumed to operate in the cerebellum might also help us to understand what is happening in the parietal cortex. and Arbib (2006) as well as Blakemore and Sirigu (2003) have suggested that both the parietal cortex and cerebellum might host the forward model. For example. Following Ito’s idea. however. Mitsuo Kawato and Daniel Wolpert constructed detailed forward models—​ computational models—​ t hat account for optimal control of arm movements (Kawato. Wolpert & Kawato.6). Masao Ito. Oztop. 1998). the forward model predicts changes in the angular positions of the arm joints as output when given joint motor torques as input. If the predictive model attempts to learn to predict . suggested that the cerebellum might host internal models for action (Ito. In a similar manner. The forward model basically predicts how the current sensory inputs change in the next time step for arbitrary motor commands given in the cur- rent time step. Adequate training of the forward model based on iterative past experience of how joint angles change due to particular applied motor torques can generate a good predictive model. including many unrealistic ones. such predictive models acquired by a skilled carpenter can predict the visuo-​auditory-​proprioceptive flow associated with an intention of “hitting a nail.58 58 On the Mind Now. In the case of arm movement control. I. let’s consider how predicting perceptual sequences could facilitate the generation of skilled actions in the parietal cortex. who is famed for his findings linking long-​term depres- sion to the cerebellum. Kawato. 1990.” These illustrations just follow the aforementioned thought by Gibson and Pick. 1970). In addition.” the correspond- ing visuo-​proprioceptive flow consisting of proprioceptive trajectory of body posture change and visual trajectory of the ball falling into the net can be predicted. Ito (2005) suggested that the forward model might be first acquired in the parietal cortex and the model further consolidated in the cerebellum later. The point here is that a predictive model may not need to predict the perceptual out- comes for all possible combinations of motor commands. for a given intention of “throwing a basketball into a goal net. The intention is modified in the direction of minimizing the mismatch between the prediction and the perceptual outcome. Motor imagery is a mental process by which an individual imagines or simulates a given action without physically moving any body parts or sensing any signals from the outside world. If the predictive model just predicts perceptual sequences for given intentions for action. Indeed. 1963) in AI research. how can motor command sequences be obtained? It can be considered that a predicted body posture state in terms of antici- pated proprioception might be sent to the premotor cortex or primary . Prediction of proprioception in terms of body posture results in the generation of necessary motor command sequences for achieving it. Jeannerod (1994) has conjectured that individuals have so-​ called motor imagery for their well-​practiced behaviors. The predictive model assumed in the parietal cortex can generate motor imagery by means of a look-​ahead predic- tion of multimodal perceptual trajectories over a certain period.  59 Introducing the Brain and Brain Science 59 Mismatch info. a predictive model needs to predict possible perceptual trajectories associated only with a set of well-​ practiced familiar actional intentions. patients with damage to the primary motor area. Instead. which has been known as the “frame problem” (McCarthy. Intention M1 S1 Parietal Proprioceptive prediction Mismatch Visual prediction Visual perception Motor command Figure 4. all possible motor command combinations. and ones in the parietal cortex reported that patients with lesions in the parietal cortex showed selec- tive impairment in generating motor imagery. By receiving intention for action from the prefrontal cortex it predicts perceptual outcomes such as visuo-​proprioceptive trajectories.6.  Predictive model in the parietal cortex. Sirigu and colleagues (1996) compared healthy individuals. such an attempt will face a combinatorial explosion. however. When the visual or tactile sensation actually perceived is something different from the predicted sensation.1). current intentions can be reformed in light of a changing situation or mistaken environment. the current intention of continuing to hit the nail would be shifted consciously to a different intention such as look- ing for the mishit nail or in searching for an unbroken hammer. Such shifts in intentional states might be brought about through the mismatch error between prediction and perceptual reality. the cerebellum might compute the necessary motor torque to be exerted on the thumb and finger joints in order to achieve the expected reaction force. If the miss-​hit does not happen. where the necessary motor commands or muscle forces to achieve this target posture might be composed. like in the situation described by Heidegger wherein the hammer misses hit- ting a nail (see ­chapter 3). When action changes the perceptual reality from the one expected. for example. As the consequence of interaction between these top-​down and bottom-​up processes. the intention state may be updated in the direction of minimizing the mismatch error. The target sen- sory signal could be a reaction force that is anticipated to be perceived. The prediction of sensory modali- ties such as vision and tactile sensation that is projected to each periph- eral sensory area through the top-​down pathway might be compared with the actual outcome.60 60 On the Mind motor cortex (M1) via primary somatosensory cortex (S1) as a target posture to be achieved in the next time step. everything will continue on auto- matically as expected without any shifts occurring in the current inten- tion. When such a mismatch is generated. Again. Let’s look next at the bottom-​up recognition that is thought to be the counterpart to top-​down prediction. This information is further sent to the cerebellum. There is some recent evidence to this effect based on human brain imaging techniques including functional magnetic resonance imaging (fMRI) . This constitutes the top-​down sub- jective intentional pathway acting on the objective world as introduced through the brief review of phenomenology given in c­ hapter 3. The obvious question to ask is whether in fact the brain actually employs such intention adjustment mecha- nisms by monitoring the outcomes of its own predictions or not. the recognized perceptual reality alters the current intention. This aspect of top-​down and bottom-​up interaction is analogous to predic- tive coding suggested for hierarchical visual processing as proposed by Rao and Ballard (see section 4. in the thumb and index finger in the case of precisely grasp- ing a small object. 2  Returning to Merleau-​Ponty The concept behind the predictive model accords well with some of Merleau-​Ponty’s thinking. he writes that the stick can be also a part of the body when the man scans his surroundings by touching its tip to things. In his analysis of a blind man walking with a stick.” as reviewed in c­ hapter 3). as described in ­chapter 3. That said. he acquires a model through which he can anticipate how tactile sensation will propagate from the tip of stick while touching things in his environment. 2012).. rather than in one specific local region. Both techniques are known to be good at measuring global brain activity and to compliment one another. it may be reasonable to consider the alternative. whatever regions are actually involved. provided that the anticipation agrees with the outcome. the stick could be felt to be a part of the body. the inferior frontal cortex. 4. From this more distributed point of view. it is the interactions between them that are indispensable in the organization of diverse intentional skilled actions in a changeable environment..  61 Introducing the Brain and Brain Science 61 and electroencephalography (EEG).g.2. with relatively good spatial resolution from fMRI and good temporal resolution from EEG. where the temporal and parietal lobes meet. which we can think about in terms of Husserl’s notion of protention (e. This phenomenon can be accounted for by the acquisition of a predictive model for the stick. and the vari- ous peripheral sensory areas.. and recalling Heidegger’s treatment of equipment as extensions of native capacities for action. 2005. Because of this unconscious anticipation. Frith & Frith. Atsushi Iriki and colleagues (1996) made an impor- tant finding in their electrophysiological recording of the parietal cortex . 2012). During a lengthy period in which the man uses the same stick. 2000. Related to this. that inter- actions between top-​ down prediction with a specific intention and bottom-​up modification of this intention take place in a web of local networks including the frontal cortex. parietal cortex. These imaging studies have suggested that the temporoparietal junction (TPJ). Balslev et al. we would anticipate hearing the next note of “mi” when hearing “re” in “do-​re-​mi. and the SMA may all be involved in detecting mismatches between expected and actual perception in mul- timodal sensations (Downar et al. It may be the TPJ that triggers adjustments in current action by detecting such mismatches (Frith & Frith. 62 62 On the Mind in monkeys during a tool manipulation task. The phantom limb phenomenon described in ­chapter 3 can be under- stood as an opposite case to that of the blind man’s stick. in the same way that the stick becomes a part of the body of a blind man. each neuron fires only when the visual or tactile stimulus comes to a specific position relative to the palm (Figure 4. the same neurons fired when the visual stimulus approached the vicinity of the rake. It was shown that these particular neurons have a certain receptive field. In the without-​rake phase. they found that some bimodal neurons fired either when a tactile stimulus was given to the palm of the hand or when a visual stimulus approached the vicinity of the palm.7. neurons in the intraparietal sulcus.  The receptive field of neurons in the intraparietal sulcus (a) in the vicinity of the hand in the without-​rake phase and (b) extended to cover the vicinity of the rake in the with-​rake phase.7b). After the training. This shifting of the receptive field from the vicinity of the hand to that of the rake implies that the monkey perceives the rake as a part of the body when extended from the hand and purposefully employed. were recorded for two phases: cap- turing the food without the rake and capturing the food with it. in the with-​rake phase. Monkeys thus seem to embody a predictive model that includes possible interactions between the rake and the food object. thus demon- strating an extension of the visual receptive field to include the rake (Figure  4. . Surprisingly. Monkeys confined to chairs were trained to use a rake to draw toward them small food objects located in front of them. Thus. a part of the parietal cortex.7a). Even though (b) (a) Figure 4. Instead. Several fMRI studies of object manipulation and motor imagery for objects have shown signifi- cant activation in the inferior parietal cortex. the how pathway stretching through the parietal cortex is reminiscent of ambiguity in Merleau-​Ponty’s sense. In sum then. The psychosomatic treatment invented by Ramachandran and Blakeslee (1998) using the virtual-​reality mirror box provided patients with fake visual feedback that an amputated hand was moving. the predictive model for the limb might remain as a “familiar horizon. Merleau-​Ponty held that synesthesia. it is not feasible to assume that each modality of sensation anticipates this independently. which is then sent to the phantom limb from the motor cortex. via the bottom-​up . a shared structure should exist or be organized that can anticipate incoming sensory flow from all of the modalities together.” as Merleau-​Ponty would say. Actually. and it makes sense to consider that the bimodal neurons found in the parietal cor- tex as well as in the premotor cortex might in part constitute such a structure. Probably the goal of object manipulation propagates from the prefrontal cortex through the supple- mentary motor area to the parietal cortex via the top-​down pathway. This feedback to the predictive model would have evoked the propriocep- tive image of “move” for the amputated limb by modifying the current intention from “freeze” to “move.  63 Introducing the Brain and Brain Science 63 the limb has been amputated. the functional role of the parietal cortex in many ways reflects what Merleau-​Ponty was pointing to in his philosophy of embodi- ment. including the visual cortex and somatosen- sory cortex for tactile and proprioceptive sensation.” which might result in the feeling of twitching that patients experience in phantom limbs. as it is located mid- way between the visual cortex that receives visual inputs from the objec- tive world and the prefrontal cortex that provides executive control with subjective intention over the rest of the brain. It is speculated that a dynamic structure such as this is composed of collective neuronal activity. wherein sensation in one modality unconsciously evokes perception in another. whereas perceptual reality during manipulation of the object propagates from the sensory cortices. which would generate the expectation of a sensory image corresponding to the current action intention. might originate from iterative interactions between multiple modalities of sensation and motor outputs by means of reentrant mechanisms established in the coupling between the world and us (see ­chapter 3). If we consider that the predictive model deals with the anticipation of multimodality sensations. Here. Figure 4. . With a grad student bringing an ice cream cone to his mouth. and whenever they fired.8a shows a PMv neuron firing as the monkey observes the experimenter grasping a piece of food. every time he brought it to his lips.4  The Evidence for Mirror Neurons In the mid-​1990s.8b. electrodes activated electronic cir- cuitry to give an audible beep. researchers in the Rizzolatti laboratory in Parma were investigating the activities of neurons in the ventral premotor area (PMv) in the control of hand and mouth movements in monkeys. They had found that these neurons fired when the monkey grasped food objects.8 shows the firing activity of a mirror neuron responding to a particular self-​generated action as well as to the same action per- formed by an experimenter.64 64 On the Mind pathway. the same neuron fires again when the monkey grasps the food given by the experimenter. In Figure 4. Personally. I find the idea of mir- ror neurons very appealing because it promises to explain how the two essential cognitive processes of generating and recognizing actions can be unified into a single system. with close interaction occurring in the parietal cortex.3  Mirror Neurons: Unifying the Generation and Recognition of Actions Many researchers would agree that the discovery of mirror neurons by Rizzolatti’s group in 1996 is one of the most important findings for sys- tems neuroscience in recent decades. it can be seen that the same neuron does not fire when the monkey observes the experimenter picking up the food with an (unfamiliar!) tool. 4. we see that the firing of the neuron ceases as the experimenter moves the food toward the monkey. mirror neurons were discovered! Figure 4.2. Both of these pathways likely intermingle with each other. Then. Serendipitously. 4.2. one day when a graduate student entered the lab with an ice cream cone in his hand. but thereafter firing occurs as described for the rest of the sequence of events in (a). the system responded with a beep! The same neurons were firing both when the monkey grasped food objects and moved them to its mouth and when the monkey observed others doing a similar action. the . as recent human fMRI imaging studies found mirror systems also for intransitive actions (Rizzolatti & Craighero. not for parts of them. Adopted from (Rizzolatti et al. There are two important characteristics about these mirror neurons. In their experiments. (b) The same mirror neuron does not fire when the monkey observes the experimenter pick up the food with a tool (left). 2005) have indicated that mirror neurons can be observed in the inferior parietal lobe (IPL) and that these function to both generate and recognize goal-​directed actions composed of sequences of elementary movements.  How mirror neurons work. it looks to be a different case for humans. monkeys were trained to perform two different goal-​directed actions: to grasp pieces of food and then move them to their own mouths to eat. The second characteristic is that all the mirror neu- rons found in the monkey experiments are related to transitive actions toward objects. Recent monkey experiments by Rizzolatti’s group (Fogassi et  al. that is. the grasping neurons do not fire when the monkey is just about to grasp the object. Besides these “grasping neurons. That said. but it fires again when the monkey grasps the same piece of food (right). 2004). 1996) with permission. however.8. Interestingly. and to grasp solid objects (the same size and shape as the food objects) and then place them into a cylinder. Mirror neurons in the monkey so far do not respond to intransitive behaviors such as reaching the hand toward a part of the body. (a) Firing of a mirror neuron shown in raster plots and histograms in the two situations in which the monkey observes the experimenter grasp a piece of food (left) and thereafter when the monkey grasps the same piece of food (right).” they also found “holding neurons” and “tearing neurons” that functioned in the same way.. The first is that they encode for entire goal-​d irected behaviors.  65 Introducing the Brain and Brain Science 65 (a) (b) 20 20 Spikes s–1 Spikes s–1 10 10 0 0 0 1 2 3 4 5 0 1 2 3 4 5 Time (s) Time (s) Figure 4.. Recent imaging studies focusing on imitative behaviors have also iden- tified mirror systems in humans. but also occurs with compositional goal-​directed actions consisting of chains of elementary movements. it is indeed feasible that these local sites could host mirror neurons in humans. Based on accounts of the evolutionary pathway from nonhuman primates to human. he has developed the view that the involvement of mirror neurons in embod- ied experience grounds brain structures that underlie language. Michael Arbib (2012) has explored possible linkages between mir- ror neurons and human linguistic competency. dolphins. 2010). He has hypothesized that what he calls the “human language-​ready brain” rests on evolutionary developments in primates including mirror system pro- cessing (for skillful manual manipulations of objects. fMRI experimental results have shown that neural activa- tion in the posterior part of the left inferior frontal gyrus as well as in the right superior temporal sulcus increases during imitation (Iacoboni et al. This view is reinforced by the fact that the same IPL neurons fired when the monkeys observed the experimenters achieving the same goals. 1999). These IPL neurons can therefore also be regarded as mirror neurons. and parrots can perform imitation. He further pro- posed that the development of protosigns provided the scaffolding essen- tial for protospeech in the evolution of protolanguage (Arbib. even though the kinematics of grasping in both cases are the same. If we consider that the posterior part of the left inferior frontal gyrus (also called Broca’s area) in humans is homologous to the PMv or F5 in monkeys. namely to eat or to place. but from the difference between goals. imitation of the manipulations performed by others. and conventionalized manual gestures) that initiates the protosign system. it is still widely held that the imita- tion capability uniquely evolved in humans has enabled them to acquire wider skills and knowledge about human-​specific intellectual behaviors including tool use and language. ..66 66 On the Mind activation patterns of many IPL neurons while grasping the objects differ depending on the subsequent goal. Imitation is considered to be cognitive behavior whereby an individual observes and replicates the behaviors of others. pantomime. Although it is still a matter of debate as to how much other animals including nonhuman primates. Supplemental experiments confirmed that the activation preferences during grasping do not originate from differences in visual stimuli between food and a solid object. It is certainly interesting that mirror neuron involvement is not limited to the generation and recognition of simple actions. 5  How Might Mirror Neurons Work? The reader may ask how the aforementioned mirror neural functions might be implemented in the brain. researchers have exam- ined this idea using various brain imaging techniques such as fMRI. “pick” activated the dorsolateral sites of the motor cortex. whereby the men- tal states of others are interpretable through mental simulations that adopt their perspective. Tettamanti and colleagues (2005) observed similar types of activation patterns when their subjects listened to action-​related sentences such as “I bite an apple. Vittorio Gallese and Alvin Goldman (1998) suggest that mirror neu- rons in humans play an essential role in theory of mind in social cogni- tion. These results also suggest that Broca’s area might be a site of mirror neuronal activ- ity in humans. 4. to protolanguage and to theory of mind. which in turn initiate corresponding activations in motor-​related areas. Hauk and colleagues (2004) showed in an fMRI experiment that reading action-​related words with different “end effectors.” evoked neural activities in the motor areas that overlap with the local areas responsible for generating motor movements in the face. these results suggest that understanding action-​related words or sentences generates certain canonical activa- tion patterns of mirror neurons.  67 Introducing the Brain and Brain Science 67 This hypothesis is interesting in light of the fact that mirror neurons in human brains might be responsible for recognizing the intentions of others as expressed in language. If these aforementioned human cases are granted.” “pick.” “I grasp a knife. positron emission tomography. They argue for a simulation theory. and leg. “lick” activated the sylvian fissure.” Taken together. possibly in Broca’s area. respectively. Let’s consider the mirror neuron .” namely “lick. Broca’s area was acti- vated for all three words. from manual object manipulation.2. it can be said that the mirror neuron system has played an indispensable role in the emergence of uniquely human cognitive competencies from evolu- tionary pathways.” and “I kick a ball. and “kick” activated the vertex and interhemispheric sulcus. by tracking or matching their states with states of one’s own. More specifically. and EEG. The theory of mind approach postulates that although the mental states of others are hidden from us. arm. Actually. they can be inferred to some extent by applying naïve theories or causal rules about the mind to the observed behavior of others.” and “kick. . which we assumed may be located in the parietal cortex. thereby minimizing the mismatch error.68 68 On the Mind mechanism in terms of the aforementioned predictive model (see Figure 4. On this model. and posture of one’s own hand in relation to the cup are predicted by receiving inputs from mirror neuron activation that rep- resents the intentional state for this action. (Although Figure 4. expected perceptual sequences in terms of relative position.” We will come back to the idea of the predictive coding model for mirror neu- rons in greater detail as we turn to related robotics experiments in later chapters. If we assume that mirror neurons encode intention for action. it could be by mirror neurons present in this area including Broca’s area for humans.) In generating one’s own actions. They suggested that mirror neuron discharge serves the purpose of retrodicting target mental states. whereby the mirror neurons function as a switcher between a set of intentional actions. due to the generation in the mirror neurons of motor imagery representing the same intentional state. moving back- ward from the observed action. Different actions can be generated by receiving inputs of different mirror neuron activation patterns. In the case of observing others grasp the coffee cup. This assumption accords exactly with what Gallese and Goldman (1998) suggested for mirror neurons in terms of simulation theory as described previously.6 assumed that the intention might be hosted somewhere in the prefrontal area. ori- entation.6). the recognition of other’s actions causes one to feel as if one’s own actions were being generated. Recognition of the same action performed by others can be achieved by utilizing the mismatch information as described previously. such as grasping a coffee cup. thus representing a primitive version of a simulation heuristic that might underlie “mind-​reading. the corresponding intentional state in terms of the mirror neuron activity pattern can be searched such that the reconstructed percep- tual sequence evoked by this intentional state can best fit with the actually perceived one in the coordinate system relative to the cof- fee cup. we can easily explain how a particular activation pattern of the mirror neurons can lead to the generation of one’s own specific action and how recogni- tion of the same action performed by others can lead to the same action pattern in the mirror neurons. Libet was trying to measure the exact timing when the subjects became conscious of their decision to initiate the button press action. This question is related to the problem of free will that was introduced in the description of Williams James’s philosophy (see c­ hapter 3). If every aspect of free will can be explained by deterministic physical laws.3. first.  69 Introducing the Brain and Brain Science 69 4. there have been some interesting experimen- tal results showing possible neural correlates of intention and free will. we have seen that voluntary actions might be generated by means of a top-​down drive by an intention. how can this determination be accompanied by consciousness? Or more simply. 4. which he called “w-​judgment” time. By asking the subjects to report the position after each button press . subjects were asked to press a button with their right hands at whatever moment they wished and their EEG activity was recorded from their scalp. The intention could be hosted by the mirror neurons or other neurons in the prefron- tal cortex. 1985). The problem about free will then concerns its origin. As he says.3. we are left with the essential question of how an intention itself can be set or generated. If we can freely determine our actions. free will might be the capability of an agent to choose independently a course of action freely from among multiple alternatives. there should be no space actually remaining for free will. The subjects were asked to watch a rotating clock hand and to remember the exact position of the clock hand when they first felt the urge to move their hand to press the but- ton. In his experiments (Libet. Wherever it is represented in the brain. the seminal study on conscious inten- tion conducted by Benjamin Libet.  How Can Intention Arise Spontaneously and Become an Object of Conscious Awareness? In this chapter so far. how can I feel consciously that I have just determined to do one thing and not another? Although there have been no definitive answers to this philo- sophical question thus far.1  Searching for the Neural Correlates of Intention I would like to introduce. Can our minds set intentions for actions absolutely freely without any other causes? Can intentions shift from one to another spontaneously in a chain for gen- erating various actions? Another interesting question concerns the issue of consciousness. 70 70 On the Mind trial. noted that Libet’s experiment has drawn sub- stantial criticism along with enthusiastic debates on the results. namely the readiness potential evoked in the SMA. However. 2008).9). This EEG activity was localized in the SMA. It is said that subjective estimate of time for consciousness arising is not reli- able (Haggard. It was found that the average timing of conscious intent to act is 206 ms before the onset of muscle activity and that the Readiness Potential (RP) to build up brain activity (as measured by EEG) started 1 s before movement onset (Figure 4. it is also true that Libet’s study has been replicated by oth- ers and further extended experiments have been conducted (Haggard. This is a somewhat sur- prising result because it implies that the voluntary action of pressing the button is not initiated by conscious intention but by unconscious brain activity. At the very least. recorded during a free decision task conducted by Libet (1985). It should be. however. .  The readiness potential to build up brain activity prior to movement onset.9. Trevena and Miller (2002) reported that many reported conscious decision times were before the onset of the Lateralized Readiness Potential that represents actual preparation for movement as opposed to RP representing contemplation for movement as a future possibility. 2008). Also. Soon and colleagues (2008) showed that this unconscious brain activity to initiate voluntary action begins much longer before the onset Conscious decision Readiness potential onset –206 ms – 500 ms – 1000 ms Voltage + –2 –1 0 Time (s) Movement onset Figure 4. it demonstrates that one prepares to act before one decides to act. the exact timing of their conscious intention to act could be mea- sured for each trial. with consciousness of this inten- tion to act arising only a few hundred milliseconds before movement onset. the outcome of the motor decision to select between the two actions (a selection the subjects did not consciously make) could be predicted from this early brain activity. Moreover. Brain activity for selecting a voluntary action is initiated unconsciously in the frontopolar part of the prefrontal cortex or in the precuneus in the parietal cortex from more than several seconds to 10 seconds before the onset of cor- responding physical movement. they demonstrated that brain activity is initiated in the frontopolar part of the prefrontal cortex and in the precuneus in the medial area of the superior parietal cortex up to 7 s before a conscious decision is made to select either pressing the left button with the left index finger or the right button with the right index finger. from the brain activity observed. This notion relates to the “spontaneous generation of alternative images and thoughts” put forward by William James.4). Controversially. let’s assume that the unconscious activ- ity in the beginning might not be caused by anybody or anything. and what is the role of this conscious intention if it is not behind determining subse- quent voluntary actions? To address the first question. As described previously (see Figure  3. as an aspect of continuously chang- ing brain dynamics. an image may be regenerated with spontaneous variations into streams of .2  How to Initiate Intentions and Become Consciously Aware The experimental evidence provided by Libet and Soon’s group can be integrated to produce the following hypothesis. then is transmitted downstream to the SMA 1 second before the movement. because our conscious intent that seemingly determines free next actions appears to actually be caused by preceding unconscious brain activities arising a long time before. this implies that there is no room left for free will. Can we freely initiate unconscious brain activ- ity in the frontopolar part of the prefrontal cortex or in the parietal cortex? And second.3. by itself. 4. when memory hosts com- plex relations or connections between images of past experiences. If this is indeed true.  71 Introducing the Brain and Brain Science 71 of physical action. it raises two fundamental questions. but may appear automatically. why do we feel conscious intention for voluntary action only at a very late stage of preparing for action. prior to reported consciousness of such selection. By utilizing fMRI brain imaging. 10). This idea of James leads to the conjecture that continu- ous transitions of images are generated spontaneously along trajectories of brain activation states visiting first one image state and then another iteratively. First. Based on other work done by Churchland and colleagues (2010).72 72 On the Mind consciousness. Then. Ikegaya and colleagues (2004) observed the activ- ities of a large number of neurons in the in vitro hippocampus tissue of rats. Although those motifs seem to appear randomly in many cases. Such spontaneous transitions can be accounted for by observations of the autonomous dynamic shifts of firing patterns in collective neurons in the absence of external stimulus inputs. They also found that those trajectories from the go cue until the onset of movement were mostly repeated for different trials of normal cue response cases (Figure 4. it was seen that the neural activation trajectories fluctuated significantly. They recorded the simultaneous activities of 96 PMC cells of monkeys during the preparatory period for a go-​cue–​triggered visual target reaching task 2 over many trials. In such cases. Such fluctuating trajecto- ries appeared even though the setting at each trial was identical. An exception to the preceding schema was observed during prepar- atory periods leading to generation of failure behaviors such as abnor- mally delayed responses. Their main finding concerns what the authors metaphorically call a “cortical song” wherein various spatiotemporally distributed firing patterns of collective neurons appear as “motifs” and shift from one to another spontaneously. The animals are trained to reach a position that was prior-​specified visually immediately after a go-​cue. . they found that the trajectories of the collective neural activities could be projected into a two-​d imensional axis from 96 original dimensions by a mathematical analysis similar to principal component analysis. how can such fluctuating activities of collective neurons occur? Freeman (2000) and many others have speculated that such spontaneous fluctu- ation might be generated by means of deterministic chaos developed in the neural activity either at the local neuronal circuit level or at larger 2. Using an advanced optical imaging technique. we now also know how fluctuations in activities of collective neurons in the PMC during the preparation of movements can affect the gener- ation of succeeding actual movements. they often repeat in sequences exhibiting some regular- ity. With stronger stimulation. 2010) with permission. they were not consciously aware of the movements generated. cortical area levels. The second question concerning why we become conscious of inten- tion for voluntary action only at a very late stage of preparation for action remains difficult to answer at present. stimulation of the parietal cor- tex created an intention or desire in the patients to move. Desmurget and colleagues (2009) offer us two complementary pieces of evidence obtained in their cortical electrical stimulation study conducted in patients with brain tumors. trajectories for failure cases are shown with thick lines. Conversely.  73 Introducing the Brain and Brain Science 73 (a) Failure (b) Go cue Go cue Movement Pre-target onset Movement onset Pre-target Failure Figure 4.. several reports on cor- tical electrical stimulation in human subjects might open a way to an answer.  Overwriting of 15 trajectories by means of two-​d imensional projection of the activities of 96 neurons in the dorsal premotor cortex area of a monkey during repeated trials of reaching for a visual target task (a) on one occasion and (b) on a different occasion. In both plots. The study employed periop- erative brain stimulations with a bipolar electrode during awake surgery for tumor removal. Desmurget and col- leagues speculated that the parietal cortex might mediate error monitor- ing between the predicted perceptual outcome for the intended action . continuous change in the cortical dynamical state might account for the spontaneous generation. These possibilities are explored in Chapter 10. However. the patients firmly denied mak- ing the movements they actually made. they reported that they had moved their limbs even though they had not actually moved them. without any external causes. But what was interesting was that in the absence of visual feedback. Stimulations of the premotor cortex evoked overt mouth and contralateral limb movements. Given this result. Adopted from (Churchland et al. To sum up then. of various intentions or images for next actions.10. Another difference between the two studies is that more intense stimulation tended to produce actual movement of the same body part when the presupplementary motor area. this is a feeling of imminence for movements of specific body parts in specific ways. (These results also imply that the depotentiation of the parietal cortex without an error signal signifies successful execution of the intended action. This imminent intention for quite specific movements with stimulation of the presupplementary motor area contrasts with the case of parietal stimulation mentioned earlier. This urge to move the limbs is similar to a compulsive desire and in fact the patients reported that they felt as if they were not the agent of the generated movements. By generating a prediction of the overall profile of action in terms of its accompanying perceptual sequence. we can create a hypothesis for how conscious intention to initiate actions is organized in the brain as follows. In other words. the patients could describe precisely the urges evoked. but not the parietal cortex. Actually. for example. to recall Heidegger once again). Putting all of this evidence together. Subsequently. where prediction of perceptual sequences based on this intention is generated. Stimulation at a low current elicited the urge to move a specific body part contralateral to the stimulated hemisphere.6. Then. the next target position for movement predicted by the parietal cortex in terms of body posture or proprioceptive state is sent to the presupplementary motor area. This idea follows the aforementioned assumption about functions of the parietal cortex shown in Figure 4. The intention for action is built up from a vague intention to a concrete one by moving downward through the cortical hierarchy. the left arm was about to move inward toward the body. in which the patients felt a relatively weak desire or intention to move. possibly in the frontopolar part as described by Soon and colleagues. the signal carrying this early form of intention is propagated to the parietal cortex. where a specific motor . the intention generated might be too vague to access its contents and therefore it wouldn’t be consciously accessible (beyond a general mood of anticipa- tion. In the first stage (several seconds before movement onset). was stimulated. the contents of the current intention become consciously accessible. At this stage. the very early form of the intention is initiated by means of spontaneous neuronal state transitions in the prefrontal cortex.74 74 On the Mind and the actual one.) Fried and colleagues (1991) reported results of direct stimulation of the presupplementary motor area in patients as part of neurosurgical evalua- tion. Their electrophysiological experiments with monkeys showed that this area includes neurons responsible for organiz- ing sequences of primitive movements. 1994.  75 Introducing the Brain and Brain Science 75 program for the required immediate movement is generated online. 1998. because it con- tains some unclear parts. 1995). Shima & Tanji. this hypothesis conflicts on a num- ber of points described thus far in this book. however. Second. As related to this problem.4. A big “however” needs to follow this hypothesis. obtained by human brain . Next. as described by Fried and colleagues. This hard problem is contrasted with the so called easy problem in which a target neural function can be understood by its reduction into processes of physical matter. as it is central to this current book.” Analogously. This process is assumed to be essentially unconscious on the basis of the findings of Desmurget and colleagues (2009) mentioned earlier. Yet. and the details of these conflicts are examined in the next section. First. Shima & Tanji. how can we account for causal relationships between consciousness of one’s own actions and neural activity in the parietal cortex? This problem will be revisited repeatedly in later chapters. 2000). it has not been clarified yet how the contents of the current intention become con- sciously accessible in the parietal cortex in the process of predicting the resultant perceptual sequences. their findings conflict with those of Fried and colleagues (1991). However. we must look at some of remaining open problems. David Chalmers speculates that it is nontrivial to account for the quality of human experiences of consciousness in terms of neuroscience data alone. 4. This process generates the feeling of imminence for movements of spe- cific body parts in specific ways.  Deciding Among Conflicting Evidence Let’s remind ourselves of the functional role of the presupplementary motor area described by (Tanji & Shima. This is the “hard problem. there is no way to explain how the firings of these neurons result in the conscious experiences of the subjects. The motor program is sent to the premotor cortex and primary motor cortex to generate corresponding motor commands. Suppose that a set of neurons that fire only at conscious moments are successfully identi- fied in subjects. This is what he calls the hard problem of consciousness (Chalmers. not just those in the premotor cortex but also those in the parietal cortex. ultimately. fire as mirror neu- rons in the case of generating as well as recognizing single actions like grasping food objects. responsible for the prediction of action-​related perceptual sequences? Or. later experiments by Desmurget and Sirigu (Sirigu et al. These mirror neurons in the parietal cortex seem to encode intention for sequences of actions for both one’s own action sequence generation and while observing similar action generation by others. 2009) suggest that it may not be the premotor cor- tex that is involved in conscious intention for action but the parietal cortex. we might be disappointed that circuit-​level mechanisms for the cognitive functions of interest are still not accounted for exactly by current brain research. not for the expectation or desire for whole actions consisting of sequences of elemental movements. the original mirror neuron site? Or.76 76 On the Mind electrical stimulation. What is the primary area for generating voluntary actions? Is the presupple- mentary motor area to be considered the locus for generating voluntary action? Or is it the premotor cortex. as the evidence currently available to us is apparently contradictory. premotor cortex. is it the prefrontal cortex. then electrical stimulation of the presupple- mentary area in humans should likewise evoke desire or expectation for sequences of elementary movements. If Tanji and Shima’s findings for the role of the presupplementary motor area in monkeys hold true for humans. . We may ask then whether some neurons. 2005) did later find mirror neurons in the parietal cortex of monkeys. 2003. as described in the previous section. Another conflict concerns the functional role of the premotor cortex... In fact. 1996). as described in the original mirror neuron paper (Rizzolatti et al.. or parietal cortex. Desmurget et al. Rizzolatti and col- leagues (Fogassi et al. 1996).. Neuroscientists have taken a reductionist approach by pursuing possible neural correlates of all manner of things. The puzzle we have here is the following. These researchers claim that the presupplemen- tary motor area is responsible for generating merely the urge for immi- nent movements. is it the parietal cortex. Finally. as the center for executive control? Although it could be the supplementary motor cortex.. We’ll come back to the pos- sible role of the presupplementary motor area in human cognition in a moment. we simply cannot tell right now. Although the premotor cortex (F5 in monkeys) should host intentions or goals for the next actions to be generated according to the mirror neuron theory put forward by Rizzolatti’s group (Rizzolatti et  al.  Summary This chapter explored how cognitive minds can be mechanized in bio- logical brains by reviewing a set of empirical results. . Thus. First. like the firing of presup- plementary motor area cells in action sequencing or of mirror neurons in the premotor cortex in action generation and recognition.  77 Introducing the Brain and Brain Science 77 They have investigated mappings between neuronal activities in specific local brain areas and their possible functions. earlier stages of the visual system (in the primary visual cortex) are thought to deal with the processing of detailed information in the retinotopic image and later stages to deal with more abstract information process- ing (in the inferior temporal cortex). we reviewed general understandings of possible hierarchical architectures in visual recognition and motor action generation.5. we might need future technical breakthroughs in measurement methods such as simultaneous recording of a good number of neurons and their synaptic connectivity in target functional circuits which are associated with modeling scheme of good quality. such evidence cannot yet tell us the exact mechanisms underlying different types of subjective experience. 4. In the visual pathway. at least not in a fine-​grained way ade- quate to confirming one-​to-​one correlative mappings from the “what it feels like” to specific physical processes. with the hope of clarifying some mechanisms at work in the mind and cognition. It is assumed that the supplementary motor area (SMA) and the premotor cortex (PMC) perform higher level coordination for gen- erating voluntary action and sensory-​guided action by sending control signals to the primary motor cortex (M1) in the lower level. Although clearly the accumulation of such evidence serves to inspire us to imagine how the mind may arise from activity in the brain. How can the firings of specific cells in the presupplementary motor area mechanize the generation of corresponding action sequences? How can the firings of the same pre- motor cells in terms of mirror neurons mechanize both the generation of specific actions and the recognition of the same actions by others? What are the underlying circuitry level mechanisms accounting for both. some have assumed that complex visual objects can be recognized by decomposition into specific spatial combinations of visual features represented in the lower level. The action generation pathway is also presumed to follow hierarchical processes. as well as the feeling of witnessing either? In order to answer questions like these. and that the pre- dictive model can regenerate this sequence. So. And with this. It was also speculated that a particular perceptual sequence can be recognized by means of inferring the corresponding intention state. they are yet speculative. An essential question remained. Also in this chapter. there has arisen some conflicting evidence that does not support the existence of a rigid hierarchy both in the visual recognition and in action generation. but many important questions about the nature of the mind remain to be answered. we have found that neuroscientists have taken a reductionist approach by pursuing possible neural correlates of all man- ner of things. Furthermore. We entertained the hypothesis that the pari- etal cortex may host a predictive model that can anticipate perceptual outcomes for actional intention encoded in mirror neurons. Although the accumulation of such evidence can serve to inspire us to hypothesize how the normal functioning brain results in the feeling of being con- scious. How is intention itself set or gener- ated? This question is related to the problem of free will. A hallmark of this view is that action might be generated by the dense interaction of the top-​down proactive intention and the bottom-​up recognition of perceptual reality. neurological evidence alone cannot yet specify the mechanisms at work. and the parietal cortex significantly before individuals become consciously aware of the decision. the prefrontal cortex. How might we see neural correlates for our conscious experi- ence? Suppose that we might be able to record all essential neuro- nal data such as the connectivity. we have seen that not one.78 78 On the Mind However. The first question concerns how “unconscious” neural activities for decisions are initiated in those related regions. We found evidence for this new approach in the review of recent experimental studies focusing on the functional roles of the parietal cortex and mirror neurons distributed through differ- ent regions of the brain. . The second question concerns why conscious awareness of free decisions is delayed. synaptic transmission efficiency. They have investigated mappings between neuronal activ- ities in specific local brain areas and their possible functions. We reviewed findings that neural activities correlated with free decisions are initiated in various regions including the SMA. we next examined a new way of conceiving of the processes at work in which action generation and sensory recogni- tion are inseparable. These findings raise two questions. Although we have provided some possible accounts to address these questions. we showed how this portrait is analogous to to Merleau-​ Ponty’s philosophy of embodiment. The second focus con- cerns the embodiment of the cognitive processes. By taking seriously limitations inherent to the empirical neuroscience approach. . Although we would find various interesting correlations in such massive datasets. The first is to use dynam- ical systems perspectives to understand various complicated mecha- nisms at work in cognition. 2000.) The next chapter provides an introductory account that consid- ers such problems. which were briefly described in the previous chapter.g. a synthetic modeling approach that attempts to understand possible neu- ronal mechanisms underlying our cognitive brains by reconstructing them as dynamic artifacts.  79 Introducing the Brain and Brain Science 79 and neuronal firings of all related local circuits in the future. this book now begins to explore an alternative approach. for instance. they would still just be correlations. The dynamical systems approach is effective in articulating circular causality. Will this enable us to understand the mechanisms behind all of our phe- nomenological experiences? Probably not. Can we understand the mechanisms of a computer’s operating system (OS) just by putting electrodes at various locations on the mother board circuits? We may obtain a bunch of correlated data in relation to voltages but probably not enough to infer the principles behind the workings of a sophisticated OS. The synthetic modeling approach described in this book has two complementary focuses.. The role of embodiment in shaping cognition is crucial when causal links go beyond brains and establish cir- cular causalities between bodies and their environments (e. Freeman. not proof of causal mechanisms. like the cor- relations between synaptic connectivity and neuronal firing patterns or those between neuronal firing patterns and behavioral outcomes. 80 .  This statement was found on his blackboard at the time of his death in February 1988. I cannot understand. By this line of reasoning. Although we don’t yet know exactly their organizing principles. then. —​Richard Feynman1 Conversely. thus: I can understand what I can create. we should begin by deriving the most likely ones through a thorough survey of results from neuroscience. the plan is to put some computer simulation models of the brain into robot heads and then examine how the robots behave as well as how the neural activa- tion state changes dynamically in the artificial brains while the robots interact with the environment. 81 . we should know its orga- nizing principles. we might be able to understand the cognitive mind by synthesizing it. This seems to make sense because if we can synthesize something.  81 5 Dynamical Systems Approach for Modeling Embodied Cognition Nobel laureate in physics Richard Feynman once wrote on the chalk- board during a lecture: What I cannot create. The clear difficulty involved in doing this is how to build these brain models. But how can we synthesize the mind? Basically. 1. 1991). especially Gibsonian and Neo-​Gibsonian approaches have been well incorporated into dynamical system theories. Rather.82 82 On the Mind psychology. computa- tional aspects in cognitive science. let’s assume that the mind is a product of emergent processes appearing in the structural interactions between the brain and the envi- ronment by means of sensory–​motor coupling of a whole. we will consider particular neural network models as . Readers will see that some psychological views. After this review. Moreover. If such emer- gent phenomena observed in experiments correspond to various bodies of work including empirical observations in neuroscience. Many phe- nomena emergent from embodied cognition can be efficiently described in the language of dynamical systems. it would be great if just a small set of principles in the model could account for numerous phenomena of the mind through their synthesis. to articu- late the processes essential to cognition as we experience it and ideally nothing more. are dominated by nonlinear dynamics for which numerical solutions cannot be obtained analytically. we can examine neural activation dynamics (of a brain model) and behaviors (of such embrained robots) as robots attempt to achieve goals of cogni- tive tasks designed by experimenters. Now. or embodied cognition (Varela et al. These include psychological studies focusing on embodiment and “new-​trend” artificial intelligence robot- ics studies exemplifying behavior-​based robotics involving the synthe- sis of embodied cognition. This is the goal of the synthetic approach. wherein the mind is considered a nontrivial phenom- enon appearing as a result of such interactions. This comes from the fact that all interactions that occur within the model brains. Subsections of the current chapter will explore the idea of embodied cognition by visiting different approaches taken so far. It is not trivial to anticipate—​dare we say guess—​what sorts of phe- nomena might be observed in such experiments even though the prin- ciples used in engineering relevant brain models are well defined. This assumption refers to the embodied mind. embodied agent through behavior. the presumed principles behind the models would seem to hold. and have thus pro- vided useful insights guiding behavior-​based robots and neurorobots. as we will see. as well as between them and the environment by circular causal- ity. and cognitive science.. we should expect that such robotics experiments might evidence nontrivial phenomena that are not to be inferred from formative principles themselves. In robotics experiments. and reports from phenomenological reduction. the time development of the system state is obtained by iterating the mapping of the current state at t to the next state at t+1 starting from given initial state. and with P as a set of parameters of interest that characterize the function G(  ): X t +1 = G(X t . the time development of the dimensions in the system can be described by the following difference equation (also called a “map”): ( )  x1t +1 = g 1 x1t .…. 1 can be rewritten with N dimensional state vector X t . xt2 .  83 Dynamical Systems Approach for Embodied Cognition 83 abstractions of brains. But. x tN  (  xt2+1 = g 2 x1t . 1)    N N (  xt +1 = g xt . I would like to start with a very intuitive explanation. First.…. Eq.1. and suppose that this system can be described at any time as exhibiting an N dimensional system state where the ith dimensional value of the current state is given as x ti . the next section presents an introduction to dynamical systems’ theories that lay the groundwork for the synthetic modeling studies to follow. however.…. Let’s assume that there is a dynamical system. and can be deter- mined solely by way of all dimensional values at the current time step. When x ti +1 as the ith dimensional value of the state at next time step. xt2 . 5. 2) A given dynamical system is often investigated by examining changes in time-​development trajectories versus changes in the representative . The first section provides an introduc- tory tutorial on general ideas of dynamical systems. readers should note that this is not the end of the story: Chapter 6 discusses some of the crucial ingredients for synthesizing the “mind” that have been missed in conventional studies on neural network model- ing and behavior-​based robotics. and then consider a set of neurorobotics studies by using those models that demonstrate emergence through synthesis by capturing some of the essence of embodied cognition. xt . xtN  ) (Eq. P ) (Eq.  Dynamical Systems Here. xt 1 2 ) N Here. 2 is given as a nonlinear function.1c). The fourth one is a chaotic attractor (a “strange attractor”) in which the tra- jectory exhibits infinite periodicity and thereby forms fractal structures (Figure 5. In such cases. Dynamical systems can be described also with an ordinary differen- tial equation in continuous time with X as a vector of system state. Finally.1d). the trajec- tory converges to a cyclic oscillation pattern with constant periodicity.1b). X = F ( X . In most cases. representing stable state behaviors charac- terizing the system. The third one is a limit torus that appears when there is more than one frequency involved in the periodic trajectory of the system and two of these frequencies form an irrational fraction. in some cases multiple local attractors can coexist in the same state space as illustrated in Figure 5. 3) The exact trajectory in continuous time can be obtained also by integrat- ing the time derivative from a given dynamical state at the initial time. outside of attractors or invariant sets are transient states wherein trajectories are variable. If the function G(  ) in Eq. they are no longer variable and are instead determined. they become invariant trajectories. the trajectory is no longer closed and it exhibits quasi-​periodicity (Figure 5.1a). and with F(  ) as a nonlinear dynamic function parameterized by ∂t P as shown in Eq. P ) (Eq. the time development of the state cannot be obtained analytically. In this case. In this type of attractor. with X as a vector of the time derivative of the state (it can be also written as ∂X ). The easiest attractor to envision is a fixed point attractor in which all dynamic states converge to a point (Figure 5. Attractors are basins toward which trajectories of dynamical states converge.1e. Attractors can be roughly categorized in four types as shown in Figure 5.3. That is. the . The second one is a limit cycle attractor (Figure 5. It can be obtained only through numerical computation as integration over time from a given initial state X0 and this computation can only be executed with the use of modern digital computers. On the other hand. The structure of a particular dynamical system is characterized by the configuration of attractors in the system.1a–​d.84 84 On the Mind parameter set P. which determines the time evolution profiles of different states. the trajectories of time development can become complex depending on the nonlinearity. after trajectories converge (perhaps after infinite time). An attractor is called an invar- iant set because. The logistic map is written in discrete-​time form as: x t +1 = a x t (1 − x t ) (Eq. its behavior is nontrivial as will be seen in the following. x n as shown in the diagram at the left of Figure 5. 1 with a one-​ dimensional dynamic state.1. and (d) chaotic attractor.1. 4) Here. we look at the case of discrete time dynamics in detail. x 0 . (b) limit cycle attractor.  Different types of attractors. . which was introduced by Robert May (1976). it will recursively gen- erate a trajectory x1. … . If a particular value is taken for the initial state. x2 . as a simple illustrative example of Eq. Next. respec- tively. 5.1e a state trajectory starting from the left side and the right side of the dotted curb will converge to a fixed point and a limit cycle. (a) Fixed point attractor.2a. (c) limit torus characterized by two periodicities P1 and P2 which form an irrational fraction. attractor to which the system converges depends on the initial state. In Figure 5.. x t is a one-​d imensional dynamic state and a is a parameter. Even with a one-​d imensional dynamic state.1  Discrete Time System Let us examine the so-​called logistic map. (e) Shows multiple attractors consisting of a fixed point attractor and a limit cycle attractor. Note that all four types of attractors are illustrated in terms of continuous time dynamical systems.  85 Dynamical Systems Approach for Embodied Cognition 85 (a) (b) (c) x P2 P1 (d) (e) x Figure 5. 4 3.8 0.2b left.2 x0 x3 x1 x2 xt 0. where an invariant set means a set of points within the convergence tra- jectory as mentioned previously. a limit cycle alternating between 0.0 3.). 3.8 4.6 1.3 0 10 20 30 0 10 20 30 0 10 20 30 40 50 t t t Figure 5.0 1. Now.52 and 0. For this purpose.2 a = 3. and therefore this point is a fixed-​point attractor (see Figure 5.2b middle.4 2.0.4 0. . (b) Time developments of the state with different values of a where a fixed point attractor.6 x 0.2. This diagram shows an invariant set of attractors for each value of a.0 x x x 0. (a) Dynamic iteration corresponding to a logistic map is shown on the left and its bifurcation diagram with respect to the parameter a is shown on the right. right.61 from any initial state.6 2.3 0.0 a (b) a = 2. For example. With a set to 3.) When a is increased to 3. the fixed-​point attractor bifurcates into a limit-​cycle attractor with a period of 2.6 3. respectively.2a. a bifurcation diagram of the logistic map is shown in Figure 5. when a is set to 2.86 86 On the Mind (a) 1.0 1.0 xt+1 xt+1=xt 0. the trajectory of x t converges toward a point around 0.2 3.2.8 3.0 2.80 appears (see Figure  5.6.  A logistic map. and 3.6 a = 3.2. and chaotic attractor appear from left to right for a = 2. let’s examine how the dynamical structure of a logistic map changes when the parameter a is varied continuously.3 0.6. limit cycle attractor.6. Two adjacent initial states denoted by a dot and a cross are mapped to two points that are slightly further apart from each other after the first mapping. “H” for values greater than 0.0 are mapped to x1 values between 1. In this case.0.) The time evolutions of x starting from different initial states are plotted for these values of a. the logistic map gener- ates chaos that covers the range of x from 0.5 is mapped to x1 values between 0.0 (again with magnification. This sensitivity to initial conditions determines the ability of chaos to generate nonrepeatable behaviors even when a negligibly small per- turbation is applied to the initial conditions. If a is set to 4. but in the opposite direction). as can be seen in Figure 5. We’ll turn now to look briefly at a number of characteristics of chaos.  A  limit cycle alternating sequentially between 0. 0. This essentially represents the process of stretching and folding in a single mapping step of the logistic map. whereas x 0 values between 0.5. It should be noted that no periodicity is seen in the case of chaos.88 appears when a is set to 3.51. whereas when a is increased to 3.2a. it is known . where it is clear that the transient dynamics of the trajectory of x converge toward those fixed-​point. the limit cycle with a period of 2 bifurcates into one with a period of 4. the distance between these two trajectories increases exponentially as iterations progress.  87 Dynamical Systems Approach for Embodied Cognition 87 and when a is further increased to 3. and chaotic attrac- tors. the range of values for x 0 between 0.82.0 and 0.3b. further bifurcation takes place from a limit cycle to a chaotic attractor characterized by an invariant set with an infinite number of points (see Figure 5. look at an interesting relation between chaotic dynamics and symbolic processes.43. If we observe the output sequence of the logistic map and label it with two symbols.0 to 1.5 and 1.0 and 0.5. when two trajectories are generated from two initial states separated by a negligibly small distance in phase space. This peculiarity of chaos can be explained by the process of stretching and folding in phase space as illustrated in Figure 5. the distance between the two states increases exponentially.3a shows an example of such development. Further. When this mapping is repeated n times. This iterated stretching and folding is considered to be a general mechanism for generating chaos.0.2b right.5 and “L” for those less than or equal to 0.60.0 and 1. 0.0 as can be seen in Figure 5.0 with magnification.” When the parameter a is set at 4.3b.38. One of the essential characteristics of chaos is its sensitivity with respect to initial conditions. resulting in the complex geometry generated for x n by means of iterated stretching and folding. Figure 5. and 0. In chaos. we get probabilistic sequences of alternating “H” and “L. limit-​cycle. 3. Devaney 1989).88 88 On the Mind (a) 1 0. which provides a theorem to connect real number dynamical systems and dis- crete symbol systems.8 0.  Initial sensitivity of chaotic mechanisms.9 0. . a different form of a probabilistic FSM with a different number of discrete states and different probability assignments for output labels is reconstructed for each.5 0. (a) Distance between two trajectories (represented by solid and dashed lines) starting from their initial states separated by a distance of ϵ in phase space.7 x 0. This can be represented by a one-​state proba- bilistic finite state machine (FSM) with an equal probability output for “H” and “L” from this single state. Distance between the two grows exponentially as time goes by in chaos generated by a logistic map with a set to 3.4 5 10 15 20 25 30 35 40 45 50 t (b) x0 x1 x2 x3 xn 1.0 1st 2nd 3rd 0. If the parameter a is changed to a different value in the chaotic region. (b) The mechanism of generating chaos is by stretching and folding.5 0. 1989. like a coin flip.0 Figure 5. that the logistic map generates “H” or “L” with equal probability with no memory.6 0.6. This is called symbolic dynamics (Crutchfield & Young. 1989). This generates the phenomena known as intermittent chaos in which the passing through appears intermittently. When the curve of mapping function becomes tangent to the line of identity mapping. at the ends of window regions in which the periodicity of the attrac- tor moves from finite to infinite (Crutchfield & Young. or sometime after infinite steps.4. . passing through the tangent point could take infinite steps depending on the value of x to enter the passing through. as shown in Figure 5.  89 Dynamical Systems Approach for Embodied Cognition 89 Tangency Xt+1 Xt Figure 5. at the “edge of chaos.” including at the ends of window parameter regions where quite rich dynamic patterns following power law can be observed. Edge of chaos can be observed also under another critical condition when “tangency” exists in mapping of function. It is known that nonlinear dynamic systems in general develop critical behaviors upon exhibiting state trajectories of infinite complexity. These properties of edge of chaos in critical conditions are revisited in later chapters as we examine the behavioral characteristics of neurorobots observed in our experiments. only after sev- eral steps. One interesting observation of logistic maps in terms of symbolic dynamics is that the complexity of symbolic dynamics in terms of the number of states in the reconstructed probabilistic FSM can be infinite especially in the parameter region at the onset of chaos. The passing through of the state x slows down in the vicinity of the tangency point.  Tangency in nonlinear mapping.4. 1. and c). accompanied by a one-​time folding and stretching.5e). In the Rössler attractor. the sensitivity of this system to initial conditions becomes apparent in the same way as with the logistic map.90 90 On the Mind 5. If we conduct a phase space analysis on this system. let’s examine the case of continuous time. y) converge toward three different types of attractors (fixed point. represented by Eq. and the chaotic attractor shown in (c) is the Rössler attractor. limit cycle. Such an attractor is called a global attrac- tor.1. b.5d). As shown in Figure 5. Importantly. This line segment is folded and stretched once during a single rotation. which is known as a Poincaré section (Figure 5. It should be noted that in each case the trajectory converges to the same attractor regardless of the initial state. or chaotic) depending on the values of the parameters. I will argue that one emergent property of nonlinear dynamical systems is the appearance . 1976) as a simple example that can be described by the following set of ordinary differential equations: x = − y − z y = x + ay (Eq. If this process is iterated. b. and no inputs. we’ll see a line segment consisting of an infinite number of trajectory points. We’ll take the Rössler system (Rössler. continuous trajectories of the dynamical state projected in the two-​d imensional space (x.2  Continuous-​Time Systems Next. and c). we can see different dynamical structures appearing for different parameter set- tings (of a. y. 5. The phenomena corresponding to these changes in the dynamical structure caused by parameter bifurcation are quite similar to those observed in the case of the logistic map.5. If we take a section of the sheet. The mechanism of generating chaos with the Rössler attractor can be explained by the process of stretching and folding previously mentioned. and z). three parameters (a. 5. which is mapped again onto the line segment (see Figure 5.3  Structural Stability This subsection explains why structural stability is an important char- acteristic of nonlinear dynamical systems. a bundle of tra- jectories constituting a sheet rotates in a counterclockwise direction. 5) z = b + z(x − c ) This continuous-​time nonlinear dynamical system is defined by a three-​ dimensional state (x. b = 0. b = 0. An important point here is that attractors as trajectories of steady states cannot exist by themselves in isolation. In other words.2. c = 5. c = 5. b = 0. the vector field itself cannot tell us what the attractor looks like.2. Rather.5. through the transient process of converging toward the attractor.1.0). . (a) A fixed-​ point attractor (a = −​0.2.7). The attractor emerges only after a certain number of iterations have been performed. Illustrations of (d) the Poincaré section and (e) the process of folding and stretching in the Rössler attractor that accounts for mechanism of generating chaos. A  particular equation describing a dynamical system can indicate the direction of change of state at each local point in terms of a vector field. as illustrated in Figure 5. they need to be “supported” by transient parts of the vector that converge toward these attractors. of a particular attractor configuration for any given dynamical system.6a. and (c) a chaotic attractor (a = 0. transient parts of the vector flow make attractors stable. c = 4.1.  Different attractors appearing in the Rössler system.2.7).  91 Dynamical Systems Approach for Embodied Cognition 91 (a) (b) (c) (d) Poincare section (e) Fold and stretch Figure 5. (b) a limit-​cycle attractor (a = 0. However. such as the frictionless spring-​mass system described by Eq. the amplitude of oscillation will change immediately. Sometimes I ask students to give me an example of a system that generates oscillation patterns and a common answer is a sinusoidal function or a harmonic oscillator. 6) x = v Here. let’s take a counterex- ample in terms of a system that is not structurally stable. The equation represents a second order dynamic system without damping terms. Most rhythmic patterns in biological systems are thought to be gener- ated by limit-​cycle attractors because of their potential stability against . such patterns are not structurally stable because if we apply force to the mass of the oscillator instantaneously. v) space. we will see that the vector flow describes concentric circles where there is no convergent flow that constitute a limit-​cycle attractor (see Figure 5. v is its velocity.6b). This is the notion behind the structural stability of attractors. and the original oscillation pattern will never be recovered automatically (again.92 92 On the Mind (a) 3 (b) 3 2 2 1 1 V 0 V 0 –1 –1 –2 –2 –3 –3 –3 –2 –1 0 1 2 3 –3 –2 –1 0 1 2 3 X X Figure 5. However. and k is the spring coefficient. v) in which the vector flow converges toward a cyclic trajectory. (b) A vector field for a harmonic oscillator in which its flow is not convergent but forms concentric circles. it is frictionless). Indeed. If the vector field is plotted in (x. To pro- vide a more intuitive explanation of this concept. mv = − k x (Eq. a sinusoidal wave function is also simply the trace of one point on a circle as it rolls along a plane.  Vector flow. A frictionless spring-​mass sys- tem can indeed generate sinusoidal oscillation patterns. x is the one-​d imensional position of a mass m. (a) Appearance of a limit-​cycle attractor in a vector field of a particular two-​d imensional continuous dynamical system with the system state (x. 6.6. swimming.1  The Gibsonian Approach A concept central to this approach. the structural stability of dynamic patterns in terms of physical movements or neural activity in biological systems can be achieved through attractor dynamics by means of a dissipative struc- ture. On the other hand. has signifi- cantly influenced not only mainstream psychology and philosophy of .2. a harmonic oscillator without a dampening term. this results in the formation of an attractor of the limit cycle type (or it could also result in the generation of chaos under certain con- ditions). In short. Such limit-​cycle attractor dynamics in real physical systems are generated by nonlinear dynamical systems called dissipative systems. Further. locomotion. There is no dampening term to dissipate energy from the system. A  dissipative system consists of an energy dissipation part and an energy supply part. as is described briefly in the next section. and many oth- ers. the particular attractors appearing in different cases are the products of emergent properties of such nonlinear (dissipative) dynamic systems.2. it returns to the original attractor region by means of automatic compensation by dissipating an appropriate amount of energy corresponding to the input energy. followed by Neo-​Gibsonian variants and infant developmental psy- chology using the dynamical systems’ perspectives. is not a dissipative system but an energy conservation system. 5. breathing. Once perturbed. Energy can be dissipated by dampening caused by friction in mechanical systems or by electric resistance in electrical circuits. When a larger or smaller amount of energy is supplied momentarily due to a perturbation from an external source.  93 Dynamical Systems Approach for Embodied Cognition 93 perturbations. Neo-​Gibsonian psychologists have taken advantage of these interesting dynamical properties of dissipative systems to account for the generation of stable but flexible biological movements. its state trajectory will not return to the original one. known as affordance. such as that shown in Eq. 6. However. If the amounts of energy dissipation and energy supply during one cycle of oscillation are balanced. The next section explores such concepts by introducing the Gibsonian approach first. These include central pattern generators in neural circuits for the heart beat.  Gibsonian and Neo-​Gibsonian Approaches 5. the state trajectory deviates and becomes transient. Indeed. A similar example. the role of the controllers is to preserve perceptual constancy. which allows us to walk down the middle of the corridor without colliding with the walls (see Figure 5. a chair affords sitting on it. Relationships between actors and objects within these environments afford these agents opportunities to generate adequate behaviors. J. adjusting the direction of flight so that the focus of expansion (FOE) in the visual optical flow becomes superimposed on the target (see Figure 5. has been interested in the role of embodiment in generating situated behaviors from the Gibsonian perspective. is that we walk along a corridor while recognizing the difference from zero of the optical flow vectors along both sides of the corridor. affordance was defined as “all possibilities for actions latent in the environment. Optical flow is the pattern of motion sensed by the eye of an observer. In general. By considering that optical flow information can be used to perceive one's own motion pattern and to control one's own behavior. and a door knob affords pulling or pushing a door open or closed free from the resistance afforded by the door's locking mechanism. Gibson (1979). He analyzed how an outfielder positions himself to catch a fly ball as an example (Clark. which ensures that perceptual variables always converge to a constant state.94 94 On the Mind the mind. but also synthetic modeling studies including artificial intel- ligence and robotics. He illustrated this concept with the example of a pilot flying toward a tar- get on the ground. affordance can be under- stood as behavioral relations that animals are able to acquire in interac- tion with their environments. the distance between the FOE and target.7a). and the vector difference between the optical flows for both walls—​and that body movements are generated to keep these perceptual variables at constant values. In the original theory of affordance proposed by J. A simple dynamical system theory can show how this constancy may be maintained by assuming the existence of a fixed point attractor. By assuming the existence of coupled dynamics between the environment and small controllers inside the brain. Andy Clark. 1999). For example. Many of Gibson's considerations focused on the fact that essential information about the environment comes by way of human process- ing of the optical flow. Gibson came up with the notion of affordance constancy.” Put another way. a philosopher in Edinburgh. These examples suggest that for each behavior there is a crucial perceptual variable—​in Gibson’s two examples. closer to everyday life. this action is thought . This account was inspired by his own experience in training pilots to develop better landing skills during World War II.7b). Kugler.7. First. 1995). Clark explains that the task is to maintain. and distance of the ball.2  Neo-​Gibsonian Approaches In the 1980s. They considered that the ideas of dissipative structures. especially concerning limit cycle attractor dynamics. so-​called Neo-​Gibsonian psychologists such as Turvey. swimming. there is actually a simple strategy to catch it: If the outfielder continues to adjust his movement so that the ball appears to approach in a straight line in his visual field. The important theoretical ingredients of these ideas are entrainment and phase transitions. by making multiple. and hand waving. can serve as a basic principle in organiz- ing coherent rhythmic movement patterns such as walking. he can catch the fly ball easily. This means that coordination dynamics like this naturally appears under relatively simple principles. Redrawn from Gibson (1979). simulated Cartesian coordinate system.2. 5. as described briefly in the previous section. (a) Flying while superim­ posing the focus of expansion on the target heading and (b) walking along a corridor while balancing optical flow vectors against both side walls. speed. real-​time adjust- ments to the running motion. a kind of coordination between the inner and the outer. coupled oscillators that initially oscillate with .  Gibson’s notion of optical constancy. ongoing.  95 Dynamical Systems Approach for Embodied Cognition 95 (a) (b) Figure 5. breathing. By maintaining this coordination for perceptual constancy. However. and Kelso started investigating how to achieve the coordination of many degrees of freedom by applying the ideas of dissipative struc- tures from nonlinear dynamics to psychological observations of human and animal behavior (see the seminal book by Scott Kelso. acceleration. such as perceptual constancy. instead of through complicated computation involving representation in an objective. to require complicated calculations of variables such as the arc. the ball falls down to him eventually. . 1988). Right index finger Left index finger energy amp time 180° 0° Phase difference energy Frequency amp time 180° 0° Phase difference energy amp time 180° 0° Phase difference Figure 5.8). Let’s look at this in more detail by reviewing a representative experi- mental study conducted by Kelso and colleagues (Schoner & Kelso. subjects were asked to wiggle the index fin- gers of their left and right hands in the same direction (different mus- cles activated. Second. When the metronome was speeded up gradually. the characteristics of this global synchrony can be drastically changed by a shift of an order parameter of the dynamic sys- tem by means of phase transition. converge to a global synchrony with reduced dimen- sionality. In the experiment.8. by mutual entrainment under cer- tain conditions. The panel on the left-​hand side shows how oscillation coordination between right and left index fingers changes when the leading frequency is increased. It was observed that the relative phase changed 180 degrees to 0 degrees sud- denly (see the left-​hand side panel in Figure 5.96 96 On the Mind different phases and periodicities can. antiphase) in synchrony with a metronome. in-​phase).  The phase transition model by Kelso (1995) for explaining the dynamic shifts seen in bimanual finger movements. what happened was that the fin- ger movement pattern suddenly switched from the same direction to the opposite direction one (same muscles activated. The panel on the right-​hand side shows the corresponding change in the energy landscape. switching the system state suddenly from the anti-​phase to the in-​phase. is more dif- ficult to maintain (at least without lots of practice) than one or the other behaviors. This result accords with a central notion in Neo-​Gibsonian approaches.3  Infant Developmental Psychology Neo-​Gibsonian theories helped to give birth to another dynamic system theory that accounts for infant development. (Thelen & Smith.2. When a hypothetical energy landscape is computed for the movement patterns along with the order parameter of the metronome speed (see the right-​hand side panel in Figure 5. a walk-​run. but by implicit synergy among local elements including neurons. Kelso and colleagues showed by computer simu- lation that the observed dynamic shift is due to the phase transition from a particular dynamic structure self-​organizing to another.  97 Dynamical Systems Approach for Embodied Cognition 97 After this experiment. Ester Thelen and Linda B. muscles. It is common experience that the middle state. a diverse range of phenomena characterized by similar shifts in animal and human movement patterns appear very effectively explained in terms of phase transitions.8). 9. that: We invoke Gibson’s beliefs that the world contains information and that the goal of development is to discover relevant information in order to make a functional match between what the environment affords and what the actor can and wants to do. given changes in an order parameter of the system (the speed of metronome in this exam- ple). Good examples include the dynamic shift from trot to gallop in horse locomotion given a change in the system parameter “run- ning speed.” as well as the shift from walk to run in human locomotion. Indeed. A Dynamic Systems Approach to the Development of Cognition and Action. Such dramatic shifts in dynamic system state such as those seen in the bimanual finger movement illustration can be explained by means of the phenomena of phase transition. 5. Smith wrote in their seminal textbook. Introduction) . However. and skeletal mechanics. the antiphase becomes stable with its energy minimum state when the metronome speed is low. the antiphase becomes unstable as the metronome speed increases (the parameter introduces too much energy into the system) and the behavior is modulated toward the realization of a more stable system and corre- sponding energetic minimum. 1994. and that these behaviors repre- sent emergent characteristics of dissipative structures. that behaviors are organized not by an explicit central com- mander top-​down. p. One more example is perseverative reaching observed in the so-​ called A-​not-​B task as originally demonstrated by Jean Piaget. after several months of correct usage.e. when provided with an explicit cue to indicate the correct location). the movement patterns of crawling become unstable. Gershkoff-​Stowe and Thelen (2004) provide a remarkable account of so-​called “U-​shaped” development..98 98 On the Mind They suggest that development is better understood as the emergent product of many decentralized and local interactions occurring in real time between parts of the brain.” This is known as perseverative reach- ing. However. she or he shows walking-​like motion with alternate stepping. Following this line of thinking.9. This perseverative behavior is not observed in infants older than 12 months of age.” if there is a delay between hiding and allowing the child to reach. For example. often incorrectly use words like “foots” and “goed. Recovery is repeated several times at the first location “A” before the experi- menter switches the hiding place to the second location “B. When a newborn baby is held so that the feet lightly touch a solid surface. 2003). rather than as sequences of events preprogrammed in our genes. 8-​to 10-​month-​old infants are cued to recover a hidden object from one of two identical hiding places (see Figure 5. known as the father of developmental psychology. Another example is the walking reflex. a phe- nomenon whereby previously performed behaviors regress or disappear only to recover or reappear with even better performance later on. An interesting observation in the not-​hidden condition is that infants around 5 months old are correct (around 70% success rate) at location “B” and show less perseveration than infants around 8 months old who are incorrect (around 20% success rate). when they newly acquire the movement patterns of walking upright. However. the body. crawling is a stable behavior for infants for several months. this reflexive behavior is scarcely observed after a few months and does not reappear until just prior to walking. They eventually resume using these words correctly. A typi- cal example can be seen in language development around 2 or 3 years of age when children.” Although the infant watches the toy hidden at the new location “B. . infants robustly return to the original location “A. In this task.” a phenomenon known as overregularization. illustrated in Figure 5.9). This reaching can even be observed in the not-​hidden toy case (i. and the environment. Smith and Thelen hold that this happens not as the result of a genome preprogram but as the result of an efficient solution generated through self-​organization (Smith & Thelen. 2004. including both mental and behavioral components.’ but how the com- ponent processes can reorganize to produce such dramatic nonlin- earities in performance” (Gershkoff-​Stowe & Thelen.  Piaget’s A-​not-​B task. The infant then repeatedly retrieves the object. In the case of perseverative reaching. in 1. They write.  16). with a delay between seeing the hiding and retrieval. an attractive object is hidden at location “A” (left-​hand side). Instead. First. in 2 and 3. perceptual. U-​shaped behavior is the result of a continuously changing configuration between mutually interacting components.9. p. from the correct location of “A. or memory system alone. However.” What is the underlying mechanism in these examples of U-​shaped development? Gershkoff-​ Stowe and Thelen (2004) argue that U-​shaped development is not caused by regression or loss of a single element such as one in the motor.  99 Dynamical Systems Approach for Embodied Cognition 99 1 2 3 B A 4 5 Figure 5. the infant fails to retrieve the object at the correct location “B. It was found that the hand trajectories in repeated recov- eries of 8-​month-​old infants become increasingly similar to those of 5-​month-​old infants who are relatively immature in controlling their hand reaching movements.” In 4. “The issue is not how a behavior is ‘lost’ or ‘gets worse. this is not the only cause. although it can be considered that repeated recoveries from location “A” can reinforce a memory bias to select location “A” again upon next reaching. It was also found that changing the hand trajectory by adding weights to the infants’ arms significantly . the object is then hidden at location “B” (right-​hand side) while the infant attends to this. The next subsection looks at the development of a cognitive competency. The first stage starts with the sensory-​reflex response of newborns. A  drastic differentiation in development comes with deferred imitation at around 8 to 12 months in the fourthth stage.” Smith and Thelen (2003) explain that infants who have had more experience exploring environments by self-​locomotion show greater visual attention to the desired object and its hidden location. 2004). Jean Piaget proposed that imitation in infants develops through six discrete stages until 18 to 24 months of age (Piaget. such as attention switch and atten- tion maintenance that allow for tracking and preserving the alter- native cue appearing in the second location “B. This account has been supported by simulation studies using the dynamic neural field model (Schoner & Thelen. Piaget emphasized this change by . The point here is that the mutual rein- forcement of the memory bias and the persistent trajectories in the reaching movement through the repeated recoveries result in form- ing a strong habit of reliable perseverative reaching. And again.” He considers that even discrete decisions for selecting actions might be delivered through the process of gradu- ally settling partially active and competing neural activities involved with multiple psychological processes. This perseverative reaching is at its peak at 8 months of age and starts to drop off thereafter as other functions mature to counter it. the emergence of U-​shape development is a product of dynamic interactions between multiple contingent processes both internal and external to infants (Gershkoff-​Stowe & Thelen. This account of how reaching for either “A” or “B” is determined by infants is parallel to what Spivey (2007) has discussed in terms of the “continuity of minds. 2006). an ability to reproduce a modeled activity that has been observed at some point in the past emerges. 1962).100 100 On the Mind decreased the perseveration.2. namely imitation. Here. 5. which has been considered to play an important role in the cognitive develop- ment of children.4 Imitation It has been considered that imitation and observational learning are essential for children to acquire a wide range of behaviors because learn- ing by imitation is much more efficient than learning through trial and error by each individual alone. which is followed by repletion of some repertories by chance in the second stage. innate sensory–​motor mapping can generate aforementioned imitative behaviors by means of automatic responses. Finally in the third stage. the initiator left the object and turned to imi- tate the partner's ongoing behavior. In the first stage. Even Piaget believed that facial imitation could appear only after 8 months of age. In most cases.  101 Dynamical Systems Approach for Embodied Cognition 101 suggesting that this stage marks the onset of mentalization capabilities in infants. some recent studies have pursued the suspicion that the roots of the human cogni- tion may be found in the analysis of early imitation. Jacqueline Nadel (2002) proposed that imitation is a means to communicate with others. A  seminal study by Meltzoff and Moore (1977) showed that human neonates can imitate facial gestures of adults such as tongue protrusion. infants experience regular relationships between their mental states and actions generated repeatedly. A typical example is the appearance of pretend play. In these cases. however. a child pretends to call by using a banana instead of a real phone after observing the actions of his or her parents. the partner infant took the object and imitated its usage. Nadel observed a group of prever- bal infants in a natural social play setting involving a type of frequently observed communicative interaction. . in newborns. Although Piaget's emphasis was cognitive development toward men- talization or symbolism that appears in the later stages. In the second stage. For example. Sometimes. the partner refused to do so or ignored the other. Meltzoff (2005) has hypothesized a “like me” mechanism that connects the perceptions of others “like me” with one's own capacities. This finding was nontrivial because it implies that neonates can match their own unseen behaviors with those demonstrated by others. Although the exact mechanisms enabling these imitative behaviors in neonates is still a matter of debate. Typical turn taking was observed when an infant showed another infant an object similar to the one he or she was holding. and so have focused on how neuronal mechanisms of imitation appear in much earlier stages. infants come to understand that others who act “like me” have mental states “like me. when some symbolic level men- tal representation and manipulation can be observed. and thus associations between them are learned.” On a similar line. turn taking or switching roles among two or three infants. therefore grounding an embodied understand- ing of others’ minds enactive imitation. This mentalization capability is further developed in the sixth stage at around 18 to 24 months. mouth opening. and lip protrusion. 102 102 On the Mind Another remarkable finding by Nadel (2002) was that pairs of pre- verbal infants often exhibited imitation of instrumental activity with synchrony between them. but rather depends on synchronization and rhythm which appear spontaneously in the dynamical processes of sensory–​motor mapping between the perception of others of “like me” and one's own actions. Reproduced from (Nadel. 2001). infants at the very least “know how” to communicate with each other (Andry et al. 2002) with permission. . Nadel and colleagues argue that although immediate imitation generated during behavioral exchanges may not be always an intelligent process as Piaget pointed out. Based on these observations and others. The next section describes a new movement in artificial intelligence and robotics guided by these insights and many others from contemporary developmental psychology.  Preverbal infants exhibit instrumental activities with synchrony during imitative exchange. Figure 5.10. Figure 5.10 shows that when one infant dem- onstrated an unexpected use of objects (carrying an upside-​down chair on his head).. the partner imitated this instrumental activity during their imitative exchanges. This intriguing communicative activity may not require much of mental representation and manipulation or symbol- ism. The uniqueness of the book is its attempt to explore possible brain-​psychological mechanisms for generating behav- ior via synthesis. Each of the three vehicles is equipped with a set of paired sensors on the front left-​and right-​hand sides of its body. 5. The sensory inputs are transmitted to the left and right wheel drive motors at the rear through connecting lines which are analogous to synaptic connections. For example. Braitenberg’s “Law of uphill analysis and downhill invention” suggests that it is more difficult to understand a working mechanism or system just from looking at it externally than it is to create it from scratch. 3.3.1  Braitenberg’s Vehicle Thought Experiments In his book. The vehicle has light intensity sensors to the front on each side that are connected to its corresponding rear motors in an excitatory manner . Another interesting feature of Braitenberg’s book is that all of the synthesis described is done through thought experiments rather than by using real robots or computer simulations—​although many research- ers reconstructed these experiments using actual robots years later. 1984) describing the psychological perspective that led to the behavior-​ based robotics approach. Braitenberg’s thought experiments are simple. Here.  103 Dynamical Systems Approach for Embodied Cognition 103 5. that just a few years before Brooks started his proj- ect. It should be noted. a paradigm shift occurred in artificial intel- ligence and robotics research. Valentino Braitenberg. yet provide readers with valuable clues about the cognitive organization underlying adaptive behaviors. an insight parallel to the quote from Feynman introducing this chapter. we confine ourselves to looking at Vehicles 2. because they offer a good introduction to understanding the behavior-​based approach. however. Let’s begin with Vehicle 2a shown in Figure 5. published a book entitled Vehicles: Experiments in Synthetic Psychology (Braitenberg. Some representative examples of his thought experiments are introduced as follows.3.11. and 4 as representative examples. This shift occurred with the introduc- tion of behavior-​based robotics by Rodney Brooks at MIT.  Behavior-​Based Robotics At the end of the 1980s. a German neuroanatomist. Braitenberg introduces thought experiments concerning 14 different types of vehicles. rather than being excitatory as for Vehicle 2. It is gradually attracted to the light source and finally stops close enough (perhaps depending on friction of the wheels and other factors).11. If a light source is located directly ahead of the vehicle. let’s look at Vehicle 3a which has same-​side inhibi- tory connectivity. Now. Vehicles 2a and 2b are named Coward and Aggressive. respectively. . the deviation will be increased by accelerating the motor on the side closer to the light source. (same-​side excitatory connectivity).11. as shown for Vehicle 2b in Figure 5. let’s suppose that the connectivity lines. On the other hand. This is because the motor on the opposite side of the light source accelerates more and thus the vehicle moves toward the light source. This eventually gener- ates radical avoidance of the light source. This vehicle slows down in the vicinity of the light source. if there is a slight deviation toward the light source. Now Vehicle 3 has drastically different behavior characteristics from Vehicle 2. and vice versa. the motor on the opposite side slows down because it is inhibited by the sensor that perceives a stronger stimulus from the source.  Braitenberg vehicles 2a and 2b (top) and 3a and 3b (bottom).104 104 On the Mind + + + + Vehicle 2a Vehicle 2b – – – Vehicle 3b – Vehicle 3a Figure 5. are inhibitory for Vehicle 3 (Figure 5. if each sensor is connected to a motor on the opposite side (cross-​excitatory connectivity). the vehicle always crashes into the light source. If it deviates to the right. it will crash into the light source by accelerating the motors on both sides equally. then the left wheel is inhibited. If the vehicle deviates slightly to one side from the source.11). However.  First. Simply by adding some nonlinearity to the sensory–​motor mapping functions of the simple controllers. Should we wish to consider emergent behaviors beyond the limits of such thought experiments. However. although this vehicle also slows down in the presence of a strong light stimulus. In the case of Vehicle 3b.  Braitenberg vehicle 4. Braitenberg imagined that repetitions of this sort of approaching and moving away from light sources can result in the emergence of complex trajectories. respectively. Because of the potential nonlinearity in the sensory–​motor response. . It can happen that the vehicle approaches a source but changes course to deviate away from it when coming within a certain distance of it. as illustrated in Figure 5. (a) Nonlinear maps from sensory intensity to motor velocity assumed for this vehicle and (b) complex behaviors that emerge on more complex maps. Eventually. the resultant interactions between the vehicle and the environment (light sources) can become significantly complex.  105 Dynamical Systems Approach for Embodied Cognition 105 (b) (a) V I Figure 5. being thought experiments. which has cross-​inhibitory connectivity. employing the opposite control logic of Vehicle 3a. it gently turns away from the source. The vehicle heads for another light source. Vehicle 4 is added with a trick in the connectivity lines: The relation- ship between the sensory stimulus and the motor outputs is changed from a monotonic one to a non-​monotonic one. the vehicle shifts back toward the source and finally stops to stay in the vicinity of the source. Vehicles 3a and 3b are named Lover and Explorer. we require computer simulations or real robotics experiments.12a.12. this approach is quite limited. as shown in Figure 5.12b. the vehicle will not just be monotonically approaching the light sources or escaping from them. These are very interesting results. in which the motor outputs are directly mapped from the perceptual inputs at each iteration. small robots whose behavior is based on the phi- losophy of nouvelle AI are designed to move first. taking part in physical interactions with their environment and with humans while comput- ing all the necessary parameters in real time in an event-​based manner. On the other hand. The behavior-​based robots made by Brooks and his students use only a simple scheme based on the perception-​to-​motor cycle.” For example.106 106 On the Mind 5. 1990) and “Intelligence without representation” (Brooks. This problem can be attributed to the lack of direct experience. but by implementing an externally imposed artificial purpose. and intelligent behaviors than the conventional compu- tationally heavy robots used in traditional AI research. present his thoughts on what he calls “classical AI” and “nouvelle AI. arguing that a lot of the computation time is spent on logical inference or the preparation of action plans in real-​world tests even before taking a single step or indeed making any movement at all. such as “Elephants don't play chess” (Brooks. Behavior-​based robotics could provide AI researchers and cognitive scientists with a unique means to obtain a view on first-​person experi- ence from the viewpoint of a robot by almost literally putting themselves . 1991). The problem with the classical AI approach is that the representation is prepared not through actual actions taken by the agent (the robot). Brooks also criticizes the tendency of classical AI to be overwhelmed with “representation. The robots then proceed to match what they have sensed through devices such as vision cameras with the stored representation through complicated coordinate transformations for each step of their move- ment as they find their location in the stored Cartesian coordinate sys- tem. typical mobile robots based on the classical AI scheme are equipped with global maps or environment models represented in a three-​d imensional Cartesian coordinate sys- tem. which is related to Husserl’s discussions on phenomenological reduction (see ­chapter 3).2  Behavior-​Based Robots and Their Limitations Returning now to behavior-​ based robotics. realistic.” He has criticized the use of large robots programmed with classical AI schemes. Brooks elaborated on thoughts similar to Braitenberg’s by demonstrating that even small and extremely simple insect-​like robots could exhibit far more com- plex.3. This marked the beginning of behavior-​based robotics research. Argumentative papers published by Brooks.   107 Dynamical Systems Approach for Embodied Cognition 107 inside its head, thereby affording the opportunity to examine the sen- sory flow experienced by the robot. Readers should note that the idea of the perception-​to-​motor cycle with small controllers in behavior-​based robots and Braitenberg vehicles is quite analogous to the aforementioned Gibsonian theories emphasizing the role of the environment rather than the internal brain mechanisms (also see Bach, 1987)). Behavior-​based approaches that emphasize embodiment currently dominate the field of robotics and AI (Pfeifer & Bongard, 2006). Although this paradigm shift made by the behavior-​ based robotics researchers is deeply significant, I  feel a sense of discomfort that the common use of this approach emphasizes only sensory–​ motor level interactions. This is because I  still believe that we humans have the “cogito” level that can manipulate our thoughts and actions by abstract- ing our daily experiences from the sensory–​motor level. Actually, Brooks and his students examined this view in their experiments applying the behavior-​ based approach to the robot navigation problem (Matarić, 1992). The behavior-​based robots developed by Brooks’ lab employed the so-​called subsumption architecture, which consists of layers of com- petencies or task-​specific behaviors that subsume lower levels. Although in principle each behavior functions independently by accessing sensory inputs and motor outputs, behaviors in the higher layers subsume those in the lower ones by sending suppression and inhibition signals to their sensory inputs and motor outputs, respectively. A subsumption archi- tecture employed for the navigation task is shown in Figure 5.13. The subsumption control of behaviors allocated to different layers includes avoiding obstacles, wandering and exploring the environment, and building map and planning. Of particular interest in this architec- ture is the top layer module that deals with map building and planning. Building map & planning Exploring Wandering Avoiding objects Sensation Motor Figure 5.13.  The subsumption architecture used for the robot navigation problem in research by Brooks and colleagues. 108 108 On the Mind This layer, which corresponds to the cogito level, is supposed to generate abstract models of the environment through behavioral experiences and to use these in goal-​d irected action planning. An important remaining problem concerns the ways that acquired models or maps of the environment are represented. Daniel Dennett points to this problem when writing “The trouble is that once we try to extend Brooks’ interesting and important message beyond the simplest of critters (artificial or biological), we can be quite sure that something awfully like representation is going to have to creep in…” (Dennett, 1993, p. 126). The scheme by Matarić (1992) employed a topological graph representation for the environment map consisting of nodes representing landmark types and arrows representing their transitions in the course of traveling (see Figure 2.2). As long as sym- bols understood to be arbitrary shapes of tokens (Harnad, 1990) are used in those nodes for representing the world, they can hardly be grounded in the physical world in a metric space common to the physical world, as discussed earlier. In light of this, what direction of research should behavior-​based robotics researchers pursue? Should we give up involving the cogito level or accept the usage of symbols for incorporating cogito level activities, bearing in mind potential inconsistencies? Actually, a clue to resolving this dichotomy can be found in one of Braitenberg's vehicles, Vehicle 12. Although Braitenberg vehicles up to Vehicle 4 have been introduced in numerous robotics and AI textbooks, the thought experiments beyond Vehicle 4, which target higher-​order cognitive mechanisms, are equally interesting. These higher-​order cog- nitive vehicles concern logic, concepts, rules, regularities, and foresights. Among them, Vehicle 12 examines how a train of thought can be gener- ated. Braitenberg implemented a nonlinear dynamical system, a logistic map (see section 5.1), into the vehicle that enables sequences of val- ues or “thoughts” in terms of neuronal activation to be generated in an unpredictable manner but with hidden regularity by means of chaos. Braitenberg argues that this vehicle seems to possess free will to manip- ulate thoughts, at the least from the perspective of outside observers of the vehicle. We will come back to this consideration in later chapters as the issue of free will constitutes one of the main focuses of this book. So far, we have seen that Gibsonian and Neo-​Gibsonian researchers as well as behavior-​based robotics researchers who emphasize embodied cognition tend to regard the role of the brain as only that of a minimal   109 Dynamical Systems Approach for Embodied Cognition 109 controller. This is because even very primitive controllers like the Braitenberg vehicles can generate quite complex behaviors when cou- pled with environmental stimuli. It is only natural to expect that even higher-​order cognition might emerge to some extent if further nonlin- earity (like that employed in Vehicle 12) or some adaptability could be added to the controller. Now, we begin to consider minimal forms of an artificial brain, namely neural network models that are characterized by their nonline- arity and adaptability, when put into robot heads. Note, however, that these attempts do not accord with our knowledge that the brain is a complex organ, as we have seen in previous chapters. So, let’s contem- plate first how this discordance can be resolved. 5.4.  Modeling the Brain at Different Levels As for general understanding, neural activity in the brain can be described on the basis of processes that occur at multiple levels, start- ing from the molecular level (which accounts for processes such as protein synthesis and gate opening in synapses), the neurochemical level (which accounts for signal transmission), the single cell activ- ity level (which accounts for processes such as spiking), and the cell assembly level in local circuits through to the macroscopic regional activation level measurable with technologies such as fMRI or EEG. The target level depends on the phenomenon to be reproduced. If we aim to model the firing activity of a single cell, we describe precisely how the membrane potential changes as a result of ion flow in a single neuron. If we aim to model neuron interconnection phenomena, as observed in the hippocampus by Ikegaya and colleagues (2004) using optical recording techniques, the model should focus on how spik- ing activity can spread across local circuits consisting of thousands of interconnected neurons. On the other hand, if we aim to model neural processing related to the generation of cognitive behavior, it would not be a good idea to model a single spiking neuron. Rather, such modeling would require the reproduction of interactions between multiple brain regions to simulate the activities of tens of billions of spiking neurons, something that is impossible to perform with computer technology currently available to us. Another problem besides computational power is the operation and 110 110 On the Mind maintenance of such a tremendously complex simulator as well as tech- niques for processing the results of simulations. In fact, using supercomputers to reproduce neural circuits in the brain presents some considerable challenges in terms of making the simulation realistic. At present, we can obtain experimental data about connectivity between different types of neurons by using techniques such as labeling individual neurons with distinctly colored immuno- fluorescence markers appearing in specially modified transgenic ani- mals. These labeled neurons can be traced by confocal microscopy for each section of the sampled tissue, and eventually a three-​d imensional reproduction of the entire system of interconnected neurons can be pre- pared by stacking a number of the images. For example, the Blue Brain project led by Henry Markram (Markram et al., 2015) reconstructed microcircuitory in the somatosensory neocortical of rat, consisting of about 31,000 neurons in a didgital computer model. This simulation coped with neurophysiological details such as reconstruction of firing propertis of 207 morpho-​electrical types of neural cells in the circuit. The project is now attempting to reproduce the entire visual cortex, which consists of about a million of columns each of which consists of about 10,000 cells. If this is achieved, it may also be possible to create a cellular-​level replica of the entire brain! Of course, such an accomplish- ment would provide us with vast amounts of scientific insight. At the same time, however, I wonder how tractable such a realistic brain simulator would be. I  imagine that for a realistic replica of the brain to function properly, it might also require realistic interactions with its environment. Therefore, it should be connected to a physical body of some sort to attain equally realistic sensory–​motor interactions with the environment. It may take several years for the functions of a human-​level brain replica to develop to a sufficiently high level by being exposed to realistic sensory–​motor interactions, as we know that the development of cognitive capabilities in human infants requires a comparably long period of intensive parental care. Also, if a human-​level brain replica must be embedded in various social contexts in human society to ensure its proper development, such an experiment may not be feasible for various other reasons, including ethical problems associ- ated with building such creatures. These issues will arise again in the final chapters of this book. If the goal of modeling, though, is to build not a complete replica of the human brain but rather an artifact for synthesis and analysis that   111 Dynamical Systems Approach for Embodied Cognition 111 can be used to obtain a better understanding of the human mind and cognition in general in terms of its organizational and functional prin- ciples, such models must be built with an adequate level of abstraction to facilitate their manipulability. Analogously, Herbert Simon (1981) wrote that we might hope to be able to characterize the main properties of the system and its behavior without elaborating the detail of either the outer or inner environments in modeling human. Let us remember the analyti- cal results obtained by Churchland and colleagues (2010) showing that the principal dimensions of ensembles of neuronal firing can be reduced to a few, as introduced in ­chapter  4. Then, it might be reasonable to assume that the spiking of some hundreds of neurons can be reproduced by simulating the activities of a few representative neural units modeled as point masses. An interesting observation is that the macroscopic state of collective neural activity changes continuously and rather smoothly in low-​dimensional space, even though the activity of each neuron at each moment is discontinuous and noisy in regard to spiking. So, cogni- tion and behavior might just correlate with this macroscopic state, which changes continuously in a space whose dimensionality is several orders lower than the original dimensionality of the space of spiking neurons. Consequently, it might be worthwhile to consider a network model consisting of a set of interacting units in which each unit essentially represents a single dimension of the original collective activity of the spiking neurons. Actually, this type of abstraction has been assumed in the connectionist approach, which is described in detail in the seminal book Parallel and distributed processing: Explorations in the microstructure of cognition, edited by Rumelhart, McClelland, and the PDP Research Group (1986). They showed that simple network models consisting of sets of activation units and connections can model various cogni- tive processes, including pattern matching, dynamic memory, sequence generation-​recognition, and syntax processing in distributed activation patterns of the units. Those cognitive processes are emergent proper- ties of the interactive dynamics within networks, which result from the adjustment of connectivity weights between the different activation units caused by learning. Among the various types of connectionist network models proposed, I find particularly interesting a dynamic neural network model called the recurrent neural network (RNN) (Jordan, 1986; Elman, 1990; Pollack, 1991). It is appealing because it can deal with both spatial and tem- poral information structures by utilizing its own dynamic properties. 112 112 On the Mind However, the most important characteristics of RNNs are their gen- erality. As we proceed, we’ll see that RNNs, even in their minimal form, can exhibit general cognitive functions of learning, recognizing, and generating continuous spatiotemporal patterns that achieve gener- alization and compositionality while also preserving context sensitiv- ity. These unique characteristics of RNNs are due to the fact that they are nonlinear dynamical systems with high degrees of adaptability. It is well known that any computational process can be reconstructed by nonlinear dynamical systems as long as their parameters are adequately set (Crutchfield & Young, 1989). A study by Hava Siegelmann (1995) has established the possibility that analog computations by RNNs can exhibit an ultimately complex computational capability that is beyond the Turing limit. This can be understood by the fact that a nonlinear dynamical system can exhibit complexity equivalent to an infinite state machine depending on its parameters, as described in section 5.1. Next, we start to look at a simpler neural network, the feed-​forward network that can learn input-​output mapping functions for static pat- terns. Then, we show how this feed-​forward network can be extended to RNNs, which can learn spatio-​temporal patterns. At the same time, we examine the basic characteristics of the RNN model from the per- spective of nonlinear dynamical systems. 5.5.  Neural Network Models This section introduces three types of basic neural network models including the three-​ layered feed-​ forward network, the discrete-​time RNN, and the continuous-​time RNN (CTRNN). All three types have two distinct modes of operation. One is the learning mode for deter- mining a set of optimal connectivity weights from a training dataset, and the other is the testing mode in which an optimal output pattern is generated from an example test input pattern. 5.5.1  The Feed-​Forward Network Model The feed-​forward network model is shown in Figure 5.14. It consists of an input unit layer, a hidden unit layer and an output unit layer. Neural activations propagate from the input units to the hidden units and to the output units through the connectivity weights spanning between oni .  The feed-​forward network model. 7b) Where bni is a bias value for each unit and f is a sigmoid functiion. and output units (indexed with i) and is trained to produce input-​output mapping for P different patterns. The learning is conducted by utilizing the error back-​propagation scheme that was conceived independently by Shun-Ichi Amari (1967). respec- tively. (1 – oni ( ( Wij ∆Wij = – εδni . Wij . and oni . and Rumelhart and colleagues (1986). We assume that the network consists of input units (indexed with k). Paul Werbos (1974). where innk . anj Hidden a nj δnj = Σi (δni . 7a) oni = f (uni ) (Eq. The poten- tials of the hidden and output units are denoted as unj and uni . the forward activation of an output unit is written as: uni = ∑ j wij anj + bni (Eq. anj . each layer. anj . respectively. The objective of learning is to determine a set of optimal connectivity weights that can reconstruct input-​output patterns given in the target training dataset. hidden units (indexed with j.14. (1 – anj ( ( Wjk ∆Wjk = – εδnj . ank Input in nk Figure 5. Feed-​forward activation and error back-​propagation schemes are illustrated in the model. 8a) .  113 Dynamical Systems Approach for Embodied Cognition 113 Oni Oni Output δni = – (oni – oni . is given as input. Similarly for the hidden units as: unj = ∑ k w jk innk + bnj (Eq. and the training target of the nth pattern is denoted as oni . The right side of the figure shows how the delta error and the updated weights can be calculated through the error back-​propagation process from the output layer to the hidden layer. Thus. The activations of the units when presented with the nth pattern are denoted as innk .  9 1 En = ∑ (oni − oni ) ⋅ (oni − oni ) 2 i (Eq.114 114 On the Mind anj = f (unj ) (Eq. which are denoted as ∆wij . we obtain ∂E n ∂E n (Eq. 8b) Here. which is denoteed as δ in . Because the weights should be updated in the direction of minimizing the square of the error. . the delta error at the ith unit can be obtained as follows.10) −ε = − ε i ⋅ anj ∂wij ∂un ∂E n Here. as shown in Eq. the goal of learning is to minimize the square of the error between the target and the output. we formulate how to update the connection weights in the output layer. 9 to the first term on the right side and taking the deriv- ative of the sigmoid function with respect to the potential for the second term of the preceding equation. the direction can be obtained by taking the derivative of E n with respect to wij as follows: ∂E n ∆wij = − ε ∂wij The right side of this equation can be decomposed as: ∂E n ∂E n ∂u i −ε =−ε i ⋅ n ∂wij ∂un ∂wij By applying Eq. 9) First. 7a to the second derivation on the right side. is the delta error of the ith unit. The ∂uni delta error represents the contribution of the potential value of the unit to the square error: ∂E n δ in = ∂uni ∂E n ∂oni = ⋅ ∂oni ∂uni By applying Eq. ∂E n ∆w jk = − ε ∂w jk ∂E n ∂unj =−ε ⋅ ∂unj ∂w jk ∂E n By substituting with the delta error at the jth δ nj and folding by ∂ u j ∂unj n at ank by applying Eq. we obtain the updated connection weights of the hidden layer. 11) Furthermore. by taking the derivative of E n. 12) Next. 10. which are denoted as ∆w jk . with respect to w jk . δ nj can be derived from the previously obtained δ in as folllows : ∂E n δ nj = ∂unj  ∂E n ∂u i  ∂a j = ∑ i  i ⋅ nj  ⋅ nj  ∂un ∂an  ∂un = ∑ i ( δ in ⋅ wij ) ⋅ anj ⋅ (1 − anj ) (Eq. the updated weights can be written as: ∂w jk ∆w jk = − εδ nj ⋅ ank (Eq.  115 Dynamical Systems Approach for Embodied Cognition 115 δ in = −( oni − oni ) ⋅ oni ⋅ (1 − oni ) (Eq. the same error back-​propagation scheme is repeated. by utilizing the delta error in Eq. 8a. If there are more layers. 13) Here. the updated weight can be written as: ∆wij = − εδ in ⋅ anj (Eq. in the course of which (1) the delta error at each unit in the current layer is obtained by back-​propagating the error from the previous layer through the connec- tion weights and (2) the incoming connection weights to the units in the . 14) It should be noted that ∑ (δ i i n ⋅ wij ) in the first term on the right side represents the sum of delta errors δ in back-​propagated to the jth hidden unit multiplied by each connection weight wij . (a) Forward activation and (b) the error back-​propagation through time scheme in the cascaded RNN.2  Recurrent Neural Network Models Recurrent neural network models have been used to investigate the human cognitive capability of dealing with temporal processes such as in motor control (Jordan. Williams & Zipser.116 116 On the Mind current layer are updated by using the obtained delta errors. 1991. Although various types of RNNs have been investigated so far (Jordan. Let’s look at the exact form of the RNN model.15. In the forward dynamics. The context units represent context or the internal state in representing dynamic sequence patterns. Schmidhuber. 1989.  Jordan-​t ype RNN. 1986. Doya & Yoshizawa. illus- trated in Figure 5. 1992). Elman. The actual process of updating the connection weights is implemented through summation of each update for all training patterns as: P w new = w old + ∑∆w n (Eq. 1986) and language learning (Elman. it might be helpful to look at the Jordan-​type RNN (Jordan. 1990. Pollack. 15) n 5. the current step context unit activation c t is mapped to its next step activation c t +1. 1986).15. as it is one of the simplest implementations. Let us consider an example of learning to generate a simple 1-​d imensional cyclic sequence pattern of period 3 such as “0 0 1 0 0 (a) (b) Out 3 error Outt+1 Out3 Contextt+1 In2 t=2 Outt+1 Out2 error Out2 In1 t=1 Int Out1 error Contextt Out1 t=0 In0 Figure 5. 1990). This model has context units in addition to current step inputs int and next step output out t +1.5. 1989. . Connectivity weights correspond to the parameter P of the dynamical map. Therefore.2 0. “0 1 0 0 1 0 … 0 1.2 0.e. This phenomenon is explained quali- tatively by Maturana and Varela (1980).” In this example.1 by considering that the current state Xt consisting of the current step input and the current context state is mapped to next state Xt +1 con- sisting of the input and the context at next step. including utilization of the autonomous internal dynamics of the closed network. that is.4 0. the RNN model can acquire desired dynamic structures by adequately tun- ing the connectivity weights to the learnable parameters of the dynami- cal map.4 0.” It is noted that the Jordan-​t ype RNN operated in discrete time steps can be regarded as a dynamical map as shown in Eq. 2 in sec- tion 5. Iterative interactions between interconnected neural units afford the RNN a certain amount of autonomy. a 1-​d imensional context activation sequence is formed such as “0. the RNN model with the context units can learn it if the context unit activation states can be differentiated from ambiguous outputs. this process should in fact involve additional steps. . Later sections return to this point. two 0s in the first step and the second step in this cyclic sequence pattern. which might well constitute the origin of voluntary or contextual sensitivity. who state that neural circuits are closed circuits without any input or output functions.8 0. as we focus in on the issue of free will.. the aforementioned cyclic pattern of repeating “0 0 1” can be learned as a limit cycle attractor of period 3.8 … 0.) Although this type of sequence cannot be learned by the feed-​forward network by means of simple input-​output mapping. Closed cir- cuits maintain endogenous dynamics that are structurally coupled with sensory inputs and produce motor outputs. One of the important characteristics of RNNs is that they can exhibit dynamic activities autonomously without receiving any inputs when operated in closed-​loop by feeding the prediction output for next step to the input of the current step.  117 Dynamical Systems Approach for Embodied Cognition 117 1 … 0 0 1… .4 0. For example. again.” The learning of this type of sequence faces the hidden state problem because the sequences include the same target output value in different orders in the sequence (i.2 0.” which is mapped to the output activation sequence of “0 0 1 0 0 1 … 0 0 1.8. the input is given as a sequence of “0 0 1 0 0 1 … 0 0” and the target output is given as this sequence shifted for- ward one step. wherein sensory inputs are considered to be perturbative inputs to the endogenous dynamics. Although it is tempting to think that motor outputs are generated sim- ply by mapping different sensory states to sensory reflexes in different situations. 118 118 On the Mind The RNN employs a learning scheme. in the backward computation for the BPTT. He showed that a version of RNN.15b. which is repeated from the start step to the end step in the forward computation. which has been devel- oped by extending the conventional error back-​propagation scheme in the backward time direction to develop adequate dynamic activation patterns in the context units. the relative pronoun .. the example sentences for training the network were generated by using a lexicon of 23 items including 8 nouns. Then the connectivity weights between the input layer and the hidden layer are updated using the delta error back-​propagated from the output units to the hidden units. now called an Elman net (Figure 5.16a) can learn to extract grammatical structures from given exemplar sentences. the connectivity weights between the output layer and the hidden layer are updated by using the error generated between the tar- get output and the generated output. the error generated in the output units at a particular time step is propagated through the context input units to the previous step context output units. In this cascaded net- work. which is repeated until the delta error signal reaches the context input units in the start step. Therefore there are no means to update the connectivity weights between the con- text output units and the hidden units. The capability of the RNN for self-​ organizing context-​ dependent information processing can be understood well by looking at a prominent research outcome presented by Jeffrey Elman (1991) on the topic of lan- guage learning utilizing RNN models. In the aforementioned feed-​forward net- work model. However. Werbos. the current step activation of the context output units is copied to the context input units in next step. in this situation. This scheme of the BPTT can be well understood by supposing that an identical RNN is cascaded in the direction of time to form a deep. the error signals originating from the output units of different time steps are accumulated as the time step folds back and by which all the connectivity weights of the identical RNN can be updated. feed-​forward network as shown in Figure 5. However. In the BPTT. in the case of RNN. 12 verbs. 1986. back-​propagation through time (BPTT) (Rumelhart et al. On the other hand. there are no error signals for the con- text output units because there are no target values for them. 1988). In his simulation experiment. if the delta error back-​propagated from the hidden units to the context input units is copied to the context output units in the previous step. the connectivity weights between the context output units and the hidden units can be updated by utilizing this copied information.   119 Dynamical Systems Approach for Embodied Cognition 119 (a) Next word prediction output (b) S  NP VP “. the Elman network was used for the generation of succes- sive predictions of words in sentences based on training of exemplar sentences.” and a period for indicating ends of sentences. the first word in the next sentence was input. the correct target output was shown and the resultant pre- diction error was back-​propagated. The input and the output units had the same representation. As described in ­chapter  2. (b) the context free grammar employed.) In the experiment. the presence of a relative clause with “who” allows generation of recursively complex sen- tences such as: “Dog who boys feed sees girl.  Sentence learning experiments done by Elman. It is noted that the Elman network in this experiment employed a local representation in the winner-​take-​all way using a 31-​bit vector for both the input and the output units. (a) The Elman network. and (c) an example sentence generated from the grammar (Elman. 1991). “who. and the network predicted the next word as the output. Especially. A particular word was represented by an activation of a corresponding unit out of 31 units. After the prediction.” NP  PropN | N | N RC VP  V (NP) RC  who NP VP | who VP N  boy | girl | cat | dog | boys | girls | cats | dogs Context loop PorpN  John | Mary V chase | feed | see | hear | walk | live | chases | feeds | sees | hears | walks | lives Current context (c) S Current word input NP VP N RC V NP who NP VP N N V dog who boys feed sees girl Figure 5. This process was repeated for thousands of the exemplar sentences generated from the aforementioned grammar.16. thereby adapting the connectivity weights.” (See Figure 5.16c.16b. The analysis of network performance after the training of the tar- get sentences showed various interesting characteristics of the network . words were input one-​at-​a-​time at each step. various sentences can be generated by recursively applying substitution rules starting from S as the top of the tree representing the sentence structure. More specifically. The sentence gen- erations followed a context-​free grammar that is shown in Figure 5. At the end of each sentence. 120 120 On the Mind behaviors. all plural verbs and “who” were activated. 16. In this model. For the case of “boy sees. respectively. we look at an RNN model operated in continuous time. for the case of “boy chases. all three singular verb categories as well as “who” for relative clause were activated as possible predicted next words and all other words were not activated at all.” only noun words were activated. First. It also understands that “see” can be both.3  Continuous Time Recurrent Neural Network Model Next. equations in which each neural .” To keep singular–​plural agreements between subjects and distant verbs. look at simple sentence cases.” both a period and noun words were activated for the next prediction. On the other hand. It was also observed that the network captures verb agreement struc- tures as well. after two succeeding words “boy lives” are input. An example exists in the following paired sentences: 1. which is known as a continuous-​time recurrent neural network (CTRNN) (Doya & Yoshizawa. boy who boys chase chases boy 2.16a as an example). a period is predicted. This means that the network seems to capture the singular–​plural agree- ments between subject nouns and verbs. singular–​ plural agreements were preserved. Let us consider a CTRNN model without an explicit layer structure in which each neu- ral unit has synaptic inputs from all other neural units and also from its own feedback (see Figure 5. Finally. Elman found that context acti- vation dynamics can be adequately self-​organized in the network for this purpose. boys who boys chase chase boy Actually. when a plural noun “boys” was input. Moreover. 1989. actual activation val- ues encoded the probability distribution of next-​coming words. 1989). the information of singular or plural of the subjects had to be preserved internally. the activation dynamics of each neural unit can be described in terms of the differential equations shown in Eq.5. because only “boy” or “boys” cannot determine the next words deterministically. the network activated the singular verbs after being input “boy who boys chase” and activated the plural ones after being input “boys who boys chase. Williams & Zipser. The network seems to understand that “live” and “chase” are an intransitive verb and a transi- tive verb. 5. When a singular noun “boy” was input. Although the presence of a relative clause makes a sentence more complex. For example. .17a.17d. the slower or faster the change of the potential ui. which will be especially important as we consider MTRNNs later. 16a) i a i = 1 / (1 + e − u ) (Eq.17b–​d shows that different attractor configura- tions can appear depending on the connection weights. as shown in Figure 5. 1995]). The time constant τ plays the role of a viscous damper with its positive value. This means that positive and negative synaptic inputs increase and decrease the potential of the unit. let’s examine the dynamics of CTRNNs. respectively. The larger or smaller the time constant τ . which is equated with the sum of synaptic inputs subtracted from the first term −u i. This type of complexity in attractor con- figurations might be the result of mutual nonlinear interactions between multiple neural units. a single chaotic attractor appears with a dif- ferent connection weight matrix. If the sum of synaptic inputs is zero. Next. respectively. In Figure 5. You may notice that this equation is analogous to Eq. In summary then. 16b) The left side of Eq. The CTRNN model examined by Beer consists of three neural units as shown in Figure 5. τu i = − u i + ∑ j a j wij + I i (Eq.  121 Dynamical Systems Approach for Embodied Cognition 121 unit has synaptic inputs from all other neural units as well as from its own feedback. depending on the parameters represented by connection weights (this characteristics is the same also for discrete time RNN [Tani & Fukumura. Figure 5. This fea- ture can be used for memorizing multiple temporal patterns of perceptual signals or movement sequences. CTRNNs can autonomously gen- erate various types of dynamic behaviors ranging from simple fixed-​point attractors through limit cycles to complex chaotic attractors. The eight stable fixed-​point attractors and the two limit-​cycle attractors appear with each specific connection weight. An interesting observation is that multiple attractors can be generated simultaneously with a given specific connection weight matrix.17b and c. 3 of representing a general form of the continuous-​time dynamical system. 16a represents the time differential of the poten- tial of the ith unit multiplied by a time constant τ . and the attractor towards which the state trajectories converge depends on the initial state. especially the values of the connection weights. the potential converges toward zero. Randall Beer (1995a) showed that even a small CTRNN consisting of only three neural units can generate complex dynamical structures depending on its param- eters. (b). After the forward computation with these leaky integrator neural τ . Figure 5.  Different attractor configurations appear in the dynamics of a continuous time RNN model consisting of three neural units receiving synaptic inputs from the other two neural units as well as own recurrent ones. 1995a) with permission. (c) two limit cycles denoted as line circles with arrows. First. 17b) What we have here is the leaky-​integrator neuron with a decay rate of 1 1 − i .18 illustrates how the BPTT scheme can be implemented in a CTRNN. Eq. 16 from a differential equation form into a difference equation by using Euler’s method for the purpose of numerical computation. (a) The network architecture.122 122 On the Mind (a) w11 1 w31 w21 w13 w12 w32 2 3 w33 w23 w 22 (b) (c) (d) u1 u2 10 2 u1 2 5 4 4 6 6 0 10 2 2 u3 5 u3 u3 1 1 0 0 0 0 5 4 4 2 2 u1 10 0 u2 0 u2 Figure 5. 17 is obtained by converting Eq.17. (b) eight stable fixed-​point attractors denoted as black points. 17 with a given initial neural activation state at each unit. 17a) τ τ i ati = 1 / (1 + e − ut ) (Eq. 1 i 1 uti = (1 − i )ut −1 + i (∑ j ati−1wij + I ti−1) (Eq. the forward activation dynamics of the CTRNN for n steps is computed by following Eq. In the case of a CTRNN characterized by the time constant param- eter � τ . (c). and (d) are adopted from (Beer. and (d) chaotic attractors. the BPTT scheme for supervised learning is used with slight modifications to the original form. The figure shows how the error generated at the nth step is propagated back to the (n−2)nd step. is recursively calculated from the following formula: ∂E   ( i i ) i i (   ) 1  ∂E  − ot − ot ⋅ ot ⋅ 1 − ot + 1 −  i τ i  ∂ut +1 i ∈ Out = (Eq. 4th.  An extension of the error back-​propagation scheme to CTRNNs.18. 18) ∂ut  i ∂E   1 1  ∑ k ∈N ∂u k δ ik 1 − τ  + τ wki ⋅ at 1 − at  i i ( ) i ∉ Out  t +1   i k . as denoted by continuous arrows. the (n−​1)st step. 2nd. This back-​propagation process is recursively repeated until the 1st step of the sequence is reached. These errors continue to back-​propagate along the forward connection over time. dotted lines. Then. through forward connections. units. One important note here is that the way of computing the delta error in CTRNN is different from the one in the conventional RNN because of the leaky integrator term in the forward activation dynamics defined in Eq. Additionally. The delta error at the ith unit ∂E . and chain lines denote the back-​propagation error generated at the nth step. the delta errors generated at the output unit in steps n−​1 and n−2 are also back-​propagated in the same manner. 2nd. 2nd. The delta error at the output unit in the nth step is computed as δ 0n . respectively. These delta errors propagated to local units are further back-​propagated to the 1st. respectively. 17a. Arrows with continuous lines. and 4th units in the (n−​1)st step and to the 1st. and 5th units. and the (n−2)nd step. the back-​propagation computation is initiated by computing the error between the training target and the current output in the nth step. 3rd. and 4th units in the (n−​2)nd step. ∂uti either for an output unit or an internal unit. this delta error is back-​propagated to the 1st.  123 Dynamical Systems Approach for Embodied Cognition 123 On On–1 On–2 errorn errorn–1 errorn–2 On On–1 On–2 0 0 0 1 1 1 3 5 3 5 2 5 2 4 2 4 2 4 n step n–1 step n–2 step Figure 5. as denoted by dotted-​line and chain-​line arrows. the updated weights for the input connections to those units in step n−1 are obtained by following Eq. In light of this evidence. 2nd. It has been observed that the action potential back-​propagates through dendrites when postsynaptic neurons in the downstream side fire upon receiving synaptic inputs above a threshold from the presynaptic neu- rons in the upstream side.0. 18 it can be seen that the ith unit in the   current step t inherits a large portion 1 − 1 of the delta error ∂E  τ i  ∂uti +1 from the same unit in the next step t+1 when its time constant τ i is relatively large. their biological plausibility in neuronal circuits has been questioned. It should also be noted. Although the aforementioned models of feed-​ forward networks. 13. RNN and CTRNN employ the error back-​propagation scheme as the central mechanism for learning. however. therefore. at the 1st unit in the (n−1)st step. speculate that the retrograde axonal signal (Harris. the delta errors propagated from the 0 th. a version of RNN in which internal units are connected with randomly predetermined constant weights and . Du & Poo. It is noted that Eq.124 124 On the Mind From the right-​hand side of Eq. All delta errors propagated from different units are summed at each unit in each step. the bio- logical plausibility of this approach appears promising. that counterintuitive results have been obtained by other researchers. This means that. This enables the learning of long-​term cor- relations latent in target time profiles by filtering out fast changes in the profiles. What Poo has further suggested is that such synaptic inhibition or potentiation depending on information activity can propagate backward across not just one but some successive synaptic connections.. as well as by Harris (2008) in related discussions. By utilizing the delta errors com- puted for local units at each step. some supportive evidence has been provided by Mu-​ming Poo and colleagues (Fitzsimonds et al. 2008) conveying error information might propagate from the peripheral area of sensory–​motor input-​output to the higher-​order cortical area. using the “echo-​state network” (Jaeger & Haas. 1997. modulating its contextual memory structures by passing through multiple layers of synapses and neurons in the real brains like the delta error signal back-​propagates from the output units to the inter- nal units in the CTRNN model. For example. 2004). However. and 1st units are summed to obtain the error for the (n−1)st step. discrete time version of BPTT when τ i is set at 1. We can. 2004). 18 turns out to be the conventional. then. in a network with a large time constant. error back-​propagates through time with a small decay rate. For example. 1993. various experiments were conducted in which different neural adaptation schemes were applied in the development of sensory–​ motor coordination skills in robots. and the environment. Beer 1995. The next section introduces neurorobotics studies that use some of the neural network models. Krichmar & Edelman. Jaeger and Haas showed that quite complex sequences can be learned with this scheme. other researchers have explored such topics while seriously considering the issues of embodiment emphasized on the behavior-​based approach. 2000. Morimoto & Doya. Ziemke & Thieme. In this setting. 1987. 2002. 2000. 2001. 1997. proposed the idea of considering the structural coupling between the neural system. A representative researcher in this field. Di Paolo.  Neurorobotics from the Dynamical Systems Perspective Although Rodney Brooks did not delve deeply into research on adap- tive or learnable robots. Shibata & Okabe. Schaal. wherein the connection weights are modified in the direc- tion of reward maximization. Nolfi & Floreano. the body. 2002. Doya & Uchibe. and supervised and imitation learning (Tani & Fukumura. 1997.. Billard. Obviously. My question here would be what sorts of internal structures can be gener- ated without the influence of error-​related training signals. so the three can be viewed as a coupled dynamical system.. Demiris & Hayes.. it is argued that the objective of neural adaptation is to keep the behavior of the whole system within a viable zone. Endo et al. 5. 2002. which uses artificial evolution of genomes encoding connection weights for neural networks based on principles such as survival of the fittest. Randall Beer (2000). The internal neural system interacts with its body and the body inter- acts with its surrounding environment. this thought is quite analogous to the Gibsonian and Neo-​Gibsonian approaches as described in section 5. 1996. Cliff et al..2. Ijspeert.  125 Dynamical Systems Approach for Embodied Cognition 125 only the output connection weights from the internal units are modu- lated without using error back-​propagation. These schemes included: evolutional learning (Koza. as illustrated in Figure 5. 2005. 2008). Ikegami & Iizuka. Meeden. 2000. 2007). 1999. 1992. wherein a teacher or . Gaussier et al.19. 2001. including the feed-​forward network model and the RNN model. Steil et al. In the 1990s. 2004). value-​ based reinforcement learning (Edelman. 1998.6. 6. Especially. By constructing synthetic . Let’s look now at a few examples of these studies from among the many remarkable studies that have been conducted. such as loco- motion. the fol- lowing emphasize the dynamical systems perspectives in developing and generating the minimal cognitive behaviors by neurorobots. Although the experi- ments might have lacked scalability both with respect to engineering applications and accounting for human cognitive competence. imitation targets exist.19. 1980). they do demonstrate that nontrivial structures in terms of “minimal cognition” can emerge in the structural coupling between simple neural network models and the environment.1  Evolution of Locomotion with Limit Cycle Attractors It is widely held that rhythmical movements in animals. which generate oscillatory signals by means of limit-​cycle dynamics in neural circuits (Delcomyn.126 126 On the Mind Environment Body Neural System Figure 5. are generated by neural circuits called central pattern genera- tors (CPGs). and the environment are considered as a coupled dynamical system by Randall Beer (2000). the body.  The neural system. Most of these experiments were conducted in minimal settings with rather simple robots (mobile robots with range sensors in many cases) with small-​scale neural controllers (influenced by Gibsonian and behavior-​based philosophy). 5. In this artificial evolution scheme. and the fitness of the individual is evaluated by measuring the maximum for- ward walking distance within a specific time period. the connectivity weights in CTRNN are randomly modulated in terms of “mutation.” with their “offspring” inheriting the same connectivity weights of the networks. the connectivity weights within the local CTRNN as well as the interconnections between the six local CTRNNs are mutated.  127 Dynamical Systems Approach for Embodied Cognition 127 simulation models and conducting robotics studies based on the con- cept of CPGs. Gait and motor outputs serve as sensory inputs for the network in terms of the torques generated when the legs move forward and backward. 1999). 1991. Especially. 1995b).. a number of researchers have investigated the adaptation mechanisms of walking locomotion in a number of animals: six-​legged insects (Beer. This evolved control- ler could generate autonomous rhythmic oscillation without having any . Because the leg movements were generated by means of reflections of sensory inputs. Otherwise. In Beer’s model. a “reflective pattern genera- tor” evolved. as well as in walk- ing and swimming via the spinal oscillation of four-​legged salamanders (Ijspeert. connectivity weights are adapted in the direction of maximizing fitness over genera- tions of population dynamics. each leg is controlled by a local CTRNN consist- ing of a small number of neural units. First. four-​legged dogs (Kimura et al. characteristic connec- tivity weights within networks are not “reproduced. 2001). if the sensory inputs were constantly enabled during evolution. these robots are allowed to “reproduce. Second. An interesting finding from Beer’s simulation experiments on artificial evolution is that evolved locomotion mechanisms were qualitatively dif- ferent under different evolutionary conditions. The six local CTRNNs are sparsely connected to generate overall body movement. the locomotive motor pattern was easily distorted when the sensory inputs were disrupted. a CPG-​type locomotive controller evolved. if the sensory inputs were made inaccessible completely from the network during evo- lution.. and two-​ legged humans (Taga et al. Beer (1995b) investigated how stable walking can be achieved for six-​legged insect-​like robots under different conditions in the interaction between internal neural systems and the environment by utilizing artificial evolution within CTRNN models. Endo et al.” If some robots exhibit better per- formance in terms of the predefined fitness functions with the modu- lated weights in the networks as compared with others.” Thus. During the evolutionary learning stage. 2008).. a limit cycle is organized in the coupling between the internal dynamics of the CTRNN and the environment dynamics. 1995b) with permission. if the presence of the sensory inputs was made unreli- able during evolution. Scheier. by means of the self-​organizing limit cycle attractor in the CTRNN. the limit cycle attractor appears in the form of an autonomous dynamic in the CTRNN alone. Third.  Six-​legged locomotion patterns generated by the evolved mixed pattern generator (a) shows a gait pattern with sensory feedback. it demonstrated better locomotion performance when the sensory feedbacks were available (Figure 5. Beer speculated that the mixed strategy that emerges under the condi- tion of unreliable sensory feedback is the most typical among biological pattern generators. Although this controller could generate robust basic locomotion patterns even when the sensory inputs were disrupted.2  Developing Sensory–​Motor Coordination Schemes of evolutionary learning have been applied in robots for various goal-​d irected tasks beyond locomotion by developing sensory–​motor coordination adequate for such tasks. these experiments showed that limit cycle attractors can emerge in the course of evolving the CTRNN controller for generating locomotion in different ways. Adopted from (Beer.128 128 On the Mind (a) R3 R2 R1 L3 L2 L1 Velocity (b) R3 R2 R1 L3 L2 L1 Velocity Figure 5. and (b) shows one without sensory feedback. 5. In summary.6.20.20). external drives. When the sensory feedback is available. a “mixed pattern generator” evolved. Pfeifer. Otherwise. The case with the sensory feedback shows more stable oscillation with tight coordination among different legs. depending on the parameters set for the evolution process. and Kuniyoshi . where large and small cylin- drical objects were placed at random. They prepared a workspace for a miniature mobile robot (55 mm in diameter). and the images are low resolution.  The Khepera robot. The synaptic weights necessary for determin- ing the characteristics of mapping sensor inputs to motor outputs were obtained in an evolutionary way.22). This task is far from trivial because the sensing capabilities of the Khepera robot are quite limited. The robot . which features two wheel motors and eight infrared proximity sensors mounted in the periphery of the body. (1998) showed that nontrivial perceptual categorization capabilities can be acquired by inducing interactions between robots and their environ- ments. Source: Wikipedia.  129 Dynamical Systems Approach for Embodied Cognition 129 Figure 5. Scheier and colleagues implemented a feed-​ forward neural network model that receives six directional range images from sensors at the front and controls the speeds of the left and right motors. Therefore. It was reported that when the robot evolved a successful network to accomplish the task. the robot can acquire eight directional range images represent- ing distances to obstacles. it would wander around the environment until it found an object and then would start circling it (Figure 5. The behavioral task for the robot was to approach large cylindrical objects and to avoid small ones. called Khepera (Figure 5. The fitness value for evolutionary selec- tion increased when the robot stayed closer to large cylindrical objects and decreased when the robot stayed closer to small ones. consisting of just eight infrared proximity sensors attached to the periphery of the body.21.21). but detection occurs only when an obstacle is within 3 cm. sensory–​motor coordination was naturally selected for active perception in their experiment.22. otherwise it would keep circling if the object was large. the successfully evolved robot circled around a cylindrical object. avoiding walls while staying close to cylindrical objects. Nolfi and Floreano (2002) showed another good example of evolu- tion based on active perception. After . the evolutionary processes found an effective scheme based on active perception. This example clearly shows that this type of active perception is essential for the formation of the robot’s behavior. In this scheme. utilizing information from proximity sensors on one side of its body. It would wander around the environment until it found a cylinder of large size and then would start circling it. Eventually. A significant difference was found between large and small objects in terms of the way that the robot circled the object by generating different profiles of the motor output patterns which enabled different object types to be identified. but in this case there is the added ele- ment of self-​organization. They showed that the Khepera robot equipped with a simple perceptron-​t ype neural network model can evolve to distinguish between walls and cylindrical objects. the so-​called behavior attractor.130 130 On the Mind Figure 5. simply by following the curvature of its surface.  An illustration of the behavior trajectory generated by a successfully evolved Khepera robot. Because it was difficult to distinguish between large and small cylindrical objects by means of passive perception using the installed low-​resolution proximity sensors. whether small or large. whereby perception and action become inseparable. would eventually leave its trajectory if the object was a small cylindrical one. The scheme is based on the aforementioned thoughts by Nadel (see section 5. Here. Vision camera Arm movement Visual Arm movement percept Controller Proprioception Self-robot Other robot “like me” Figure 5. staying close to cylindrical objects does not mean stopping. Nolfi and Floreano inferred that the robot could keep its relative position by means of active perception that was mechanized by a limit cycle attractor developed in the sensory–​motor coupling with the object.  A robot generates immediate imitation of another robot’s movement by using acquired visuo-​proprioceptive mapping (Gaussier et al. the robot moves around by avoiding walls and staying close to cylindrical objects whenever encountering them. Before closing this subsection. These two experimental studies with the Khepera robot show that some nontrivial schemes for sensory–​motor coordination can emerge via network adaptation through evolution even when the network structure is relatively simple. another robot of a similar configuration was placed in front of the robot and the other robot moved its arm (Figure 5..  131 Dynamical Systems Approach for Embodied Cognition 131 the process of evolution. A steady oscillation of sensory–​motor patterns with small amplitude was observed while the robot stayed close to the object. . After the learning. Rather.23. 1998). Gaussier and colleagues built an arm robot with a vision camera that learned a mapping between the arm's position as perceived in the visual frame and the proprioception (joint angles) of its own arm by using a simple perceptron-​t ype neural network model. the robot continues to move back and forth and/​or left and right while maintaining its relative angular position to the object almost constant.23).2) that immediate imitation as a means for communication can be generated by synchro- nization achieved by a simple sensory–​motor mapping organized under the principle of homeostasis. I would like to introduce an intrigu- ing scheme proposed by Gaussier and colleagues (1998) for generat- ing immediate imitation behaviors of robots. 132 132 On the Mind When the self-​robot perceived the arm of the other robot as its own. Recalling Maturana and Varela (1980). wherein sensory inputs and motor outputs are regarded as perturbations of and readouts from the dynamical system. neural cir- cuits are considered to exhibit endogenous dynamics. let me explain a scheme called “branching” that is implemented in low-​level robot control. This study nicely illustrates that immediate imitation can be generated as synchronicity by using a simple sensory–​motor mapping that also supports the hypoth- esis of the “like me” mechanism also described in section 5. 5. The experiment was conducted with a real mobile robot named Yamabico (Figure 5. we look at a robotics experiment that uses sensory–​motor map- ping but in a context-​dependent manner. It should be noted that the robot cannot access any global information. 1997). The task was designed in such a way that a mobile robot with limited sensory capabilities learns to navigate given paths in an obstacle envi- ronment through teacher supervision. The range sensors perceive range images from 24 angular directions covering the front of the robot. Instead. its own arm was moved and synchronized with the one of the other for the sake of minimizing the difference between the current propriocep- tion state and its estimation obtained from the output of the visuo-​ proprioceptive map under the homeostasis principle.3  Self-​Organization of Internal Contextual Dynamic Structures in Navigation We should pause here to remind ourselves that the role of neuronal sys- tems should not be regarded as a simple mapping from sensory inputs to motor outputs. Next. such as RNNs or CTRNNs. This should also be true if we assume dynamic neural network models with recurrent connections.2. respectively. which was done in collaboration with Naohiro Fukumura (Tani & Fukumura. First.24a). The following study shows such an example from my own investigations on learning goal-​d irected navigation. The robot is preprogrammed with a colli- sion avoidance maneuvering scheme that determines its reflex behav- ior by using inputs from the range sensors. such as its position in the X-​Y coordinate system in the workspace. the robot has to navigate the environment depending solely on its own ambiguous sensory inputs in the form of range images representing the distance to surrounding obstacles. 1993. .6. a branching decision is required when a new open space appears. Then. Figure  5. where brighter (closer) and darker (farther) parts indicate their ranges.c illustrates how branching takes place in this workspace. (b) An example of a collision-​ free movement trajectory that contains four branching points labeled 1 to 4. (c) The corresponding flow of range sensor inputs.  133 Dynamical Systems Approach for Embodied Cognition 133 (a) CCD cameras Laser projector (b) (c) (d) Branching Time Action 4 (branching) 4 3 3 start 2 2 1 context units 1 Range recurrent loop Sensor L R Figure 5.24b. The exact range profile at each branching point is shown on the right.24. The robot essentially moves toward the largest open space in a forward direction while maintaining equal distance to obstacles on its left and right sides. (d) The employed RNN model that receives inputs from range sensors and outputs the branching decision at each branching point. .  Yamabico robot and its control architecture. Arrows indicate the branching decision to “advance” to a new branch or to “stay” at the current one. (a) The mobile robot Yamabico employed in this experiment. It is expected that such differentia- tion of context unit activation can be achieved through adaptation of the connection weights. wherein the experimenter trains the robot to gener- ate correct branching on specified target routes. because the sensory inputs cannot necessarily determine the branching outputs uniquely. the decision whether to move left and down or straight and down at the switching position denoted as A in Figure 5. and even if the robot leaves the . because the latter are the same in both cases. as shown in Figure 5.25c shows two examples of evaluation trials. Figure 5. the original 24-​d imensional range images are reduced to a three-​d imensional vector by using a prepro- cessing scheme. In this architecture. The target route in this experiment is designed such that cyclic trajectories emerge in the form of a figure-​8 and a circular trajectory at the end. Here. the experimenter examines how the robot can accomplish the learned task by placing the robot in arbitrary ini- tial positions. This reduced sensory vector is provided as input to the RNN at each branching step. a set of sequential data consisting of the sensory inputs and branching decisions along with the branching sequences are acquired. Then. Note that this is not just simple learning of input-​output mapping. In the actual training of the robot.24d shows how the Jordan-​t ype RNN (Jordan. This sequential data is used to train the RNN so it can generate cor- rect branching decisions upon receiving sensory inputs in the respective sequences. This is called the sensory aliasing problem.25a. For example.. Figure 5.25a should depend on the current context (i. the essence of learning how to navigate the environment is reduced to the task of learning the correct branching sequences associated with the sensory inputs at each branching point. the RNN model is used for learn- ing the branching sequences. in which it can be seen that the robot always converges toward the desired loop regardless of its starting position. whether the last travel was a figure-​8 or a circular trajectory) instead of on solely sensory inputs. the robot is guided repeatedly to enter this target cyclic route by starting from various locations outside the cyclic route (see Figure 5.e. 1986)  explained previously was used in the current navigation task.25b for the traces of the training trajecto- ries). Learning proceeds under supervision.134 134 On the Mind Once this branching scheme is implemented in the robot. and the RNN outputs the corresponding branching decision along with the context outputs. alternating them. The time required for achieving convergence is different in each case. After the training stage. it always returns to the loop after a time. it is found that the robot is exposed to a lot of noise during navigation. These observations indicate that the robot has learned the objective of the navigation task as embedded in the attractor dynamics of limit cycles.25. (a) The target trajectory.  135 Dynamical Systems Approach for Embodied Cognition 135 (a) (b) A (c) Figure 5. and (c) the traces of evaluation trajectories starting from arbitrary initial positions. It is interesting to examine how the task is encoded in the internal dynamics of the RNN. with A as the switching point between two sequences. By investigating the activation patterns of the RNN after its convergence toward the loop. which the robot loops around. which are structurally stable. loop after convergence under the influence of noise. It is found as well that the sensing input vector becomes unstable at particular locations and that .  Training and evaluation trajectories. 1997) with permission. Adapted from (Tani & Fukumura. (b) the traces of the training trajectories. forming a sequence of figure-​8 and circular trajectories. 136 136 On the Mind the number of branches in one cycle is not constant, even though the robot seems to follow the same cyclic trajectory. At the switching point A for either route, the sensory input receives noisy jitter in different pat- terns independent of the route. The context units, on the other hand, are completely identifiable between two decisions, which suggests that the task sequence between two routes is hardwired into the internal contextual dynamics of the RNN, even in a noisy environment. To sum up, the robot accomplished the navigation task in terms of the convergence of attractor dynamics that emerge in the coupling of internal and environmental dynamics. Furthermore, situations in which sensory aliasing and perturbations arise can be disambiguated in navi- gating repeated experienced trajectories by self-​organizing the autono- mous internal dynamics of the RNN. 5.7. Summary The current chapter introduced the dynamical systems approach for modeling embodied cognition. The chapter started with an introduc- tion of nonlinear dynamics covering characteristics of different classes of attractor dynamics. Then, it described Gibsonian and Neo-​Gibsonian ideas in psychology and developmental psychology, ideas central to the contemporary philosophy of embodied minds (Varela et  al., 1991). These ideas fit quite well with the dynamical systems approach, and this chapter looked at how they have influenced behavior-​based robotics and neurorobotics researchers who attempt to understand the essence of cognition in terms of the dynamic coupling between internal neural systems, bodies, and environments. This chapter also provided brief tutorials on connectionist neural network models with special focus on dynamic neural network models including RNN and CTRNN. The chapter concluded by introducing some studies on neurorobotics that aim to capture minimum cognitive behaviors based on the ideas of nonlinear dynamical systems and by uti- lizing the schemes of dynamic neural network models. Although the dynamical systems views introduced in this chapter in terms of Gibsonian psychology, connectionist level modeling, and neurorobotics may provide plausible accounts for some aspects of the embodied cognition, some readers might feel that these do not solve all   137 Dynamical Systems Approach for Embodied Cognition 137 of the essential problems outstanding in the study of cognitive minds. They may ask how the dynamical systems approach described so far can handle difficult problems including those of compositionality in cogni- tion, of free will, and of consciousness. On the other hand, some others such as Takashi Ikegami have argued that simple dynamic neural net- work models are sufficient to exhibit a variety of higher-​order cognitive behaviors such as turn taking (Ikegami & Iizuka, 2007) or free decision (Ogai & Ikegami, 2008), provided that the dynamics of the coupling of bodies and environments are developed as specific classes of complex dynamics. The next chapter introduces my own thoughts on the issue, and I put more emphasis on subjectivity than on the objective world as we try to articulate a general thought of embodied cognition through the study of neurodynamical robot models. 138   139 Part II Emergent Minds: Findings from Robotics Experiments 140   141 6 New Proposals The examples of “learnable neurorobots” described in c­ hapter 5 illustrate how various goal-​directed tasks can be achieved through self-​organizing adequate sensory–​motor coupling between the internal neuronal dynam- ics and the body–​environment dynamics. Although the adaptive behav- iors presented so far seem to capture at least some of the essence of embodied cognition, I feel that something important is still missing. That something is the subjectivity or intentionality of the system. 6.1.  Robots with Subjective Views Phenomenologists might argue that subjectivity cannot be detected explicitly because the goal of embodied cognition is to combine the subjective mind and the objective world into a single inseparable entity through interactions with the environment. However, I argue that such a line of robotics research focuses only on “reactive behavior” based on the perception-​to-​motor cycle, and, therefore, might never be able to access the core problem of the dichotomy between the subjective mind and the objective world. All these robots do is to generate adequate motor commands reactively to current sensory inputs or to current inter- nal states summed with past sequences of sensory inputs. 141 construct their own subjective views of the world by structuring and objectifying . in considering this problem. As described in ­chapter  3. this development is achieved through a process of interweaving double intentionality.2).6. he considered that the world consists of objects that the subject can consciously meditate on or describe. Husserl assumed a three-​level structure in phenomenological time that consists of the absolute flow at the deepest level. He also considered that the continuous flow of experiences becomes articulated into consciously accessible events or objects as a result of its development though these phenomenological levels. This is both good and bad.142 142 Emergent Minds: Findings from Robotics Experiments When my group conducted the robot navigation experiments aimed at learning cyclic trajectories mentioned in section 5. The behaviors of these robots seem too automatic and not requiring any effort. they are fundamentally different from those generally expected from humans in the contexts of both phenomenology and neuroscience. as happens in machines. and the objective time at the surface level. repeatedly bouncing against the pins until they finally disappear down the holes. However. Going back to Husserl (2002). in the begin- ning I was interested in observing the emergent behaviors of the robot in terms of generating diverse trajectories in its transient states before converging to a limit cycle. into the unitary flow of consciousness. which show no traits of subjectivity. Although such robots might be able to mimic smart insects. after a while. as afforded by related perception without sub- jective or intentional control (see section 4. robots characterized with reactive behav- ior have nothing to do with such intentionality for consolidating as-​yet-​ unknown everyday experiences into describable or narrative objects. The behaviors of these robots might be analogous to patients with alien hand syndrome who show behaviors generated automatically. According to Husserl. at this level of sophistication they are not yet capable of authentic and of inauthentic being as characterized by Heidegger (see section 3. Although we might see some complexity on the surface level of these behaviors. I began to feel that robots with such reactive behaviors are simply like the steel balls in pinball machines. the preempirical time level of retention and protention.4). This is to say that current robots cannot. like human beings. namely transversal (retention and protention) and longitudinal (immanence of levels) intentionality. However. Certainly. such as tumblebugs that skillfully roll balls of dung down path- ways. the bottom line is that direct experiences for humans originate not in such consciously representable objects but in the continuity of direct experi- ences in time. 1988).  Engineering Subjective Views into Neurodynamic Models So. we are at the formative stages of this work. How can the subjective views be constructed? Clearly. a set of perceptual structures obtained when an active learner engages in perceptual interaction with the environment and extracts information from it can be regarded as a subjective view belong- ing to that individual. 2004). They track moving objects even when temporarily hidden from view by making a saccade to the reappearance point before the object reap- pears there (Rosander & von Hofsten. 6. some clue as to how to begin—​ and make no mistake. The developmental psychologist Claes von Hofsten has demonstrated that even 4–​ month-​ old infants exhibit such anticipatory behaviors. as a first step in understanding how an artificial agent such as those under consideration in this book may be engineered with the capacity to act and eventually to be responsible for its actions. These infants have prospects for their actions. such characteristic viewpoints and the experiences that underlie them repre- sent the subjectivity of the individual within the greater social system. Such an agent can have a proactive expectation of what the world should look like as it performs its intended actions. However. a character that presumably emerges in dynamic interplay between looking ahead toward possible futures and reflecting on one’s own unique past in order to recruit the resources necessary to enact and realize the most possible future shared with others (see section 3. What I would like to build are robots that realize what Heidegger con- sidered authentic being.2. When they plan to reach for an object. their hands start to close before the object is encountered as they take into account the direction of and distance to the object (von Hofsten & Rönnqvist. and especially with other beings more or less like themselves within it.2.4). this is the very beginning—​appeared first in sec- tion 4. Constructed by each individual when constantly facing various prob- lems unique to that individual’s place within the objective world. which explained the possible role of the predictive model for action generation and recognition in the brains. As Gibson and Pick con- jectured (2000). These are the formative stages in the development of a poten- tially authentic being.  143 New Proposals 143 experiences accumulated through interactions with the world. and moreover for how . which corresponds to limit cycle attractor dynamics. Let me explain this idea by considering some familiar examples.1). Another example is that of shaking a bottle of juice rhythmically. the corresponding sub- jective view in terms of perceptual trajectories is generated in a top-​down manner. In terms of the neurodynamic models from which our robots are constructed. In this case.5). the perceptual trajectories for reaching the bottle from arbitrary posi- tions in this visuo-​proprioceptive space can be illustrated with reduced dimensionality as shown in Figure  6. but all actions of a similar form can be generated by this type of attractor. In this case. as Merleau-​Ponty (1968) explained in terms of visual palpation (see section 3. wherein the views at higher levels . we can imagine the vector flow in the perceptual space as illustrated in Figure 6.144 144 Emergent Minds: Findings from Robotics Experiments the world turns out because of them. The top-​down projection of the subjective view should (only implic- itly) have several levels in general. The vector flows constitute a structurally stable attractor. By switching from one intention to another. and the actions that arise from them.2). perception is active. These perceptual structures might be stored in the parietal cor- tex associated with intentions received from the prefrontal cortex.2.5). can be gen- erated by fixed point attractor dynamics (see section 5. the position of the fixed point varies depending on the position of the object in question. Suppose the intended action is your right hand reaching to a bottle from an arbitrary posture. This idea is analogous to the Neo-​Gibsonian theory (Kelso. the per- ceptual structure for a particular intended action can be viewed as vector flows in the perceptual space as mapped from this intention. The essence here is that subjective views or images about the intended actions can be developed as perceptual structures represented by the correspond- ing attractor embedded in the neural network dynamics. If we consider a perceptual space consisting of the end-​point position of the hand that is visually perceived and proprioception of the hand posture at each time step. as we have seen with CTRNN models that can develop various types of attractors (section 5. Here. and should be considered as a subject acting on objects of perception. as dis- cussed in section 4.1a as a flow toward and a conver- gence of vectors around an attractor that stands as the goal of the action. 1995) in which movement patterns can be shifted by phase transi- tions due to changes in the system parameters (see section 5.1b. These trajectories. we now need to consider a theo- retical conversion from the reactive-​type behavior generated by means of perception-​to-​action mapping to the proactive behavior generated by means of intention-​to-​perception mapping.  1988). top-​down views of the world should be “composi- tional” enough so that proactive views for various ways of intentionally interacting with the world can be represented by systematically recom- bining parts of images extracted from accumulated experiences.  145 New Proposals 145 (a) (b) Proprioception Proprioception Vision Vision Figure 6. are: semanti- cally combinatorial language of thought (Fodor and Pylyshyn. Do we need a framework of symbol representation and manipulation. Ultimately. For example.1. One essential question is how the higher level can manipulate or com- bine action primitives or words systematically. that can afford compositionality as well as . for this purpose? If I said yes to this. I would be criticized just like Dreyfus criticized Husserl or like Brooks criticized conventional AI and cognitive science research. for (a) approaching an object and (b) shaking it. to recall once again the very familiar image of everyday rou- tine action with which this text began.  The perceptual trajectories for different intended actions in visuo-​proprioceptive space. when we intend to drink a cup of coffee. grasping-​cup. and bringing-​cup-​to-​ mouth in sequences that may be projected downward to a lower level where detailed proactive images of corresponding perceptual trajectories can be generated. perceptual experiences. Also. especially in the higher cog- nitive level. might be more abstract and those at lower levels might be more concrete and detailed. the higher level may combine a set of subintentions for primi- tive actions such as reaching-​to-​cup. well-​ formed through adaptation. which are associ- ated with various intentional interactions with the world. What I  propose is this:  We need a neurodynamic system. as well. as combinations of symbols.” which are represented in terms of logical operators. and these chunks can be referenced as things in themselves. various combinatory sequences of the primitive actions could be generated. as propositions.2 illustrates the idea. as can be seen in bifurcations from one attractor structure to another or in phase transitions by means of controlling relatively low-​dimensional external parameters. This modification of the intentions for action in the bottom-​ up process can be achieved by utilizing information about the prediction error. symbolized. What I am saying here is that the segmentation of “thinking” into discrete “thoughts. can be per- formed by dynamic models of mind that do not employ discrete symbolic computation in their internal operations (Tani & Fukumura. its cognitive mind should also reflect on unex- pected outcomes through the bottom-​up process to modify the current intention. continuous descriptions of perception into the theoretical language of discrete. we may suppose that the higher level sending sequences of parameter values to the lower level in the network results in sequential switching of primitive actions by means of the parameter bifurcation in this lower neurodynamic system.) What about creative composition of primitives into novel sequences of action? Neurodynamic models account for this capacity.1. proposing that the promise of symbolic dynamics lies in articulating the transition from dynamical. as described in section 5. And if the neurodynam- ics in the higher level for generating these parameter sequences is driven by its intrinsic chaos. Dale and Spivey (2005) have provided a sym- pathetic argument.146 146 Emergent Minds: Findings from Robotics Experiments systematicity and gives an impression that as if discrete symbols existed within the system as well as that as if these symbols were manipulated by that system.2 illustrates the process whereby the state values in . algorithmic processes for high-​level cognition. 1995. Although an agent driven by top-​down intentions for action has proac- tive subjective views on events experienced during its interaction with the objective environment. This chaotic dynamic can produce combinatory sequences of symbols in terms of symbolic dynamics via partitioning processes of the continuous state space with a finite number of labels. if we remember that the sensitivity of chaos toward initial condi- tions exhibits a sort of combinatory mechanics by folding and stretching in phase space. the possibility of which having been briefly mentioned in the previ- ous section. In simpler terms. Nonlinear dynamics can exhibit structural changes of varying discrete- ness. represented. So. the continuous space of action can be cut up into chunks. Figure 6. The model of ‘compositional’ or ‘symbolic’ mind to which I am now pointing is not impossible to achieve in a neurodynamic sys- tem. Figure 6. .65. These state values are input to a parameterized dynamic system in the lower level as its parameters successively (along the solid arrow) cause sequential bifurcation in the parameterized dynamic system and associated action primitives. Bottom-up error related signal Lower level Sequences of parameter bifurcation Predicting sequences of action primitives parameter (0..37.37. 0.24).91). (0.. 0.91. ify Sequences of state values sampled at Poincaré section.. In the higher level.  An illustration showing how chaos can generate diverse sequential combinations of action primitives. (0. Mod (0.24) parameter (0.91.) .. The lower level predicts the coming visuo-​proprioceptive state and its prediction error is monitored. 0.  147 Higher level Set intention in terms of initial state.2.91) Proprioception Proprioception Vision Vision Predicting visuo-proprioceptive state Error Actual Figure 6. 0.55) . 0. the state trajectory is generated by a chaotic dynamic system with a given initial state and the state values are sampled at each time they cross a Poincaré section. The state in the higher level is modified in the direction of minimizing this error (along the dashed arrow. weight. Figure 6. and with this newfound clarity we may anticipate many his- torical problems regarding the nature of representation in cognition in philosophy of mind to finally dissolve. they can interact seamlessly and thus densely.3 illustrates conceptually how the interactions between top-​down and bottom-​up processes take place in the course of executing intended actions. These two processes interact.148 148 Emergent Minds: Findings from Robotics Experiments the higher level are modified to minimize the prediction error in the lower level. what I am suggesting is that nonlinear neurodynamics can support discrete computational mechanics for compositionality while preserving the metric space of real-​number systems in which physical properties such as position. speed. My tempting speculation is that the authen- tic being could be seen in a certain imminent situation caused by such error or conflict between the two. In summary. the significance of symbolic expression is not only retained on the neurodynamic account but it is clarified. so-​ called hybrid architectures.3 (left panel). as it were. 6. In this way.  The Subjective Mind and the Objective World as an Inseparable Entity Next. Meanwhile. resulting in the “recognition” of the perceptual reality in the subjective mind and the “generation” of action in the objective world (middle panel). and color can be represented. “out of place” and thus at a difference from its own self-​projection. not like symbols and patterns that interact somewhat awkwardly in more common. Because both of these share the same phase space in a coupled dynamical sys- tem. let’s extend such thinking further and examine how the subjec- tive mind and the objective world might be related. This “recognition” results in the modification of the subjective mind―and . This error signal might convey the experience of consciousness in terms of the first-​person awareness of one’s own subjectivity because the subjective intention is directly differentiated from the objective reality and the subject feels. neurodynamic systems are able to host both semantically combinatorial thoughts at higher levels and the corre- sponding details of their direct perception at lower levels.3. It is thought that the intention of the subjective mind (top-​down) as well as the perception of the objective world (bottom-​up) proceeds as shown in Figure 6. The cognitive mind is best represented by nonlinear dynamical systems defined in the continuous time and space domain.  149 New Proposals 149 subjective mind subjective mind subjective mind intention intention modify predict recognize predict recognize predict perceive perceive act perceive act physical interaction objective world objective world objective world Figure 6. whereas the current intention is adapted by using bottom-​up signals of error detected between the prediction and the actual perceptual outcome in the action-​perception cycle. as its subjective mind and the objective world could finally become inseparable. . Both natural and artificial cognitive systems should be capable of predicting the perceptual outcome for the current intention for acting on the outer world via top-​down pathways. and the interactions continue with the modified states of the mind and the world (right panel). This circular causality results in inseparable flows between the subjective mind and the objec- tive world as they reciprocally intertwine with each other via action-​ perception cycles. wherein their nonlinearity can provide the cognitive competence of compositionality.  The subjective mind and the objective world become an insep­ arable entity through interactions between the top-​down and bottom-​up pathways. potential consciousness―whereas the “generation” of action modifies the objective world. If we were able to achieve this scenario in a robot.3. we see the circular causality between action and recognition. I want to conclude this chapter by pointing out what I consider to be essential for constructing models of the cognitive mind: 1. 1968). the robot would be free from Descartes’s Cartesian dualism. 2. as Merleau-​Ponty proposed (Merleau-​Ponty. Redrawn from Tani (1998). In this process. The next chapter examines how robots can lean about the outer environment by using a sensory prediction mechanism in the course of exploration. especially how thought and its interaction with the world could arise. We concentrated on models and experiments with specific focuses. mechanisms. readers will be able to share the deep insights into the nature of the mind. My hope is that. whicht I have come to in performing and reflecting on the actual experiments day-​to-​day. my colleagues and I have never tried to put all of the assumed elements of the mind that we have discussed thus far into our synthetic robotic models. The remaining chapters test these conjectures by reviewing a series of synthetic robotics experiments conducted in my laboratory. in reviewing the outcomes of our series of synthetic robotic stud- ies. The underlying structure for consciousness and free will should be clarified by conducting a close examination of nonstationary characteristics in the circular causality developed through the aforementioned top-​down and the bottom-​up interaction between the subjective mind and the objective world. in dynamic neural network models of the RNN type.” so to speak. therforee in each new trial we added elements relevant to the focus and removed irrelevant ones. and anatomy into the brains of our tiny robots. It was not our aim to put all available neuroscience knowledge about local functions. We tried neither to implement all possible cognitive functions into a particular robotic model nor to account for the full spectrum of phenomenologi- cal issues in each specific experiment. they became consolidated over time as the modeling studies were conducted.150 150 Emergent Minds: Findings from Robotics Experiments 3. It also explores the issue of self-​consciousness as related to this sensory prediction mechanism. Moreover. in each trial we varied and developed “minimal brains. . Rather. Readers should be aware that my ideas were not at all in a concrete nor com- plete form from the very outset. The essence of authentic being also might be clarified via such examination of the apparent dynamic structure. Instead. if a robot becomes able to acquire a subjective view of the world. I attempt to clarify its underlying structure by examining the possible interaction between top-​down prediction and bottom-​up recognition during robot navigation.  151 7 Predictive Learning About the World from Actional Consequences The previous chapter argued that understanding the processes essential in the development of a subjective view of the world by way of interac- tive experiences within that world is crucial if we are to reconstruct the cognitive mind in another medium. These experiments were conducted in a relatively simple setting more than 20 years ago in my lab. But. this chapter reviews a set of robotics experiments in the domain of navigation learn- ing. such as in our neurodynamic robots. how exactly can robots develop such subjective views from their own experiences? Furthermore. 151 . how does it also become aware of its own subjectivity or self? In considering these questions. but they addressed two essential ques- tions. The second experiment inquires into the phenomenology of the “self” or self-​consciousness. The first experiment addresses the question of how the compo- sitional representation of the world can be developed by means of the self-​organization of neurodynamic structures via the accumulated learn- ing of actional experiences in the environment. a robot navigating a workspace . As my colleagues and I  had just completed experiments on robot navigation learning with homing and cyclic routing. achieving the convergence of learning with the sensory–​ motor data acquired in the original workspace proved to be very difficult. I  decided to pursue this new problem in the context of robot navigation. I wondered if robots could also develop a similar competence via learning.  Development of Compositionality: The Symbol Grounding Problem In the mid-​1990s. I thought that a recurrent neural network (RNN) would work as a forward dynamics model that predicts how the sensa- tion of images at range changes in response to arbitrary motor command inputs for two wheel drives at every 500-​ms time interval. From this assumption. provided that the RNN had already learned a sufficient number of branching sequences in the topological trajecto- ries. Instead. First. By utilizing this scheme. I tried to apply the forward dynamics model proposed by Masao Ito and Mitsuo Kawato (see c­ hapter 4) directly to my Yamabico robot navigation problem. 1996).152 152 Emergent Minds: Findings from Robotics Experiments 7. I speculated that the RNN could acquire com- positional images while traveling around the workspace by combining various branching decisions. A focal question that the experiment was designed to address was this one: What happens when the prediction differs from the actual out- come of the sensation? In this situation. I decided to employ the scheme of branching with collision-​free maneu- vering shown in section 5. the problem could be simplified to one wherein an RNN learns to predict just the sensation of the next branch- ing point in response to action commands (branching decision) at the current branching point. The rea- son for this failure was that it was just asking too much of the network to learn to predict the sensory outcomes for all possible combinations of motor commands at each time step. collision-​free maneuvering. This branching scheme enables the robot to move along “topological” trajectories in a compositional way by arbitrarily combining branching decisions in sequence. Because humans can men- tally generate perceptual images for various ways of interacting with the world. it seemed reasonable to assume that the trajectory of the robot should be generated under the constraint of smooth. I  started to think about how robots could acquire their own images of the world from experiences gathered while inter- acting with their environment (Tani. as described in c­ hapter  5. However.6 again.1. the robot explores a given environment containing obstacles by taking random branching decisions. the RNN is trained so that it can predict the next sensory input pn+1 in terms of the current sensory input pn and the branching action xn taken at branching point n (see the right panel in Figure 7. Using these sample pairs of sensory inputs and actions.1). where it receives sensory input (range image vector plus travel distance from the previous branch point) pn and randomly determines the branching (0 or 1) as xn. Let’s assume that the robot arrives at the nth branching point. . Once the RNN is trained.  An RNN learning to predict sensation at the next branching point from the current branching decision. discussed in c­ hapter 2. In this predictive navigation task. The robot acquires a sequence of pairs of sensory inputs and actions (pn. 7.  153 Predictive Learning About the World from Actional Consequences 153 by referring to an internal map with a finite state machine (FSM)-​like representation of the topological trajectories would experience the sym- bol grounding problem (see Figure 2. it can perform two types of prediction.1  Navigation Experiments with Yamabico In the learning phase.1). The actual training of the RNN is conducted in an offline manner with the sample sequence data saved in short-​term memory storage.1.2).1. the context units in the RNN play the role of storing the current state in the work- ing memory. xn) throughout the course of exploring its environment. which is analogous to the previous Yamabico experiment described in c­ hapter 5. after which it moves to the (n+1)st branching point (see the left side of Figure 7. One is the online prediction of the sensory inputs at the next branching Prediction of sensation at next branch Context loop Pn+1 Cn+1 p3 x3 x2 p2 x1 p1 Cn Pn Xn Sensation at current branch Current Branch decision Figure 7. and the rightmost four units are the context units. In the forward dynamics of an RNN with a closed sensory loop. the robot’s performance for online one-​step prediction was tested. However. Figure 7.g. It navigated the workspace from arbitrarily set initial positions by following an arbitrary action program of branching and tried to predict the upcoming sensory inputs at the next branching point from the sensory inputs at the cur- rent branching point. sensory input sequences “entrained” context activations into the normal/​steady-​state transition sequence. initially. Although. the following unit is the action command (branching into 0 or 1).2a) and undergoing offline learning for one night.2b presents an instance of this process. the prediction failed at the very beginning.154 154 Emergent Minds: Findings from Robotics Experiments point for an action taken at the current branch.1. the leftmost five units represent sensory input.. We also found that although the context was easily lost when perturbed by strong noise in the sensory input (e. as the robot contin- ued to travel. In the evaluation after the learning phase. This enables the robot to perform the mental simulation of arbitrary branching action sequences as well as goal-​d irected planning to achieve given goal states. The other is the offline look-​ahead prediction for multiple branching steps while the robot stays at a given branching point. after which the RNN became capable of producing correct predictions. Look-​ahead prediction is performed by making a closed loop between the sensory prediction output units and the sensory input units of the RNN. as denoted with a dotted line in Figure 7. wherein the left panel shows the trajectory of the robot as observed and the right panel shows a comparison between the actual sensory sequence and the predicted one. when the robot failed to detect a . Because the context units were initially set randomly. as described later. it became increasingly accurate after the fourth step. the robot was tested for its predictive capacity during navigation of the workspace. the next five units represent the predicted state for the next step. The figure shows the nine steps of the branch- ing sequence. After exploring the workspace for about 1 hour (see the exact tra- jectories shown in Figure 7. arbitrary steps for look-​ahead prediction can be taken by feeding the current predictive sensory outputs as sensory inputs in the next step instead of employing actual external sensory inputs. We repeated this experiment with various initial settings (different initial positions and different action programs) and the robot always started to produce correct predictions within 10 branch steps. the robot could not make correct predictions. Adopted from Tani (1996) with permission. This autore- covery feature of the cognitive process is a consequence of the fact that a certain coherence in terms of the close matching between the inter- nal prediction dynamics and the environment dynamics emerges during their interaction. and (c) generated after offline look-​ahead prediction (left) and comparison between an actual sensory sequence and its look-​ahead prediction (right). .  155 Predictive Learning About the World from Actional Consequences 155 (a) (b) sensory one-step prediction sequence sequence branching step start p p x c sensory look-ahead prediction (c) sequence sequence branching step start p p x c Figure 7. branch and ended up in the wrong place).  Trajectories of Yamabico (a) during exploration. (b) during online one-​step prediction (left) and comparison between the actual sensory sequence and its corresponding one-​step prediction (right). the prediction accuracy was always recovered as long as the robot continued to travel.2. The robot.3. The arrow in the workspace in the left panel of the figure denotes the branch- ing point where the robot performed look-​ahead prediction for an action program represented by the branching sequence 1100111. the robot generated three different action plans. It can be seen that the look-​ahead prediction agrees with the actual sequence.3 shows the result of one particular trial. after conducting look-​ahead prediction. We repeated this experiment of look-​ahead prediction for various branching sequences and found that the robot was able to predict sensory sequences correctly for arbitrary action programs in the absence of severe noise affecting the branching sequence. It is also observed that the context val- ues as well as the prediction of sensory input at the initial and final steps are almost the same. The figure shows the three corresponding trajectories successfully reaching a given goal from a starting position in the adopted workspace. Finally. the robot was instructed to generate action plans (branch- ing sequences) for reaching a particular goal (position) specified by a sensory image. Figure 7. This indicates that the robot predicted its return to the initial position at the end step in its “mental” simulation for traveling along a figure-​8 trajectory. it was able to perorm multistep look-​ahead prediction from branching points. the robot searched for adequate action sequences that could be used to reach the target sensory state in the look-​ahead prediction of sensory sequences from the current state while minimizing estimated travel distance to the goal. Some may consider that the process of the goal-​d irect plan by the RNN is analogous to the one by GPS described in section 2. This self-​organized mechanism enabled the robot to gen- erate diverse navigational plans as if segments of images obtained during actual navigation were combined by following acquired rules. Although the third trajec- tory might look redundant due to the unnecessary loop. each of which was actually executed. generating a figure-​8 trajectory. A  comparison between a look-​ahead prediction and the actual sensory sequence during travel is shown in Figure 7. The right panel in the figure shows a comparison between the actual sensory input sequence and its look-​ahead prediction associated with the action program and the context activation sequence. In the planning process. In this example in Figure 7.2. the creation of such trajectories suggests that a sort of compositional mechanics in the forward dynamics of the RNN had developed as a result of consolida- tion learning.2c. actually traveled following the action program.156 156 Emergent Minds: Findings from Robotics Experiments Once the robot was “situated” in the environment by the entrainment process. because . Adopted from Tani (1996) with permission. Therefore I conducted a phase-​space analysis of the obtained RNN to . 7. Trajectories corresponding to three different generated action programs are shown. such as the look-​ahead prediction of combinatorial branching sequences and the autorecovery of internal contexts by environmental entrainment.  The result of goal-​d irected planning.3. there are crucial differences between the for- mer functioning in a continuous state space and the latter in a discrete state space. as well as how such attractors could explain the observed phenomena. However.2  Analysis of the Acquired Neurodynamic Structure After the preceding experiment. I thought that it would be interesting to see what sorts of attractors or dynamical structures emerged as a result of self-​organization in the RNN and its coupling with the environment. the forward prediction of the next sensory state for actions to be taken at each situation of the robot by the RNN seems to play the same role as that of the causal rule described for each situation in the problem space in GPS. We will come to understand the significance of these differ- ences through the following analysis.1.  157 Predictive Learning About the World from Actional Consequences 157 (a) (b) start start goal goal (c) (d) start goal start goal Figure 7. the RNN in the closed-​ loop mode was dynamically activated for thousands of steps while being fed random branching sequences (1s and 0s). in which the transient part corresponding to the first several hundred steps was excluded. an analysis (a) 1.4 0. the context state shifted from one segment to another at each step.5 1. a magnification of a particular segment shows an assembly of points resembling a Cantor set (Figure 7.4a).4b). and moreover it was found that each segment corresponded to a particular branching point.0 0.4. It was found that. the activation values of two representative context units were plotted for all steps.0 0.5 0.0 0.  Phase space analysis of the trained RNN. (a) An invariant set of an attractor appeared in the two-​d imensional context activation space. Then. We can see a set of segments (Figure7. as shown for the Rössler attractor in ­chapter 5. Additionally. Moreover. This means that the context state initialized with arbitrary values always converged toward steady-​state transitions within the invariant set after some transient period. Adopted from Tani (1996) with permission.8 0. The resultant plot can be seen in Figure 7. (b) A magnification of a section of the space in (a).0 (b) 0. It was like looking at trajectories from the mental simulation of thousands of consecutive steps of random branching sequences in the workspace while ignoring the initial transient period of state transitions. The plot represents the invariant set of a global attrac- tor.158 158 Emergent Minds: Findings from Robotics Experiments examine its dynamical structure. as the assembly appears in the same shape regardless of the initial val- ues of the context units or the exact sequences of randomly determined branching sequences.95 context-2 context-2 0 .4.55 context-1 context-1 Figure 7. . One difference was that time integration by forward dynam- ics of the RNN required feeding external inputs in the form of branching action sequences into the network. after convergence was reached. Therefore. which has appeared in the phase space of the RNN by means of iterative random shifts of the dynamical system triggered by given input sequences of random branching. 1995]). The distance between two points in a segment repre- sents the difference between past trajectories arriving at the node. 1994] and random dynamical systems [Arnold. the perturbed context state always returned to the original invariant set after several branching steps because the invariant set had been generated as a global attractor. On the other hand. the context state left the invariant set. Fukushima and colleagues (2007) recently showed supportive biological . Our repeated experiments with different robot workspace configura- tions revealed that the observed properties of the RNN are repeatable and therefore general. as shown in Figure 2. Readers should note. First. However. Interestingly.1. each segment observed in the phase space of the RNN dynam- ics is not a single node but a set of points. they arrive at points in the segment that are also far apart.2.  159 Predictive Learning About the World from Actional Consequences 159 of the aforementioned experiments for online prediction revealed that. This fractal structure is actually a signature of compositionality. Theoretically speaking. we can consider that what the RNN reproduced in this case was exactly an FSM consisting of nodes representing branching points and edges corresponding to transitions between these points. namely a Cantor set spanning a metric space. they arrive at arbitrarily close neighbors in the same segment. If the two trajectories come from different branching sequences. however.3  Is the Problem of Symbol Grounding Relevant? Given that the context state shifted from one segment to another in the invariant set in response to branching inputs. whenever the predictability of the robot was lost due to perturbations. This is analogous to what Cleeremans and colleagues (1989) and Pollack (1991) demonstrated by training RNNs with symbol sequences characterized by FSM regulari- ties. if the two trajectories come from exactly the same branching sequences after passing through an infinite number of steps except for the initial branching points. a set of points in the segment consti- tutes a Cantor set with fractal-​like structures because this infinite num- ber of points should be capable of representing the history of all possible combinations of branching (this can be proven by taking into account the theorem of iterative function switching [Kolen. that the RNNs achieve much more than just reconstructing an equivalent of the target FSM. 7. On the other hand. in the case of an FSM. They just appear as a dynamic closure1 as a result of the convergent dynamics of the RNN.. unexpected sensations during navigation).5. it simply halts operation.  The dynamic closure of steady-​state transitions organized as an attractor (solid arrows) associated with a convergent vector flow (dashed arrows). the observed segments cannot be manipulated or represented explicitly as symbols attached to nodes in an FSM. An illustration of this concept appears in Figure 7.160 160 Emergent Minds: Findings from Robotics Experiments Figure 7. evidence from electrophysiological recording data that CA1 cells in the rat hippocampus encode sequences of episodic memory with a similar fractal structure. The discussion here is analogous to that in ­chapter  5 about the advantages of utilizing dissipative dynamic sys- tems rather than sinusoidal functions for stably generating oscillatory patterns. 1980). Second. The nature of global convergence of the context state toward steady-​ state transitions within the invariant set as a dynamic closure can afford global stability to the predictive dynamics of the RNN. there is no autorecov- ery mechanism against perturbations in the form of invalid inputs because the FSM provides only a description of steady-​state transitions within the graph and cannot account for how to recover such states from dynamically perturbed ones.5. As mentioned earlier. 1.g. It is called a dynamic closure because the state shifts only between points in the set of segments in the invariant set (Maturana & Varela. when an FSM receives invalid symbols (e. .   Predictive Dynamics and Self-​Consciousness This section examines how the notion of “self” or self-​consciousness could emerge in artificial systems as well as in human cognitive minds through the review of further robotics experiments on the topics of prediction learning in navigation as extended from the aforementioned Yamabico ones. The current scheme utilizing the forward model is limited to small-​ scale problems because of the frame problem discussed in section 7. What the experiment showed is so-​called latent learning in which an agent learns an internal model of the environment via random exploration without any inten- tions. it can be said that this system is not affected by the symbol grounding problem because there are no “sym- bols” to be grounded to begin with. such learning will face the combinatory explosion problem sooner or later. Before moving on. The model worked successfully because the navigation environment was small and the branching scheme preprogramed in the lower level simpli- fied the navigation problem. Ultimately. Another problem concerns the intentionality of the robot. they still miss some essential features. Although how robots can acquire the lower sensory–​motor level skills such as branching or collision-​free maneuver- ing from their own direct sensory–​motor experiences is quite an impor- tant problem. one of which is the . at least not internally. The following robotics experiments clarify the essential role of sensory prediction mechanisms in the possible development of self-​consciousness as presumed in the earlier chapters. 7.1. I should mention some drawbacks to this approach.  161 Predictive Learning About the World from Actional Consequences 161 It is essential to understand that there is no homunculus that looks at and manipulates representations or symbols in the proposed approach. If the robot attempts to learn about all possible exploration expe- riences without any intentions or goals to achieve. we did not address the problem in this study. The next section explores how sensory prediction learning and phe- nomena of self-​consciousness could be related. there are just iterations of a dynamical system whereby com- positionality emerges. Although the experiments with Yamabico described in the previ- ous section revealed some interesting aspects of contextual predictive dynamics. Rather.2. by reviewing results of another type of robot navigation experiment. We return to these issues in the later sections. Recent evidence from neuroscience has revealed brain waves related to prediction error. it would be interesting to incorporate such a system with dynamic or incremental learning of experiences rather than looking at the result of one-​time offline “batch” learning.6a (Tani. Therefore. and it is speculated that they are used for fast modification of ongoing brain processes. as shown in Figure  7. while navigating the workspace by following the wall and the edge between the wall and the floor. Yamabico did not have a particular bias or attention control for acquiring sensory input. visual patterns of landmarks corresponding to colored objects were . It should be noted that the navigation scheme did not include branching as in the case of Yamabico. After a successful learning process. The error signal is considered to be a crucial cue for recognizing a gap between the subjective image and objective reality.2. Our findings in these robotics experiments enriched with these new elements suggest a novel interpretation of concepts such as the momentary self and minimal self. we introduced a visual system with an attention control mechanism in the robot platform that suc- ceeded Yamabico.162 162 Emergent Minds: Findings from Robotics Experiments utilization of prediction error signals.6b. as in the case of Yamabico. because the learning of compositional navigational paths was not the focus of research in this robot study. which correspond to ideas devel- oped by William James (1982) and Martin Heidegger (1962). The entire network consisted of parts responsible for prediction (performed by an RNN) and parts responsible for percep- tion. In the “what” pathway. 7. 1998). The task of this robot was to learn to dynamically predict landmark sequences encoun- tered while navigating a confined workspace. The robot was controlled by the neural network architecture shown in Figure  7. as in the case of mismatched negativity. Finally.1  Landmark-​Based Navigation Performed By a Robot with Vision I built a mobile robot with vision provided by a camera mounted on a rotating head. the latter being divided into “what” and “where” pathways thereby mimicking known visual cortical structures. It would naturally be expected that the addition of some attention control mechanism would reinforce our proposed framework of top-​down prediction–​expectation versus bottom-​up recognition. the robot was expected to be able to use its vision to recognize landmarks in the form of colored objects and corners within a reasonable amount of time before colliding with them. Also. Adopted from Tani (1998) with permission. (b) The neural network architecture employed in the construction of the robot. (a) A mobile robot featuring vision is looking at a colored landmark object.6.   163 (a) (b) Prediction by RNN sensory prediction sensory input Association network p loo categorical output text con ‘where’ ‘what’ categorical output left l whe whee winner take all neurons el right Hopfield net motors pop-up visual field camera Figure 7.  A vision-​enabled robot and its neural architecture. . a mismatch in the “where” perceptual category could result in failure to attend any of the expected landmarks to be recognized. which can store multiple static pat- terns by using multiple fixed-​point attractors. Such misrecognition outcomes were fed into the RNN. Actual recognition of the landmark objects was established by dynamic interactions between the two pathways. Note that there were no action inputs in this RNN because there was no branching in the current setting. the bottom-​up and top-​down pathways did not merely provide inputs and outputs to the system. were processed by the Kohonen network. Note that the RNN was capable of engaging in “mental rehearsal” of learned sequential images by constructing a closed loop between the prediction outputs and the sensation inputs in the same way as Yamabico. Rather. Together with both pathways. known as a Kohonen net- work. as well as the corresponding direc- tion determined by the camera orientation. Learning was initiated for both the Hopfield and Kohonen net- works whenever a visual stimulus was encountered. . and the direc- tions of the detected landmarks in frontal view. the pattern was recognized and its categorical output was generated by a winner-​takes-​a ll activation network. the RNN learned to predict in a top-​down manner the perceptual categories of “what” and “where” for landmarks to be encountered in the future. This means that if the top-​down prediction of the visual pattern failed to match the currently encountered one. were sent for prediction in a bottom-​up manner. In this model. accumulated encoder readings of the left and right wheels from the last encountered landmark to the current one. Moreover. and the next prediction was made on this basis. in which its categorical outputs were generated.164 164 Emergent Minds: Findings from Robotics Experiments processed in a Hopfield network. In the “where” pathway. “what” categories of visual landmark objects and “where” categories of the relative travel distance from the last landmark to the current one. they existed for their mutual interactions. and the system was prepared for expected percep- tual categories in the top-​down pathway before actually encountering the landmarks. the perception would result in an illusion constituting a combination of the two patterns. When a perceived visual pattern converged toward one of the learned fixed-​point attractors. In the prediction process. This expectation ensured that the system was ready for the next arriving pattern in the Hopfield network and was prepared to direct the camera toward the landmark with correct timing and direction. top-​down prediction dominated the perception. which resulted in quick convergence in the Hopfield network. in reality the process would not be so straightforward as this if the rehearsed and the newly acquired experiences conflict with . taking longer to look for landmarks while waiting for convergence in the Hopfield network. in the new scheme. and catastrophic forgetting of existing memory was avoided by retraining the RNN with both the rehearsed sequences and the newly experienced ones. the bottom-​up pathway dominated the perception. It has been considered that generalization of our knowledge proceeds sig- nificantly through consolidating newly acquired knowledge with older knowledge during sleep. 1995). in the case of fewer errors. On the other hand. Learning of the RNN was conducted for event sequences associated with encountering landmarks. More specifically. 1994. Otherwise. A shorter time period was also allocated for reading the per- ceptual outcomes in the Hopfield network in this case. RNNs lose previously learned memory content quite easily when new sequences are learned. the RNN “rehearsed” previously learned content with the closed-​ loop operation and stored the generated sequences in the “hippocampus” (corresponding to short-​term memory) together with the newly acquired sequences. experienced sequences of perceptual category outcomes were used as target sequences to be learned. Incremental training of the RNN was conducted after every 15th landmark by adopting a scheme of rehearsal and consolidation. In other words. Therefore. whereby the attention was quickly turned to upcoming expected landmarks. Squire & Alvarez. thereby altering acquired connection weights. The mechanism exerted more top-​ down pressure on the two perceptual categories (“what” and “where”) as the error between the predicted perception and its actual outcome decreased. Our robot actually stopped for rest when this rehearsal and consolidation learning was taking place after every fixed period. This rehearsal and consolidation might correspond to dreaming dur- ing the REM sleep phase reported in the literature on consolidation learning (Wilson & McNaughton. However.  165 Predictive Learning About the World from Actional Consequences 165 A particular mechanism for internal parameter control was imple- mented to achieve adequate interactive balance between the top-​down and bottom-​ up pathways. so that phenomena such as “catastrophic forgetting” could be avoided. and a longer time period was allowed for dynamic perception in the Hopfield net- work. less top-​down pressure was exerted when the error between the predicted perception and its actual outcome was larger. the state alternates between the strange attractor (chaos) and the limit cycle attractor with a periodicity of 5. After the fifth learning period. in most cases before the third learning period. the predictability was improved to a certain extent in all three trials. it should be noted that this limit cycle with a periodicity of 5 . as indicated by the strange attractor in the corre- sponding phase plot.  In fact. shown in Figure 7. the five points represent a dynamic closure for the steady-​state transitions between these five landmarks. Importantly. a highly cha- otic region appears. One of the aims behind the next experiment I will describe was to examine this point. It was repeated three times.7a. However. A typical example is shown in Figure 7.2  Intermittency During Dynamic Learning The experiment was conducted in a confined workspace containing five landmarks (two colored objects and three corners). it becomes a limit cycle with a periodicity of 5. limit-​cycle dynamics with a periodicity of 5 appeared most frequently in the course of all trials. After the first learn- ing period.2. After the third learning period. Prediction failures occurred intermittently in the course of the trials. bifurcation of the RNN dynam- ics due to iterative learning. We monitored three characteristic features of the robot’s navigation behavior in each run: prediction error. a quasiperiodic or weakly chaotic region appears. Then.166 166 Emergent Minds: Findings from Robotics Experiments each other. but the errors were not eliminated completely. a snapshot is shown in the phase plot containing five points. Indeed.7a. as can be seen from the five points plotted in the bifurcation diagram at each step during this period. In addition. and phase plots representing the attractor dynamics of the RNN at particular times during the bifurcation process. after the fourth learning period. a fixed-​point attractor appearing in the early periods of the learning iterations as a single point is plotted at each step in the bifurcation diagram. A periodicity of 5 is indicative because it cor- responds to the five landmarks that the robot encountered in a single turn around the workspace. In a typical example. which was a limit imposed by the battery life of the robot. The prediction error was quite high at the beginning of all trials because of the initially random connection weights. and in each trial the robot circulated the workspace about 20 times. 7. and we can see from the bifurcation diagram that the dynamical structure of the RNN varied. 1998) with permission. (b) The robot’s trajectories as recorded in the unsteady and steady phases.5 neural 0. does not remain stationary. bifurcation diagram of the RNN dynamics.  Experimental results for a vision robot.  167 Predictive Learning About the World from Actional Consequences 167 (a) 1 prediction error 0. because the periodicity disappears at times and other dynamical structures emerge.0 0 1 2 3 4 5 6 7 learning times 4 5 7 c1 c1 c1 c2 c2 c2 (b) Unsteady phase Steady phase Figure 7.5 0 0 15 30 45 60 75 90 10 5 event steps 1.0 activation state 0. and phase plot for two context units at particular times during the learning process. The dynamic closure observed in the current experiment is not stable but changes in the course of dynamic learning. Adopted from (Tani.7. this can be interpreted as the robot could mentally simulate various . From the view of symbolic dynamics (see ­chapter 5). (a) Prediction error. 1990) in which state trajec- tories tend to visit multiple pseudoattractors one by one itinerantly in a particular class of networks consisting of dynamic elements. Finally.1. which was indeed the case in the experiments. 1990.8). the detection sequence of landmarks became more deterministic and travel was smooth. The trajec- tory was more winding in the unsteady phase than in the steady phase. the robot faced a higher risk of misdetecting landmarks when its trajectory meandered during this period. Figure 7. particularly in the way objects and corners were approached.). We also see that transitions between these two phases took place arbitrarily over the course of time. Aihara et al. The graph indicates that the distribution of the breakdown interval has a long-​tail characteristic with a power-​law-​like profile.5) observed in three different experi- ments of the robot (Figure 7. The observation here might be also analogous to the so-​called phenomenon of chaotic itinerancy (Tsuda et  al. without dominant periodicity. and that differences appeared in the physical movements of the robot concurrently. we compared the actual robot trajectories observed in these two periods. From these results. Tsuda and colleagues (1987) showed that intermittent chaos mechanized by means . This indi- cates that the shift from the steady to the unsteady phase takes place intermittently.. we measured the distribution of interval steps between cat- astrophic error peaks (error >0. The observed intermit- tency might be due to the tangency developed in the whole dynamics (see section 5. however. 1987. What is important here is that these steady and unsteady dynamics are attributable not only to the internal cognitive processes arising in the neural network.. and an unsteady phase characterized by nonperiodic dynamics. From this it was inferred that the robot’s maneuvers were more unstable in the unsteady phase because it spent more time on the visual recognition of objects due to the higher prediction error. So. we can conclude that there were two distinct phases: a steady-​state phase represented by the limit-​cycle dynamics with a periodicity of 5. Kaneko.168 168 Emergent Minds: Findings from Robotics Experiments symbolic sequence structures for encountering landmark labels includ- ing deterministic symbol sequences of a period of 5 and the ones with probabilistic state transitions during the rehearsal.7b shows the robot trajectories measured in these two periods with a camera mounted above the workspace. 1989. but also were expressed in the physical movements of the robot’s body as it interacted with the external environment.. with greater prediction success. To clarify why this happened. In the steady phase. Ikeda et al. All the cognitive and behavioral processes proceed smoothly and auto- matically.2. whereby . In the unsteady phase. The robotics experiment described in this section has demonstrated that phenomena similar to chaos itinerancy could also emerge in the learning dynamics of a network model coupled with a physical environ- ment. of tangency in nonlinear mapping (see section 5.3  Accounting for the “Minimal Self” An interesting observation from the last experiment is that the transi- tions between steady and unsteady phases occurred spontaneously. where the x axis represents the interval steps and the y axis represents the frequency of appearance in the corresponding range. even though the workspace environment was static. coher- ence is achieved between the internal dynamics and the environmental dynamics when subjective anticipation agrees closely with observation. The dynamic learning processes while interacting with the outer environments can generate complex trajectories that alternate between stabilizing the memory contents and their breakdown. In the steady phase. it is at this moment of incoherence that the “self-​consciousness” of the robot arises.5.1) generated the chaotic itinerancy observed in his memory dynamics model. 7. Consequently.  Distribution of interval steps between catastrophic prediction error peaks greater than 0. this distinction becomes rather explicit as conflicts in terms of the prediction error are gener- ated between the expectations of the subjective mind and the outcome generated in the objective world. and both axes are in log scale.  169 Predictive Learning About the World from Actional Consequences 169 16 8 Frequency (Times) 4 2 1 1 4 16 64 128 Interval (Steps) Figure 7. and no distinction can be made between the subjective mind and the objective world.8. Ultimately. learning. catastrophic failure in the recognition of landmark sequences can occur as a result of even minor . one question still remains for us to address here:  Why couldn’t the coherence in the steady phase last longer and the break- down into incoherence take place intermittently? It seems that the com- plex time evolution of the system emerged from mutual interactions between multiple local processes. The aforementioned circular causality can be explained more intu- itively as follows. perception. Gallagher (2000) considered that this “momentary self” is in fact a “minimal self.4) as well as in James’ concept of the stream of consciousness (see section 3. This is because the same periodic patterns are learned repeatedly and the robot tends to trace exactly the same trajectories in the steady phase. which is proportional to the magnitude of the current error. more strict timing of visual recognition is required for upcoming landmarks because only a short period for recognition of the objects is allowed.6) in which the inner stream consists of transient and substantive parts. the top-​down image for each upcoming landmark pattern is shaped into a fixed one. If all goes completely as expected. and the self can become consciously aware momentarily in the discrete event of breakdown. prediction. However. which led to modification of the dynamic memory stored in the RNN and a consequent change in predictability. wherein we see the circular causality between the subjec- tive mind and the objective world. Dynamic interactions took place as chain reactions with certain delays among the processes of recognition. In addition. It was observed that changes in the visual attention dynamics due to changes in the predictability caused drifts in the robot’s maneuvers. and acting. as there are no conflicts demanding the system’s attention. When the learning error decreases as learning pro- ceeds. These drifts resulted in misrecogni- tion of upcoming landmarks. This interpretation of the experimental observations corresponds to the aforementioned analysis in Heidegger’s example of the hammer miss- ing the nail (see section 3.” which should be dis- tinguished from the self-​referential self or narrative-​self provided with a past and a future in the various stories that we tell about ourselves. this circular causality might then provide a condition for developing a certain criticality. without variance. So. With reference to the Scottish philosopher David Hume. On the other hand. this strictness grows as the prediction error decreases fur- ther. the “self-​consciousness” is reduced substantially. at the peak of strictness.170 170 Emergent Minds: Findings from Robotics Experiments the system’s attention is directed toward the conflicts to be resolved. in the steady phase. its sides became steeper.4). Incidentally. the size of the avalanches is distributed in accordance with a power law. In this. however. the . In their simulation study.” which James metaphorically spoke of as an alternation of periods of flight and perching throughout a bird’s life. the Game of Life. just one more grain would have triggered an avalanche. such as earthquakes. I consider that this critical state is analogous to the situation of generating catastrophic failures in recognizing the landmarks in the robotics experiment. the vivid “nowness” of a robot might be born in this criticality as a consequence of the dynamic interplay between looking ahead to the future for possibili- ties and regressing to the conflictive past through reflection. and it is found to be ubiquitous in various other phenomena as well. Bak found that although it is impossible to pre- dict exactly when the avalanche will occur. and stock markets. Here.1) can be preserved with structural stability in the system. Although we might need a larger experimental dataset to confirm the presence of SOC in the observed results. one at a time.  171 Predictive Learning About the World from Actional Consequences 171 noise perturbation because the entire system has evolved too rigidly by building up relatively narrow and sharp top-​down images. that the potential for an authentic robot arises from this fragility (Tani. It is as if a critical situation such as “tangency” (see section 5. A crucial point is that the evolution toward a certain critical state itself turns out to be a stable mechanism in SOC. 2009). grains of sand were dropped onto a pile. By following Heidegger. I speculate that some dynamic mechanisms for generating criticality could be responsible for the auton- omous nature of the “momentary self. At that very moment. The described phenomena remind me of a theoretical study con- ducted on sand pile behavior by Bak and colleagues (1987). 1987). remembering what Heidegger said about the authentic being of man. The natural growth of the pile to a critical state is known as self-​organized criticality (SOC). volcanic activity. I argue. eventually reaching a critical state. the structure of consciousness responsible for generat- ing the momentary self can be accounted for by emergent phenomena resulting from the aforementioned circular causality. As the pile grew. landscape formation.. who resolutely anticipates death as his own-​ most possibility (see section 3. readers may wonder how we can appreciate a robot with such fragility in its behavior characterized by SOC—​the robot could “die” by crashing into the wall due to a large fluctuation at any moment. This seems to be possible in the system with relatively larger dimensions allowing local nonlinear interac- tions inside (Bak et al. It requires differen- tiation from the objective world as another pole by means of interacting with it. The top-​down pathway of predicting perceptual event sequences exemplifies subjectivity because it is developed solely along with the first-​person experiences of perceptual events accumu- lated through iterative interactions in the objective world. the consciousness that is the first-​person awareness of one’s own subjectivity can originate only from a sense of discomfort in one’s own predictability—​t hat is the prediction error. Karl Friston (2010) proposed that a likelihood measure by prediction error divided by an estimate of its variance can represent the “surprise” of the system. Subjectivity as a mirror of the objective world cannot be aware just by itself alone. rather than merely into a particular state of consciousness or unconsciousness at a moment.3. the subject and the object turn out to be an insepa- rable entity by means of the circular causality between them wherein the open dynamics characterized by intermittent transitions between the predictable steady phase and the conflictive unsteady one emerges. we may ask whether the account provided so far could open a new pathway to access the hard problem of consciousness character- ized by Chalmers (see section 4. . I would say “yes” by observ- ing the following logic. The first experiment focused on the problem of learning to extract compositionality from 2. this interpretation of experimental results reviewed in this chapter provides insight into the fundamental structure of con- sciousness.2 which is also the first-​ person experience but in another level of the second order (where the contents of prediction are the first order). And as such. Subjectivity is not a state but a dynamic function of predicting the perceptual outcomes resulting from interactions with the objective world. This measure might quantify the state of consciousness better than simply error itself. Recently. Summary This chapter introduced two robotics experiments on the topics of pre- diction learning in the navigation domain by utilizing mobile robots with a focus on how robots can acquire subjective views of the exter- nal world through iterative interactions with it.3) or not. To this end. 7. Finally.172 172 Emergent Minds: Findings from Robotics Experiments robot may ultimately achieve authentic being in terms of its irreplace- able behavioral trajectories. If this is granted. The experimental results using the Yamabico robot showed that the compositionality hid- den in the topological trajectory in the obstacle environments can be extracted by the predictive model instantiated by RNN. It is interesting to note that. The second experiment addressed the phenomenological problem of “self” by further extending the aforementioned robot navigation experi- ments. One drawback of the models presented for robot navigation in this chapter is that the models could not provide direct experience of perceptual flow to the robots because the model oper- ated in an event-​based manner that was designed and programmed by the experimenters. which are revisited in later chapters. The next chapter introduces a set of robotics experiments focusing on mirror neuron mechanisms in which we con- sider how event-​like perception develops out of the continuous flow of perceptual experience. The navigation of the robot became inherently robust because the mechanism of autore- covery was supported by means of the development of the global attrac- tor in the RNN dynamics. We concluded that symbol-​like structures self-​organized in neurodynamic systems can be naturally grounded in the physical environment by allowing active interactions between them in a shared metric space. The apparent gap between these two has originated from two different research attitudes for exploring cognitive minds. By investigating possible analogies between this result and the phenomenological literature on the self. a vision-​based mobile robot implemented with an RNN model learns to predict landmark sequences experienced during its dynamic exploration of environment. In this new experiment. . the second experiment suggested that the momentary self could appear instead in the sense of the groundlessness of subjectivity. we drew the conclusion that the open dynamic structure characterized by SOC can account for the underlying structure of consciousness through which the “momentary self” appears autonomously. as related to phenomenological problem of time perception.  173 Predictive Learning About the World from Actional Consequences 173 sensory–​ motor experiences and their grounding. although I emphasized the grounding of the subjective image of the world in the first navigation experi- ment. It was shown that the developmental learning process during the exploration switches sponta- neously between coherent phases (when the top-​down prediction agrees with the bottom-​up sensation) and incoherent phases (when conflicts appear between the two). 174   175 8 Mirroring Action Generation and Recognition with Articulating Sensory–​Motor Flow In the physical word, everything changes continuously in time like a river flows. Discontinuity is just a special case. Sensory-​motor states change continuously and neural activation states in essential dimen- sions do so, too as Churchland observed (2010; also see section 4.3). If this is granted, one of the most difficult questions in understanding the sensory–​motor system should be how continuous sensory–​motor flows can be recognized as well as generated structurally, that is, recog- nized as segmented into “chunks” as well as generated with articulation. According to the motor schemata theory proposed by Michael Arbib (1981), a set of well-​practiced motor programs or primitives are stored in long-​term memory, and different combinations of these programs in space and time can generate a variety of motor actions. Everyday actions, such as picking up a mug to drink some coffee can be generated by con- catenating different chunks or behavioral schemes, namely those of the vision system attending to the mug, the hand approaching the handle of the mug in the next chunk, followed by the hand gripping the handle in the final chunk. Similarly, Yasuo Kuniyoshi proposed that complex 175 176 176 Emergent Minds: Findings from Robotics Experiments human actions can be recognized by their structurally segmenting visual perceptual flow of concatenated reusable patterns (Kuniyoshi et  al., 1994). Kuniyoshi and colleagues (2004) also showed in a psychological experiment that recognition of timing of such segmentation is essential to extract crucial information about the action observed. The problem of segmentation is closely related also to the afore- mentioned phenomenological problem of time perception considered by Husserl, which is concerned with the question of how a flow of experiences in the preempirical level can be consciously recalled in the form of articulated objects or events at the objective time level (section 3.2). Please note that we did not address this problem in the previous experiments with the Yamabico robot because segmentation of sen- sory flows was mechanized by the hand-​coded program for branching. Yamabico received sequences of discontinuous sensory states at each branching point. In this chapter, our robots have to deal with a continuous flow of sensory-​motor experiences. Then, we investigate how these robots can acquire a set of behavioral schemes and how they can be used for recognizing as well as generating whole complex actions by segment- ing or articulating the sensory-​motor flow. I presume that mirror neu- rons are integral to such processes because I speculate that they encode basic behavior schemes in terms of predictive coding (Rao & Ballard, 1999; Friston, 2010; Clark, 2015)  that can be used for both recogni- tion and generation of sensory-​ motor patterns, as mentioned previ- ously (see section 4.2). This chapter develops this idea into a synthetic neurorobotics model. The following sections will introduce our formulation of the basic dynamic neural network model for the mirror neuron system. The for- mulation is followed by neurorobotics experiments utilizing the model for a set of cognitive behavior tasks including creation of novel patterns via learning a set of behavior patterns, imitative learning, and acquisition of actional concepts via associative learning between a quasilanguage and motor behaviors. The analysis of these experimental results provide us with some insight into how the interaction between the top-​down prediction/​generation process and the bottom-​up recognition process can achieve segmentation of a continuous perceptual flow into mean- ingful chunks, and how distributed representation schemes adopted in the model can enhance the generalization of learned behavioral skills, knowledge, and concepts.   177 Mirroring Action Generation and Recognition 177 8.1.  A Mirror Neuron Model: RNNPB In this section, we examine a dynamic neural network model, the recurrent neural network with parametric biases (RNNPB) that I and my colleagues (Tani, 2003; Tani et  al., 2004)  proposed as a possible model to account for the underlying mechanism for mirror neurons (Rizzolatti et al., 1996.) The RNNPB model adopts the distributed rep- resentation framework by way of which multiple behavioral schemes can be memorized in a single network by sharing its neural resources. This contrasts with the local representation framework in which each memory content is stored in a distinct local module network separately (Wolpert & Kawato, 1998; Tani & Nolfi, 1999; Demiris & Hayes, 2002; Shanahan, 2006). In RNNPB, the inputs of a low-​d imensional static vector, the para- metric bias (PB) represent the intention for action to be enacted. The RNNPB generates prediction of the perceptual sequence for the out- come of the enactment of the intended action. The RNNPB can model the mirror neuron system in an abstract sense because the same PB vector value accounts for both generation and recognition of the same action in terms of the corresponding perceptual sequence pattern. This idea corresponds to the aforementioned concept about the predictive model in the parietal cortex associated with mirror neurons shown in Figure  4.6. From the viewpoint of dynamical systems, the PB vector is considered to play the role of bifurcation parameters in nonlinear dynamical systems as the PB shifts the dynamic structure of the RNN for generating different perceptual sequences. Let’s look at the detailed mechanism of the model (Figure 8.1). The RNNPB can be regarded as a predictive coding or genera- tive model whereby different target perceptual sequence patterns, pt , t = 0...l -1 can be learned for regeneration as mapped from the cor- responding PB vector values. The PB vector for each learning sequence pattern is determined autonomously without supervision by utilizing the error signals back-​propagated to the PB units, whereas the synaptic weights (common to all patterns) are determined during the learning process as shown in Figure 8.1a. Readers should note that the RNNPB can avoid the frame problem described in section 4.2 because the dynamic mapping to be learned is not from arbitrary actions to per- ceptual outcomes at each time step but from a specific set of actional 178 178 Emergent Minds: Findings from Robotics Experiments (a) (b) (c) Teaching target: pt+1 Perception target: pt+1 Error pt+1 Error pt+1 pt+1 delay line delay line PB PB PB pt ct pt ct pt ct Inferred by Externally Inferred by error-BP set error-BP Learning phase Generation phase Recognition phase Figure 8.1.  The system flow of a recurrent neural network with parametric biases (RNNPB) in (a) learning mode, (b) top-​down generation mode where intention is set externally in the PB, and (c) bottom-​up recognition mode wherein intention in the PB is inferred by utilizing the back-​propagated error. intentions to the corresponding perceptual sequences. This makes the learning process feasible because the network is trained not for all pos- sible combinatorial trajectories but only for selected ones. After the learning is completed, the network is used both for generat- ing (predicting) and recognizing perceptual sequences. The learned per- ceptual sequences can be re-​generated by means of forward dynamics of the RNNPB by the PB set given with values determined in the learning process (see Figure 8.1b). This is the top-​down generation process with the corresponding actional intention represented by the PB. Perceptual sequences can be generated and predicted either in the open-​loop mode by receiving the current perceptual inputs from the environment, or in the closed-​loop mode, wherein motor imagery is generated by feeding back the network’s own prediction outputs into the inputs (dotted line indicates the feedback loop.) On the other hand, in (c), the experienced perceptual sequences can be recognized by searching the optimal PB values that minimize the errors between the target sequences to be recognized and the output sequences to be generated, as shown in Figure 8.1c. This is the bottom-​ up process of inferring the intention in terms of the PB for the given perceptual sequences. As an experiment described later shows, gen- eration of action and recognition of the resultant perceptual sequences   179 Mirroring Action Generation and Recognition 179 can be performed simultaneously. More specifically, behavior is gen- erated by predicting change in posture in terms of proprioception, depending on the current PB, while the PB is updated in the direction of minimizing the prediction error for each coming perceptual input. By this means, the intention–​perception cycle can be achieved in the RNNPB, whereby the circular causality between intention and percep- tion appears. Note also that both action learning and generation are formulated as dynamic processes for minimizing the prediction error (Tani, 2003), the formulation of which is analogous to the free-​energy principle proposed by Karl Friston (2005; 2010). Here, I should explain the learning process more precisely, because its mechanism may not necessarily be intuitive. When learning is com- menced, the PB vector of each training sequence is set to a small random value. The forward top-​down dynamics initiated with this temporarily set PB vector generates a predictive sequence for the training perceptual sequence. The error generated between the target training sequence and the output sequence is back-​propagated along the bottom-​up path iter- ated backward through time steps via recurrent connections, whereby the connection weights are modified in the direction of minimizing the error signal. The error signal is also back-​propagated to the PB units, in which their values for each training sequence are modified. Here, we see that the learning proceeds by having dense interactions between the top-​down regeneration of the training sequences and the bottom-​ up regression of the regenerated sequences utilizing the error signals. The internal structures for embedding multiple behavior schemata can be gradually developed though this type of the bottom-​up and top-​ down interaction by self-​organizing distributed representation in the network. It is also important to note that the generation of sequence pat- terns is not limited to trained ones. The network can create a vari- ety of similar or novel sequential patterns depending on the values of the PB vector. It is naturally assumed that if PB vectors are similar, they would generate similar sequence patterns, otherwise they could be quite different. The investigation of these characteristics is one of the highlights in the study of the current model characterized by its distributed representational nature. The following subsections detail such characteristics of the RNNPB model by showing robotics experi- ments using it. PB mapping shows how points in the PB vector space can be mapped to sequence patterns to be generated after learning a set of target patterns. An impor- tant observation is that the characteristic landscape is quite smooth in the region of discrete movements. the profiles of all generated sequence patterns seem to be generated by interpolations of these three trained sequence patterns.2.  Embedding Multiple Behaviors in Distributed Representation A simple experiment involving learning a set of target motor behaviors was conducted to examine PB mapping.180 180 Emergent Minds: Findings from Robotics Experiments 8. the destination point of the discrete movement changes only slightly. 2. and the remaining area under the dotted curve is for cyclic movement patterns (including nonperiodic ones). In this experiment. in which a structure emerges as a result of self-​organization through the process of learning. the characteristic landscape in the region of periodic movement patterns is quite rugged against changes in the PB values. Teach-​(1.2. The five target movement patterns in terms of four-​d imensional proprioceptive (joint angle) sequence patterns are shown in Figure 8. and 3) are discrete movements with different end points. wherby if the PB vector is changed slightly.2. It can be seen that the PB vectors for all three discrete movement patterns appear in the upper right region and the PB vectors for the two target cyclic movement patterns appear in the lower right region in PB space. and Teach-​(4 and 5) are different cyclic movements. The area above the dotted curve is the region for generating dis- crete movements. as shown in Figure 8. Particularly inside of the triangular region defined by these three PB points corresponding to the trained discrete movements.2. The pro- files of generated patterns could change drastically as compared with changes in the PB vector in this region. . such as Novel-​(1 and 2) shown in Figure 8. which was found to be divided into two regions (the boundary is shown as a dotted curve). Patterns generated from this region could include a variety of novel patterns. an RNNPB was trained on five different movement pat- terns of a robotic arm with four degrees of freedom. The arrows associ- ated with those sequence patterns indicate the corresponding PB vector points determined in two-​d imensional space in the training. On the other hand. Novel-​2 is a nonperiodic pattern that is espe- cially difficult to imagine as being derived from the profiles of the training patterns. 0.0.86.0 1.0 PB1 1.  Mapping from PB vector space with two-​d imensional principal components to the generated movement pattern space.0 PB2 Proprioception (0.0 P[1] Proprioception P[3] p P[4] 0 0 0 0 P[2] 0 20 40 60 80 100 0 5 10 0 5 10 15 0 5 10 Novel-2 time Teach-1 Teach-2 Teach-3 1.0.91) (0.   181 1.71) (0.49) 0 (0.81) 1.0 (0.0 Proprioception 0 0 0 10 20 30 40 0 10 20 30 40 time time Teach-4 Teach-5 Figure 8.57.0 1.0.87.61.0 1.0.0 1.2.29) 0 20 40 time Novel-1 0.78. .0 1. I also intro- duced a robot study by Gaussier and colleagues (1998) that showed that robots can generate synchronized imitation with other robots using acquired visuo-​ proprioceptive mapping under the homeosta- sis principle. and amplitudes are equally difficult to extract. images can be regenerated with spontaneous variations into streams of consciousness (see section 3.3. The former successfully achieves generalization in terms of interpolation of trained sequence pat- terns because it might be easy to extract common structures shared by the three trained discrete movements. The aforementioned experiment result fits very well with James’s thought (James. 1892) that when the memory hosts complex relations or connections between images of past experiences. diverse temporal patterns can be created by changing the PB vector. James predicted this type of phenomena without con- ducting any experiments or simulations but only from formal introspec- tion.2. 8.  Imitating Others by Reading Their Mental States In section 5. I  briefly explained about the development of imita- tion behavior with emphasis on its early stage in which the imitation mechanism is accounted for by simple stimulus response. periodicities.182 182 Emergent Minds: Findings from Robotics Experiments One interesting observation here is that two qualitatively distinct regions appeared. the next subsection looks at the application of the RNNPB model to a robot task of imitation learning. This results in a highly nonlinear landscape in this region due to the embedding of quite dif- ferent dynamic patterns in the same region. On the other hand. in the latter case it is difficult to achieve generalization because structures shared between the two cyclic movement patterns with different shapes. the following subsections introduce a set of cognitive robotics experiments utilizing the RNNPB model with a focus on mirror neu- ron functions. Rizzolatti and colleagues (2001) suggested the neural mechanism at this level as response facilitation without understanding meaning. Now that we have covered the basic characteristics of the RNNPB model. namely the discrete movement part and the cyclic movement part. which exhibit fixed-​point dynam- ics with various destination points. First.6). including nonperiodic patterns. In such a highly nonlinear landscape. Experimental results using monkeys indicated that the same . and bringing it to the mouth are either gener- ated or observed.” I consider that our proposed mechanism for inferring the PB states in RNNPB can account for the “like me” mechanism at this level. The RNNPB learns to predict how the positions of the experimenter’s hands (perceived as a visual image) change in time in terms of dynamic mapping from v t to v t+1. In the learning phase of this experiment. This prediction takes the form of dynamic map- ping of arm proprioception from pt to pt+1 through direct training per- formed by a teacher who guides the movements of the robot’s arms by moving them directly while following the experimenter’s hand movements. it was observed that the same F5 neurons in monkeys fire when purposeful motor actions such as grasp- ing an object.. 2001). In the interac- tion phase. 2004. 8. wherein the focus falls again on the online error regression mechanism used in the RNNPB model (Ito & Tani.. Also. 2009).3).2. in an imitative manner. the robot learns multi- ple hand movement patterns demonstrated by the experimenter. holding it. which is considered to correspond to the third stage of the “like me” mecha- nism hypothesized by Meltzoff (2005). Simultaneously.3.1  Model and Robot Experiment Setup This experiment on imitative interactions between robots and humans was conducted by using Sony humanoid robot QRIO (Figure 8. to predict how its own arms (4DOF joints for each arm) move as corresponding to the observed movements performed by the experimenter. The neural mechanism at this level is called response facilitation with understanding meaning (Rizzolatti et al. as mentioned in section 4. the network also learns. The tutoring is conducted for each movement pattern by determining its corresponding PB vector for encoding. In this stage “my” mental state can be projected to those of others who act “like me. Ogata et al. the robot is expected to recognize it by . when one of the learned movement patterns is demon- strated by the experimenter. Let’s look here at the results of a robotics experiment that my team conducted to elucidate how the recognition of other’s actional intention can be mirrored in one’s own generation of the same action.  183 Mirroring Action Generation and Recognition 183 motor neurons in the rostral part of the inferior parietal cortex are acti- vated when a monkey generates and when he observes meaningless arm movements. Figure 8.184 184 Emergent Minds: Findings from Robotics Experiments Figure 8. When the experimenter switches his/​her demonstration of hand movement patterns from one to another freely.2  Results: Reading Others’ Mental States by Segmenting Perceptual Flow In the current experiment.3. Reproduced from Tani et al. inferring an optimal PB vector for reconstruction of the movement pattern through which its own corresponding movement pattern may be generated. after the robot was trained on four different movement patterns. (2004) with permission. It can be seen that when the movement pattern demonstrated by the experimenter was shifted from one of the learned patterns to another.  Sony humanoid robot QRIO employed in the imitation learning experiment. .3.4 shows one of the obtained results in which the experimenter switched demonstrated movement patterns twice during a trial of 160 steps. the movement patterns generated by the robot should change accordingly by inferring the optimal PB vector. it was tested in terms of its dynamic adaptation to sudden changes in the patterns demonstrated by the experimenter. 8. 8 RYH Hand Position RZH 0.2 Z: Z-axis 0.6 PB 0.4 SH: shoulder 0.2 Y: Y-axis Z: Z-axis 0.0 LZH Predicted Human 0.8 PBN3 PBN4 0.6 L: left 0.0 20 40 60 80 100 120 140 Step Figure 8. The time evolution profile of the perceived position of the experimenter’s hand and the profile predicted by the robot are shown in the first and the second rows. respectively.  185 1.4.  Dynamic changes in the movement patterns generated by the robot triggered by changes in the movements demonstrated by the experimenter.4 0.2 0.8 RSHP 0. Adopted from Tani et al. (2004) with permission.0 H: hand 20 40 60 80 100 120 140 Step Generated Robot Arm LSHP 1.0 LSHR (Joint Angle) LSHY 0.4 R: right Y: Y-axis 0. .0 H: hand 20 40 60 80 100 120 140 Step LYH 1.4 R: right 0.0 PBN1 PBN2 0.2 P: pitch 0.6 RSHR RSHY 0.0 LYH Actual Human Hand LZH 0.8 RYH RZH Position 0. The third and fourth rows show the time profiles for the predicted proprioception (joint angles) of the robot’s arm and the PB vectors.6 L: left 0. respectively.0 R: roll 20 40 60 80 100 120 140 Y: yaw Step 1. what is read from the experimenter’s mind might be his or her “free will” for alternating among primitive patterns. segmentation of the whole compositional sequence into primitives can be performed by using the resultant prediction error. When the same robot was trained for a long sequence that consisted of periodic switching between two differ- ent movement patterns. in terms of retention and protention. the whole sequence was encoded by a single PB vector without segmentation. Even if Husserl’s notion of nowness in terms of retention and protention is understood as corre- sponding to contextual dynamics in RNNs. includ- ing the moment of switching between the movement patterns due to the exact periodicity in the tutored sequence. all moments of perception belong to a single chunk without segmentation. it can be seen that the continuous perceptual flow was segmented into chunks of different learned patterns via sudden changes in the PB vector mechanized by bottom-​up error regression. This self-​ organized contextual flow of the forward dynamics in RNNs could be responsible for the phenomenon of retention.” in combining a set of primitives into the whole. Therefore. The compositionality entails potential unpredictability because there is always some arbitrariness. This description of retention and protention in the preempirical level seems to correspond directly to the forward dynamic undertaken by RNNs (Tani. Our main idea is that nowness is bounded where the flow of experience . perhaps by “free will. RNNs perform predic- tion by retaining the past flow in a context-​dependent way. When everything becomes predictable. as described in section 3. the following question still remains: What are the boundaries of nowness? The idea of segmentation could be the key to answering this question.186 186 Emergent Minds: Findings from Robotics Experiments the visual and proprioceptive prediction patterns were also changed cor- respondingly. This means that RNNPB was able to read the transition of mental states of the experimenter by segmenting the flow. 2004). In this situation. This happened because perception of every step in the trained sequence was perfectively predictable. Here. Husserl assumed that the subjective experience of “now- ness” is extended to include the fringes in the sense of both the expe- rienced past and the future. accompanied by stepwise changes in the PB vector.3. There was an interesting finding that connects the ideas of compo- sitionality and segmentation. The aforementioned results accord with the phenomenology of time perception. the interaction was not trivial for them. This identification process takes a certain period of effort accompanied by “consciousness” because of delays in the convergence of the PB regression dynamics. which are no longer just parts of the flow but rather represent events that are identified as one of the perceptual categories by the PB vector. I claim that projection of “my” mental state to those of others who act “like me” assumed in the third stage of Meltzoff’s (2005) “like me” mechanism should accompany such conscious process. This might also explain the aforementioned observation by Varela (1999) that the flow of events in the immediate past is experienced just as an impression. In the RNNPB model. If they merely attempted to follow the robot’s movement patterns. Five subjects participated in the experiment and each subject was allowed to interact with the robot for 1 hour. In this new experiment. In the imita- tion game. Although most of the subjects eventually identified all of the move- ment patterns.  187 Mirroring Action Generation and Recognition 187 is segmented (Tani. the robot was trained for four move- ment patterns by the experimenters and then human subjects who were unaware of what the robot had learned participated. when the external perceptual flow cannot be matched with the internal flow correspond- ing to the anticipated outcome. which later becomes a consciously retrieved object after undergoing segmentation. Also.3. Our next experiment examined the case of mutual interac- tion by introducing a simple game played by the robot and human subjects.3  Mutual Imitation Game The previous experiment involved unidirectional interaction in which only the robot adapted to movements demonstrated by the experi- menter. as observed in the preceding experiments. Actually. Finally. When the prediction is not fulfilled. 8. 2004). the flow is segmented into chunks. the robot tended to generate diverse movement pat- terns due to fluctuations in the PB. if the subjects attempted to . the resultant error drives PB vector change. the subjects were instructed to identify as many movement patterns as possible and to synchronize their movements with those of the robot through interactions. convergence could not be achieved in most instances because the PB values fluctuated wildly when unpredictable hand movement patterns were demon- strated. the robot could not follow them unless the movement patterns of the subjects corresponded to those already learned by the robot.2.2). We can see that diverse movement patterns are generated by the robot and the human subject. This too can be explained by the mechanism of self-​organized criticality (see section 7. However. accompanied by frequent shifts during their interactions. when they felt that their movements and those performed by the robot could not synchronize. hoping that the robot would start to follow them and eventually synchronize its movements with those of the subject. However. Even small perturbations could confuse the subjects if they were not yet fully confident of the robot’s repertoire of movement patterns. it was often observed that such matching was likely to break down before a match was achieved for another pattern. However. This observation is analogous to the turn tak- ing during imitative exchange observed by Nadel (2002) as described in section 5. they tended to keep the attained synchronization for a short period of time to memorize the pattern.5 in the same format as in Figure 8. they just kept following the movements passively to stabilize the pattern. In postexperiment inter- views. It can be seen that matching by synchronization between the human subject’s movements and the robot’s predictions is achieved after an exploratory phase (see the sections denoted as “Pattern 1” and “Pattern 2” in the figure). The movement patterns of the human and the robot as well as the neural activity (PB units) obtained during interaction in the imitation game are plotted in Figure 8. When the subjects managed to reach a synchronized movement pat- tern. An interesting observation involves the spontaneous switching of initiative between the robot and the subjects. Another interesting observation was that spontaneous transitions between the synchronized phase and the desynchronized phase tended to occur more frequently in the middle of each session.188 188 Emergent Minds: Findings from Robotics Experiments execute their desired movement patterns regardless of the robot’s move- ments.4. the subjects reported that when they felt that the robot movement pattern became close to theirs. when the sub- ject was already familiar with the robot’s responses to some degree. which can emerge only during a specific period characterized by an adequate balance between predictability . they often initiated new movement patterns. this synchronization could break down after a while due to various uncertainties in mutual interactions. 80 RYH 0. .40 0.00 PBN1 0.20 0.80 RYH RZH Position 0.60 RZH 0. Movement matching by synchronization between the human subject and the robot took place momentarily.40 R: right 0. as can be seen from the sections denoted as Pattern 1 and Pattern 2 in the plot.00 LYH Actual Human Hand LZH Position 0.60 L: left 0.60 PBN3 PB PBN4 0.  A snapshot of parameter values obtained during the imitation game.   189 Pattern1 Pattern2 1.00 20 40 60 80 100 120 140 160 180 200 Step Figure 8.40 L: left R: right 0.20 Y: Y-axis Z: Z-axis 0.00 LZH 0.80 PBN2 0.00 Z: Z-axis 20 40 60 80 100 120 140 160 180 200 H: hand LYH Predicted Human Hand 1.20 Y: Y-axis 0.00 20 40 60 80 100 120 140 160 180 200 H: hand 1.5. In this context. synap- tic connections between the two circuits can be reinforced by Hebbian . Tettamanti et al. 2004. but also provides for rich functions of spontaneously generating novel patterns from learned ones through dynamic interac- tions with others. recent reports have shown that understanding words or sentences related to actions may require the presence of specific motor circuits responsible for gen- erating those actions. linguistic competence has been regarded as independent from other competen- cies. language processing and action processing have been treated as independent areas of research simply because of the different areas of expertise necessary for conducting studies in each of those areas. we may say that imitation for human beings is a means for developing diverse creative images and actions through communicative interaction.2. and Fitch [2002] in section 2. Turn taking was observed more frequently during this period. These results imply that vivid commu- nicative exchanges between individuals can appear by utilizing and anticipating such criticality. 8. Chomsky. According to Chomskian ideas in conventional linguistics. 2005). rather than simply for mimicking action patterns as demonstrated by others “like me. and therefore the parts of the brain responsible for language and actions might be interdependent (Hauk et al. including the aforementioned studies examining the interdependence between linguistic and other modali- ties. including sensory–​motor processing (see the argument on the fac- ulty of language in narrow sense by Hauser. However...” The next subsection explores how mirror neurons may function in developing actional concepts through the association of language with action learning. however.1).  Binding Language and Action In conventional neuroscience. This view. is now being challenged by recent evidence from neuroscience. as mentioned in section 4.190 190 Emergent Minds: Findings from Robotics Experiments and unpredictability in the course of the subjects’ developmental learning in the mutual imitation game. The current experimental results of the imitation game suggest that imitation provides not only a simple function of storing and regenerating observed patterns.4. If everyday experiences involving speech and its corresponding sensory-​motor signals tend to overlap during child development. is assumed to play the role of unifying the two different modalities by means of mir- roring recognition in one modality and generation in the other modality by sharing the intention. in the course of associative learning of pairs of linguistic and behavioral sequences. wherein it is argued that linguistic competency can be acquired through statistical learning of linguistic and sensory–​motor stimuli dur- ing child development. The key idea of the model is that the PB activation vectors in both modules are bound to become identical for generating pairs of corresponding linguistic and behavioral sequences via learning. 2009). without the need to assume innate mechanisms such as Chomsky’s universal grammar.  191 Mirroring Action Generation and Recognition 191 learning.1 Model In this context. Analogous to these ideas is the view of Arbib (2012) discussed earlier. This concept is based on a predictive coding model for linguistic competence assumed in the extension of Wernicke’s area to Broca’s area and another predic- tive coding model for the action competency assumed in the extension from Broca’s area and the parietal cortex to the motor cortex. that the evolution from dex- terous manual behaviors learned by imitation to the anticipated imi- tation of conventionalized gestures (protolanguage) is reflected in the evolution within the primate line and resulted in humans endowed with “language-​ready” brains. The model consists of a linguistic RNNPB and a behavioral RNNPB that are interconnected through PB units. This suggests the pos- sibility that the meanings of words and sentences as well as associated abstract concepts can be acquired in association with related sensory–​ motor experiences. 2005)  for investigating the task of recognizing a given set of action-​related imperative sentences (word sequences) and of also generating the corresponding behaviors (sensory-​motor sequences) is shown in Figure 8. The version of the RNNPB model proposed by Yuuya Sugita and me  (Sugita & Tani. the PB activa- tion vectors in both modules are updated in the direction of minimizing . More specifically. 8. Broca’s area. we consider the possibly interdependent nature of lan- guage and motor action in terms of a mirror neuron model. as a hub connecting these two distinct pathways. Researchers working in the area of cognitive lin- guistics have proposed the so-​called usage-​based approach (Tomasello. as discussed by Pulvermuller (2005).6.4. (a) Bound learning of word sequences and corresponding sensory-​motor sequences through shared PB activation and (b) recognition of word sequences in the linguistic recurrent neural network with parametric biases (RNNPB) and generation of corresponding sensory–​motor sequences in the behavioral RNNPB.  RNNPB model extended for language-​behavior bound learning. word sequences shown to the linguistic RNNPB can be recognized by inferring the PB . By using the error signal back-​propagated from both modules to the shared PB units. st+1 Error Error wT+1 mt+1 st+1 PBI PBb wT ct mt st ct PB linguistic module Shared behavior module Learning phase (b) given word sequence wT+1 sensory motor generation Error wT+1 mt+1 st+1 PBI PBb wT ct mt st ct PB linguistic module Transfer behavior module Recognition and generation phase Figure 8. After convergence of the bound learning. their differences as well as minimizing the prediction error in both modalities (Figure 8. a sort of unified representation between the two modalities is formed through self-​organization in the PB activations. Redrawn from Tani et al. (2004).6a).6.192 192 Emergent Minds: Findings from Robotics Experiments (a) teaching word sequence teaching sensory motor target: wT+1 target: mt+1. 1980) in human language development.” Adopted from Tani et al.6b). We also addressed the issue of generalization in the process of learning linguistic concepts.  193 Mirroring Action Generation and Recognition 193 activation values by means of error regression. 2005)  conducted robotics experi- ments on this model by utilizing a quasilanguage with the aim of gain- ing insights into how humans acquire compositional knowledge about action-​related concepts through close interactions between linguistic inputs and related sensory–​motor experiences. 8. blue. and green objects were always located to the left. and to the right of the robot.7). (a) red. . Thereafter. A physical mobile robot equipped with vision and a one-​DOF arm was placed in a workspace in which red. (2004) with permission. blue and green objects (b) blue red green mobile robot with vision and 1-D hand at home position Figure 8.  Robot experiment setup for language-​behavior bound learning. (b) A trained behavior trajectory of the command “hit red.4.2  Robot Experiments Yuuya Sugita and I  (Sugita & Tani.7. respec- tively (Figure 8. which concerns the inference of the meanings of as yet unknown combi- nations of word sequences through a generalization capability related to the “poverty of stimulus” problem (Chomsky. the forward dynamics of the behavioral RNNPB activated with the obtained PB acti- vation values generate a prediction of the corresponding sensory-​motor sequences (Figure 8. in front. (a) The task environment with the mobile robot in the home position and three objects in front of the robot. it can be seen that PB points corresponding to sentences with the same verbs fol- lowed by synonymous nouns appeared close to each other on the two-​ dimensional map. blue. These PB points were obtained as a result of the recognition of cor- responding sentences. right. as are “blue” and “center” and as are “green” and “right. First. including the four untrained ones.194 194 Emergent Minds: Findings from Robotics Experiments A set of sentences consisting of three verbs (point. push. “push red” means that the robot is to move to the red object and push it with its body. Note that “red” and “left” are synonymous in the setting of this workspace. 8. To examine the internal structures emerging as a result of self-​organization in the bound learning process. For exam- ple. and motor outputs for the two wheels and the one-​DOF arm. The PB vector points for the four untrained word sequences are surrounded by dashed circles in the figure.” For given combinations of verbs and nouns. values for motor torques of the arm and wheel motors. green) were considered. an analysis of the PB mapping was conducted by taking two-​d imensional principal components in the original six-​d imensional PB space.3  Compositionality and Generalization Recognition and generation tests were conducted after convergence in learning was attained by minimizing the error. corresponding actions in terms of sensory-​motor sequences composed of more than 100 steps are trained by guiding the robot while introducing slight variations in the positions of the three objects with each trial. red. especially in the case of linguistic training. center. hit). Corresponding behav- iors were successfully generated for all 18 sentences. To investigate the generalization capabil- ities of the robot. and “hit left” means that the robot is to move to the object to its left and hit it with its arm (Figure 8.8 shows the PB vector points corresponding to all 18 sentences as plotted in a two-​d imensional space. Figure 8. This means that behavioral categories corresponding to the four untrained sentences were learned without being bound with sentences. The sensory-​motor sequences consist of sensory inputs in the form of several visual feature vectors.4. For example. only 14 out of the 18 possible sentences were trained. six nouns (left. “hit left” and “hit red” appeared close to each other in the space. Even more interesting is that the PB map- pings for all 18 sentences appeared in the form of a two-​d imensional grid structure with one dimension for verbs and another for nouns.7b). . it should be noted that even the untrained sentences (“push red/left” and “point green/right”) were mapped to appropriate points on the grid (see the points surrounded by dotted circles in Figure 8. as inferred from the successful generation of corre- sponding behaviors.2 0.  Mapping from PB vector points to generated word sequences. push left. Four PB points surrounded by dotted circles correspond to untrained sentences (push red. It is postulated that mutual interactions between 0. and point right. This explains why untrained sentences were rec- ognized correctly. . point green. Furthermore.  195 Mirroring Action Generation and Recognition 195 This means that the PB mapping emerged through self-​organization of an adequate metric space.8.2 0.) Redrawn from Sugita and Tani (2005).8 PB (2nd PC) re d int po blu no e un hit gr b ee pus h ver n 0. which can be used for compositional repre- sentation of acquired meanings in terms of combinations of verbs and object nouns.8). Such generalization cannot be expected to arise if each meaning or concept is stored in a separate local module as is the case in localist models. The two-​d imensional grid structure consists of an axis for verbs and another for nouns.8 PB (1st PC) Point red Push red Hit red Point left Push left Hit left Point blue Push blue Hit blue Point center Push center Hit center Point green Push green Hit green Point right Push right Hit right Figure 8. These results imply that meanings are acquired through generaliza- tion when a set of meanings is represented as a distribution of neural activity while preserving the mutual relationships between meanings in a binding metric space. 5. The hallmark of these robotics experiments exists in their attempt to explain how generalization in learning as well as creativity for gen- erating diversity in behavioral patterns can be achieved through self-​ organizing distributed memory structures. 8. and generating actional concepts via associative learning of proto-​language and behavior. learning . each behavioral schema is memorized as an independent template in a corresponding local module. This result suggests that compositionality explicitly perceived in the linguistic input channel can enhance the development of compositionality in the actional channel via shared neural activity. which plays an essential role in modeling mirror neural functions in both the generation and recognition of movement patterns by forming adequate dynamic structures internally through self-​organization. On the localist scheme. we found that nine different clusters corresponding to different actional categories were developed without showing any structural relations among them.4). perhaps. The model is characterized by the PB vector. The contrast between the proposed distributed representation scheme and the localist scheme in this context is clear. whereas on the distributed representation scheme. I would like to add one more remark concerning the role of language in developing compositional conceptual space. When the afore- mentioned experiments were conducted without binding the linguistic inputs in learning the same set of action categories. Finally. Summary We’ve now covered RNNPB models that can learn multiple behavioral schemes in the form of structures represented as distributions in a sin- gle RNN. such as is illustrated by the aforementioned two-​d imensional grid structure in the PB space. This idea is analogous to what the PDP group (1986) argued in their connection- ist book more than two decades ago (see section 5. within the Broca’s area of the human brain.196 196 Emergent Minds: Findings from Robotics Experiments different concepts during learning processes can eventually induce the consolidation of generalized structures in the memory structure as rep- resented earlier in the form of a two-​d imensional distribution. again. The model was evalu- ated through a set of robotics experiments involving the learning of multiple movement patterns. the imitation learning of others’ move- ment patterns. In offline learn- ing. As observed in the imitative game experiments. each instance of experi- ence is acquired. Complexity arises from the intrinsic characteristics of mutual interactions occurring in the process. This observation might account for a fascinating mechanism of human cognition by way of which we humans can develop images or knowledge through multiple stages from our own limited experiences: In the first stage.. Zhong. If there are tractable relationships between learned patterns in a set. nontrivial dynamics emerge in the close interactions between top-​down prediction and bottom-​up recog- nition. In real-​time action generation/​recognition. whereby recognition of the actions of others in the immediate past has a profound effect on the actions generated by the robot in the current step. but also deeply consolidated them. resulting in the emergence of novel or “creative” images. The aforementioned characteristics were demonstrated clearly in the analysis of the PB mapping obtained in the results of learning a set of movement patterns and of learning bound linguistic and behavioral patterns. which in turn . 2009. Such characteristics of distributed representation in RNNPB model has been investigated by others (Ogata et al. generalized images or concepts are developed by extracting relational structures among the acquired instances. iterations of top-​down and bottom-​up interactions enable long-​ term structural developments of the internal structures for PB mapping in terms of memory consolidation. in the third stage. as mentioned previously. The RNNPB model learned a set of experienced patterns not just as they were.. 2014) as well. even novel or creative ones can be found in the memory developed with the relational structures after long period of consolidation. in the second stage. Interactions between these two processes take place in offline learning processes as well as during real-​time action generation/​recognition. leading to segmentation of the continuous perceptual flow into meaningful chunks. shifts of the PB vector by means of error regression enable rapid adaptation to situational changes. Another interesting characteristic feature of the model is that it accounts for both top-​ down generation and bottom-​ up recognition processes by utilizing the same acquired generative model. et al.  197 Mirroring Action Generation and Recognition 197 is considered to include not just memorizing each template of behav- ioral patterns but also reconstructing them by extracting the structural relationships between the templates. 2006. Ogata et al.. these relationships should appear in the corresponding memory structures as embedded in a particular met- ric space. I assume there might be some concerns about the scalabil- ity of the RNNPB model. or compose them vice versa. thereby forming a circular causality over the continuum of time between protention and retention. or more specifically whether there are any limits to the degree of complexity that the learned behavioral pat- terns can have. . I just mention that this scalability issue depends heavily on how functional hierarchies can be developed that can decompose complex patterns into sets of simpler ones. Finally.198 198 Emergent Minds: Findings from Robotics Experiments affects the recognition of these perceptual inputs in the immediate future. How can motor acts demonstrated by others be imitated by reading their intentions or mental states? It was shown that imitating others by inferring their mental states can be achieved by segmenting the resultant perceptual flow by regressing the PB states with its prediction error. the next two and the final chapter of this book are entirely dedicated to the investigation of this problem. Here. The same error regression mechanism can give an account for the problem of imitation. Accordingly. This prediction error may result in the subject becoming conscious while recognizing the shift of mental states of oth- ers as they alternate their motor acts. in the network. it is generally accepted that some form of functional hierarchy exists. It also described how action generation proceeds from the sequencing and planning of action primi- tives in the supplementary motor area and prefrontal cortex (PFC) to motor pattern generation in the primary motor cortex (M1). c­ hapter 4 illustrated how visual recognition proceeds in the brain from early signal processing in the primary vision area to object recognition in the inferior temporal area. As an example.  199 9 Development of Functional Hierarchy for Action It is generally held that the brain makes use of hierarchical organization for both recognizing sensory inputs and generating motor outputs. Also. 199 . composition and decomposition of whole complex action routines from and into reusable parts. this functional hierar- chy is thought to be indispensable for expressing the essential human cognitive competency of compositionality. Although we don’t yet completely understand what hierarchy and what levels exist in the brain and how they actually function. in other words. whereby sensory–​motor processing is conducted at the lower level and more global controls of those processes occur at the higher level. and sequential switching of the primitives is achieved by a winner-​t ake-​a ll-​t ype gate-​ opening control of these RNNs performed by the higher level RNN (see Figure 9. Demiris and Hayes (2002) showed a similar idea in the proposal of Hierarchical Attentive Multiple Models for Execution and Recognition.  Hierarchical generation of perceptual sequence patterns in the hierarchical mixture of RNNs. For example. sequential combinations of primitive patterns can be generated. readers should note that there are various ways to achieve such compositions. .200 200 Emergent Minds: Findings from Robotics Experiments In speculating about possible neuronal mechanisms for a functional hierarchy that allows for complex actions to be composed by sequen- tially combining behavior primitives (a set of commonly used behavior patterns).1. 1999) proposed a localist model. As the higher level RNN dispatches the lower level RNNs sequentially by manipulating the openings of their attached gates. Tani and Nolfi (1997. Information processing at the higher level is abstracted in such a way that the higher level only remembers which RNN in the lower level should be selected next as well as the timing of switching over a longer timescale. without concerning itself with details about the WTA-type gate opening control GateT+1 Higher Gate opening Gate-2 Gate-1 Gate-3 T Gate cT T Time Steps pt+1 p p t+1 t+1 Perceptual Prediction Gate-1 Gate-2 Gate-3 Pattern2 Pattern1 Pattern3 Lower t Time Steps pt ct pt ct pt ct Figure 9. One possibility is to use a localist representation scheme.1). and also Haruno and colleagues (2003) did in the proposal of the so-​called hierarchical MOSAIC. called a “hierarchical mixture” of RNNs. The basic idea was that each behavior primitive is stored in its own independent local RNN at the lower level. Rather. if a specific PB vector value is assigned to each acquired behavior primitive. Although the proposed scheme seems to be straightforward in terms of mechanizing a functional hier- archy. sequential changes in the PB vector generated at the higher level by another RNN can cause corresponding sequential changes in the primitives at the lower level (Figure 9.2. the scheme is faced with the problem of miscategorization in dealing with perturbed patterns. The higher level RNN learns to predict event sequences in terms of stepwise changes in the PB vector.2). 2003). the discrete mechanism of dispatching behavior primitives through the winner-​take-​all-​t ype selec- tion of the lower RNNs tends to generate a certain level of information mismatch between the higher and lower levels. Redrawn from Tani (2003). As I previously proposed (Tani. this scheme could also suffer from a similar problem PBT+1 Higher PB2 PB PB1 PBT cT Time Steps T pt+1 Lower Perceptual Prediction Pattern1 Pattern2 Pattern3 PBt pt ct Time Steps t Figure 9. as well as the timings of such events. Another possible mechanism can be considered by utilizing a distrib- uted representation scheme in an extension of the RNNPB model. wherein sequential stepwise changes in the PB vector at the higher level generate corresponding changes in the primitive patterns at the lower level.  201 Development of Functional Hierarchy for Action 201 sensory–​motor profiles themselves.  Possible extension of the RNNPB model with hierarchy. However. . Instead. . depend- ing on their combination. such fine adaptation cannot take place by simply changing the components of the PB vectors in a step- wise manner within the time necessary for the primitive to change. the task requires fluid transitions between primitives by adapting them via interactions between top-​down parametric control exerted on the primitives and bottom-​up modulation of signals implementing such parametric control. The same problem is encountered in the case of gated local network models if primitives are changed by simply opening and closing the corresponding gates. The model is tested in a task involving learning object manipula- tion and developing this learning.202 202 Emergent Minds: Findings from Robotics Experiments of information mismatch between the two levels. Rather. However. A smooth connection often requires some degree of specific adaptation of profiles at the tail of the preced- ing primitive and at the head of the subsequent primitive. The crucial point here is that the generation of compositional actions cannot be achieved by simply transforming primitives into sequences in the same manner as manipulating discrete objects. it can emerge by using intrinsic constraints on timescale differences in neu- ral activity between multiple levels in the course of self-​organization. we see how the functional hierarchy that enables compositional action generation can be developed through the use of a novel RNN model characterized by its multiple timescales dynam- ics. Close interactions could minimize the possible mismatch between the two sides. whereby we might witness what Alexander Luria (1973) metaphorically referred to as “kinetic melody” in the fluid generation of actions. a smooth connection between the two primitives cannot be guaranteed. The following sections show that such fluid compositionality can be achieved without using preexist- ing mechanisms such as gating and parametric biases. If one behav- ior primitive is concatenated to another by corresponding stepwise changes in the PB vector. In the following. The discussion helps to explain that how fluid compositionality can be developed in both human and artifact through specific constraints within their brain networks. accompanied by iterative interactions between the levels in consolida- tion learning. We then discuss a possible analogy between the synthetic developmental processes observed and real human infant developmental processes. named the multiple-​ timescale recurrent neural network (MTRNN). 17 in section 5. The MTRNN consists of interconnected subnetworks to which dynamics with different timescales are assigned. Action A and Action B as triggered by the corresponding intention in terms of initial states of Init A and Init B in the intention units. The model shown in Figure 9. Each subnetwork takes the form of a fully connected continuous-​time recurrent neural network (CTRNN) with a specific time constant τ assigned for the purposes of neural activation dynamics.3.  Self-​Organization of Functional Hierarchy in Multiple Timescales 9. The left panel shows the model architecture and the right panel the information flow in the case of top-​down generation of different compositional actions.1  Multiple-​Timescale Recurrent Neural Network My colleague Yuuichi Yamashita and I  (Yamashita & Tani. Intention state Top-down Generation Update Set Slow Action Plans Slow Init A Init B Bottom-up error regression Intermediate Top-down prediction Intermediate Pool for Primitives Fast Fast Compositional Vision Generations Proprio module module Action A Pt+1 Action B Pt Vt Vt+1 Error Error Pt+1 Vt+1 motort+1 Figure 9.3 is composed of subnetworks with slow. 2008)  pro- posed a dynamic neural network model characterized by the dynamics of its neural activity in multiple timescales. is outlined in Figure 9.3.5.  203 Development of Functional Hierarchy for Action 203 9. . as can be seen in Eq.1.  The multiple-​t imescale recurrent neural network (MTRNN) model. respectively.1. This model. with the exception of some neural units in the subnetwork with slow dynam- ics referred to as intention units. and thus the initial states of the intention units play the role of selecting sequences. In the course of error back-​ propagation learning. The for- ward top-​down dynamics initiated with this temporarily set initial state generates a predictive sequence for the training visuo-​proprioceptive . The difference is that selection in the case of PB is based on parametric bifurcation. We decided to employ a switching scheme based on sensitivity to the initial conditions for the MTRNN because this feature affords learning of sequence patterns with a long time correlation. corresponding learned perceptual sequences of intended can be regenerated. we made use of the sensitiv- ity of the dynamics toward initial conditions seen in nonlinear dynamics (see section 5. the initial state of the intention units for each training sequence is set to a small random value.204 204 Emergent Minds: Findings from Robotics Experiments intermediate. and smaller values of τ. We designed this particular model to generate targets of multiple per- ceptual sequences that contain a set of primitives or chunks acquired as a result of supervised learning. When learning is commenced. Our expectation in the proposed multiple timescales architecture was that the slow dynamic subnet using large time constant leaky-​integrator units should be good at learning long-​time correlation. By providing specific initial states for these intention units. Adequate mappings between the respective initial states of the inten- tion units and the corresponding perceptual sequences are acquired by means of the error back-​propagation through time learning scheme applied for CTRNN (Eq. medium. and fast dynamics characterized by the leaky-​integrator neural units with larger. respectively. as indicated by Jaeger and colleagues (2007).1) as a mechanism for selecting a specific sequence from among multiple learned ones as intended one. whereas the fast dynamics one should be good at learning precise short-​ranged patterns. Additionally. while in the case of intention units in MTRNNs this is performed by utilizing the sensitivity of the network dynamics to the initial conditions.5). two classes of variables are determined. The network dynamic always starts with the same neutral neural states for all units. 18 in section 5. namely the connection weights in all subnetworks and a specific set of initial state values for the intention units for each perceptual sequence to be learned. the subnetwork with fast dynamics is subdivided into two peripheral modular subnetworks for proprioception/​motor operations and for vision. similar to the role of PB vec- tors in RNNPB models. In this case. This forces the learning process to extract the underlying correlations spanning lon- ger periods of time in the training sequences in the parts with slower dynamics and correlations spanning relatively shorter periods of time in the parts with faster dynamics in the whole network. The error generated between the training sequence and the output sequence is back-​propagated along the bottom-​up path through the subnetworks with fast and intermediate dynamics to the subnetwork with slow dynamics. action plans are selected according to intention and are passed down to the interme- diate dynamics subnetwork for fluid composition of assembled primi- tives in the fast dynamics subnetwork.3 illustrates how learning multiple per- ceptual sequences consisting of a set of primitives results in the devel- opment of the corresponding functional hierarchy. The right panel of Figure 9. It is noted that change in the slow dynamic activity plays a role of parameter bifurcation for the intermedi- ate and fast dynamics to generate transitions of primitives. One point to keep in mind here is that the dampening of the error signal in backward propagation though time steps depends on the time constant as described previously (see Eq. . and this back-​propagation is iterated backward through time steps via recurrent connections. The error signal is also back-​ propagated through time steps to the initial state of the intention units. 18 in section 5. in which the initial state values for each training sequence are modified. whereby the connection weights within and between these subnetworks are modified in the direction of minimizing the error signal. a set of trajectories corresponding to slower neural activation dynamics should appear in the subnetwork with slow dynam- ics in accordance with the initial state. just as the RNNPB does. of which activ- ity is sensitive to the initial conditions. In the slow dynamics subnetwork. Here. induces specific sequences of primitive transitions by interacting reciprocally with the intermediate dynamics subnetwork. This subnetwork.5). we see again that learning proceeds through dense interactions between top-​down regeneration of the training sequences and bottom-​ up regression of the regenerated sequences utilizing error signals. It becomes smaller within the subnetwork with slow dynamics (characterized by a larger time constant) and greater within the subnetwork with fast dynamics (characterized by a smaller time constant). First. it is assumed that a set of primitive patterns or chunks should be acquired in the subnetworks with fast and intermediate dynamics through distributed representation. Next.  205 Development of Functional Hierarchy for Action 205 sequence. Considering this. our robots with MTRNN can become self-​n arrative about own possibility. MTRNNs can perform both offline and online recognition of per- ceptual sequences by means of error regression. For example. First.206 206 Emergent Minds: Findings from Robotics Experiments As another function. anal- ogous to the closed-​loop forward dynamics of the RNNPB. as described later in this chapter. Kiebel and colleagues (2008). These functions have been evaluated in a set of robotics experiments utilizing this model. let’s revisit our previous discussions and examine briefly the corre- spondence of the proposed MTRNN model to concepts in system-​level neuroscience. prediction errors caused by unexpected visual sensory input due to certain changes in the environment are back-​propagated from the visual module of the fast dynamics subnet- work through the one with intermediate dynamics to the intention units in the slow dynamics subnetwork. whereby the modulation of the activity of the intention units in the direction of minimizing the errors results in the adaptation of the currently intended action to match the changed environment.2  Correspondence with Neuroscience Now. as discussed in ­chapter 4. there is a timescale difference in the buildup of neu- ral activation dynamics between the supplementary motor area (with slower dynamics spanning timescales of the order of seconds) and M1 (with faster dynamics of the order of a fraction of a second) immediately before action generation (see Figure 4. as shown by Tanji and Shima (1994).5). Because the neuronal mechanisms for action generation and recognition are still puzzling due to clear conflicts between differ- ent experimental results. as in the case of the RNNPB model. Badre and D’Esposito (2009). the correspondence between the MTRNN model and parts of the biological brain can be investigated only in terms of plausibility at best.1. Additionally. and Uddén and Bahlmann (2012) proposed a similar idea to explain the rostral–​caudal gradient of times- cale differences by assuming slower dynamics at the rostral side (PFC) . By this means. and therefore our assumption that the organization of a functional hierarchy involves timescale differ- ences between regional neural activation dynamics should make sense in modeling the biological brain. Diverse motor imagery can be generated by manipulating the initial state of the intention units. MTRNNs can generate motor imagery by feeding predicted visuo-​proprioceptive states into future inputs. 9. as described later. assuming of course that the aforementioned retro- grade axonal signaling mechanism of brains implements the error Error PFC/SMA (Slow) Motor (Fast) Parietal (Medium) Intention Error Vision (Fast) Figure 9. Parietal to Motor and Vision). by means of neural activations with fast dynamics. The subnetwork with moderate dynamics may correspond to the parietal cortex. Accordingly. respectively. whereby detailed predictions of visual sensory input and propriocep- tion are made.4). and the dotted line represents the bottom-​up error regression pathway (from Vision. Activations in the parietal cortex propagate further into peripheral cor- tices (the early visual cortex and the premotor or primary motor cortex).  Possible correspondence of MTRNN to parts of the biological brain. prediction errors generated in those periph- eral areas are propagated backward to the forebrain areas through the parietal cortex via bottom-​up error regression in both learning and recognition. the MTRNN model assumes that the subnetwork with slow dynamics corresponds to the PFC and/​or the supplementary motor area. which subsequently propagates to the parietal cortex assumed to exhibit moderate-​timescale dynamics. which can interact with both the frontal part and the peripheral part.  207 Development of Functional Hierarchy for Action 207 and faster dynamics at the caudal side (M1) in the frontal cortex to account for a possible functional hierarchy in the region. . The solid line represents the top-​down prediction pathway (from PFC/​SMA. Parietal to PFC/​SMA). On the other hand. and that the modular subnetwork with fast dynamics corresponds to the early visual cortex in one stream and to the premotor cortex or M1 in another stream (Figure 9. One possible scenario for the top-​down pathway is that the PFC sets the initial state of activations with slow dynamics assumed in the supplementary motor cortex.4. such as vision and motor outputs (Sakata et al. It is worth pausing here a moment to think about what the initial states actually mean in the brain. . They conducted simultaneous recordings of multiple neurons in the motor and premotor cortices while monkeys repeatedly reached in varying directions and at various distances. in the same way Churchland used before (see Figure 4. it can be said that motor programs can be represented by the initial states of particular neural dynamics in the brain. after movement onset. In this situation. A nontrivial finding was that. The collective activities of neuron firings were plotted into two-​d imensional state space from their principal components. Because the initial states unfold into sequences of behavior primitives. It has been speculated that populations of bimodal neurons in the parietal cortex. Churchland and colleagues published new results from monkey electrophysiological experiments that support this idea (Churchland et al. 2012). as I  was writing this section. Their interpretation is quite analogous to the idea Yamashita and I proposed: Motor programs might be represented in terms of the initial states of particular neural dynamical systems. are the consequence of synaptic modulation accompanied by top-​down prediction and bottom-​up error regression in the iterative learning of behavioral skills. The next section describes a robotics experiment pursuing this line of reasoning utilizing the MTRNN model..5. Churchland and colleagues interpreted this as follows:  The preparatory activity sets the initial state of the dynamic system for generating quasirotational trajectories and their subsequent evolution produces the corresponding movement activity.208 208 Emergent Minds: Findings from Robotics Experiments back-​propagation scheme (see section 5. 1995) or vision and somatosensory inputs (Hyvarinen & Poranen..). which are expanded into target pro- prioceptive sequences and finally into motor command sequences. 1974). The differences in the develop- ment of the neural activation state were due to differences in their ini- tial state at the moment of movement onset. Coincidentally. the neural acti- vation state exhibited a quasirotational movement in the same direction but with different phase and amplitude in the two-​d imensional state space for each different case of reaching. the pari- etal cortex wedged between the frontal and peripheral parts plays the role of an information hub that integrates multiple input modalities and motor outputs with the current intention for action. which have been shown to encode multiple modalities of information processing.12).   A robot trained on three behavioral tasks.  Robotics Experiments on Developmental Training of Complex Actions This section shows how the MTRNN model can be used in humanoid robot experiment tasks on learning and generating skilled action. 2008. Task 3 was modified. each of which Move up and down Move left and right Task 1 Home position Move forward and back Touch by each hand Back to home Task 2 Touch by both hands Rotate in the air Task 3 Figure 9. Therefore. The robot was trained on three different tasks in sequence (shown in Figure 9.  209 Development of Functional Hierarchy for Action 209 9. Adopted from Nishimoto and Tani (2009) with permission. 9.5. 2009). The robot used a vision camera that could automatically track a color point placed in the center of the object. A small humanoid robot QRIO was trained on a set of object manipulation tasks in parallel through iterative guidance provided by a teacher. . as illustrated by the dotted lines. The robot could move its arms by activating joint motors with eight degrees of freedom (DOF) and was also capable of arm proprioception by means of encoder readings for these joints.2.1 Experimental Setup I conducted the following studies to investigate how a humanoid robot can acquire skills for performing complex actions by organizing a func- tional hierarchy in the MTRNN through interactive tutoring processes (Yamashita & Tani. reading the joint angles of the camera head (two DOF) represents visual sensory input corresponding to the object position. each of which is composed of a sequence of behavior primitives. After the third session.5).2. Nishimoto & Tani. The units with slow and inter- mediate dynamics were fully interconnected. the performance of both the open-​loop physical behavior and the closed-​ loop motor imagery was tested for all three tasks.0). which was modified with the addition of a novel primitive pattern after . After each training session. whereas the units with slow and fast dynamics were not connected directly.210 210 Emergent Minds: Findings from Robotics Experiments consisted of sequential combinations of different cyclic movement pat- terns of actions applied to the object.5).0). During each session. 9. as were all the units with fast and moderate dynamics. In the course of developmental learning. The training was conducted interactively in cycles of training ses- sions.2 Results The developmental learning of multiple goal-​d irected actions success- fully converged after five training sessions. the robot was trained gradu- ally in the course of five sessions. The employed MTRNN model consisted of 36 units with fast dynamics for vision and 144 units with fast dynamics for propriocep- tion (τ  =  1. meaning that the arms were physically guided to follow adequate trajectories while the robot attempted to generate its own trajectories based on its previously acquired skills. in which the connection weights and the initial states of the intention units for all task sequences were updated. Through this physical guidance. it can be said that the actual training trajectories were “codeveloped” by the teacher and the robot. and the network was trained with all training data obtained during the ses- sion.0). even in the case of Task 3. Novel movement pat- terns were added to one of the tasks during the development process for the purpose of examining the capability of the network for incremental learning of new behavioral patterns (see Task 3 in Figure 9. In this sense. 30 units with intermediate dynamics (τ  =  5. Subsequently. the robot eventually per- ceived a continuous visuo-​proprioceptive (VP) flow without explicit cues for segmenting the flow into primitives of movement patterns.2. It was assumed that this kind of connection constraint would allow functional phenomena such as infor- mation bottlenecks or hubs to be developed in the subnetwork with intermediate dynamics. offline training of the MTRNN was conducted by utilizing the VP sequences obtained in the process of guidance. and 20 units with slow dynamics (τ = 70. all three tasks were repeated while introducing changes in the object position. and the time evolution of the activations of the units with slow dynamics was almost flat. this developmental course of the robot’s learn- ing supports the view of Smith and Thelen (2003) that development is better understood as the emergent product of many local interactions that occur in real time. In the third stage. showing some generalization with respect to changes in object position. In the first stage. all tasks were successfully generated with correct sequencing of the primitive movement patterns and with good generalization with respect to changes in object position. and Figure 9. such as Karmiloff-​Smith (1992) and Diamond (1991). motor imagery (middle). In sum- mary then. which mostly corresponds to Session 1. although they cannot reach or grip them properly due to the immaturity of their motor control skills. none of the tasks were accomplished. and the level respon- sible for the organization of these patterns into sequences developed in later periods. who consider that 2-​month-​old infants already possess intentionality toward objects they wish to manipulate. the level responsible for organization of primitive movement patterns was developed during the earlier period.  211 Development of Functional Hierarchy for Action 211 the third session. One important point I  want to make here is that there was a lag between the time when the robot became able to generate motor imagery and the time when it started generating actual behaviors. as compared to Session 3 in the case of actual generated behaviors. Plots are shown for the trained VP trajectories (left). The developmental process can be categorized into sev- eral stages. corresponding to Session 3 and subsequent sessions. Moreover. This outcome is in accordance with the arguments of some contempo- rary developmental psychologists. Motor imag- ery was generated earlier than the actual behavior. In the second stage. . as most of the actually generated movement patterns were premature. The activations of units with slow dynamics became more dynamic compared with previous sessions in the case of both motor imagery and generation of physical actions. and actual output generated by the robot (right). although correct sequencing of them was not yet complete.6 shows the process for Task 1 for the first three sessions. The profiles for the units with slow dynamics in the motor imagery and the actual generated behavior were plotted for their first four principal com- ponents after conducting principal component analysis (PCA). most of the primitive movement patterns were actually generated. as it was observed that the motor imagery for all tasks was nearly complete by Session 2. cor- responding to Session 2. motor imagery (middle).  Development of Task 1 for the first three sessions with trained VP trajectories (left). and actual generated behavior. after conducting principal component analysis. (c) Session 3. (b) Session 2. .212 Figure 9. accompanied by the profiles of units with slow dynamics. (a) Session 1.6. Adopted from Nishimoto and Tani (2009) with permission. Training patterns such as UD (moving up and down) in the first half and LR (moving left and right) in the second half did not form regular cycles. However. Figure 9. First. it can be said that the robot’s behavior trajectory and my teaching trajectory codeveloped during the experiment. The interac- tion modifies not only the robot’s action but also the teacher’s. Next. This is a typical case when cyclic patterns are taught to robots without using metronome-​like devices. This modified my teaching intention and the resultant trajectory of guidance to some degree. However. which started from different initial states for each of the three tasks. it is clear that their dynamics are correlated with the VP trajectories. let’s see how neurodynamics with different timescales success- fully generates sets of action tasks consisting of multiple movement pat- terns. the profiles changed drastically as the movement patterns changed. However. it can be seen that cyclic patterns in the training process became much more regular as the sessions proceeded. In this sense.7 shows how the robot behaviors were generated in the test run after five training sessions. Looking at the activation dynamics of the units with intermediate dynamics (shown in the fourth row) after conducting PCA. VP trajectories for trained robots were successfully generated for all three tasks accompanied by changes in the cyclic movement patterns. the activation dynamics of the units with slow dynamics. This result shows a typical example of the codevelopment process undertaken by the robot and teacher whereby the robot’s internal struc- tures develop via dense interactions between the top-​down intentional generation for the robot’s movement and the bottom-​up recognition of the teacher’s intention for guiding the robot’s movement. I  became aware of its intention of persistence by some resistance force perceived in my hands.7.  213 Development of Functional Hierarchy for Action 213 Another interesting observation taken from this experiment was that the profiles of the training trajectories also developed across sessions. Also. This is due to the development of limit-​cycle attractors in the MTRNN that shaped the trajectories trained through direct guidance into more regular cyclic ones via physical interactions. as can be seen in the first and second rows in Figure 9. When I tried to guide physically the robot’s arms to move slightly differently from its own movement by grasping the arms. the . It can be seen that the training trajectories in Session 1 were quite dis- torted. developed to be uncorrelated with the VP or the trajectories of the units with intermediate dynamics (see the bottom row). 0 0.0 0. denoted as PC 1–​4.2 0. .2 0.2 0. (b) moving forward and backward (FB) followed by touching by left hand and right hand (TchLR) in Task 2.2 0. 214 (a) (b) (c) UD LR FB TchLR BG RO 1.6 0.6 0.4 0.4 0.4 0.0 1.6 0.0 1.0 0. (c) touching by both hands (BG) followed by rotating in air (RO) in Task 3.8 0.6 0.8 Teach Teach Teach 0. Adopted from Nishimoto & Tani (2009) with permission.4 0.6 0.8 0.4 0.6 0.7.0 1.0 0 50 100 150 200 250 300 0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 2 2 2 Slow units Slow units Slow units 1 1 1 0 0 0 –1 –1 –1 –2 –2 –2 PC1 0 50 100 150 200 250 300 0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 PC2 PC3 Intermediate units Intermediate units 2 Intermediate units 2 2 PC4 1 1 1 0 0 0 –1 –1 –1 –2 –2 –2 0 50 100 150 200 250 300 0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 Step Step Step Figure 9.4 0.2 0.  Visuo-​proprioceptive trajectories (two normalized joint angles denoted as Prop 1 and Prop 2 and the camera direction denoted as Vision 1 and Vision 2) during training and actual generation in session 5 accompanied by activation profiles of intermediate and slow units after principal component analysis.0 1.0 Prop1 0 50 100 150 200 250 300 0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 Prop2 Vision1 1.0 Vision2 Generation Generation Generation 0.8 0. (a) Moving up and down (UD) followed by moving left and right (LR) in Task 1.8 0.2 0.0 0.0 0.8 0. by pulling the robot’s hand slightly. all actions are generated unconsciously because no prediction error is generated in the course of well-​practiced trajectories unless encountered with unex- pected events such as dropping the object. the experimenter. This implies that counting at the higher level is more like an elastic dynamic process rather than a rigid logical computational one. Our proposed dynamic system’s scheme can allow this type of dynamic con- flict resolution between different levels by letting them interact densely. even though the switch was made after incorrect times of cycling. could induce the robot to switch action primitives from moving up and down to moving left and right earlier than four times cycling as it had been trained. smooth profiles. as described in the previ- ous section. An interesting observation was that the action primitive of moving up and down was smoothly connected to the next primitive of moving the object to the left.  215 Development of Functional Hierarchy for Action 215 transitions were still smooth. Further insight was obtained by observing how the robot managed to generate action when perturbed by receiving external inputs. The transitions never took place half way of ongoing primitive and were made at the same connection point as always regardless of incorrect times of cycling at the transition. The bottom-​ up error regression tends to generate rapidly changing profiles at the moment of switching. whereas the lower level was successful at connecting the first primitive to the second one at the same point as having been trained. In the current observation. This observation suggests that the whole system was able to generate action sequences with fluidity and flexibility by adequately arbitrating between the higher level that has been trained to count specific times before switching and the lower level that has been trained to connect one primitive to another at the same point. which took place right after locating the object on the floor. The collaboration and competition between the two processes result in such natural. which could be modulated by external inputs like being pulled by the experimenter. whereas the top-​down forward prediction tends to generate only slowly changing profiles because of its large time con- stant. the intention from the higher level was elastic enough to give in for incorrect times of counting against the bottom-​up force exerted by the experi- menter. which were accompanied by stepwise changes. . In Task 1. unlike in the case of gate opening or PB. Such drastic but smooth changes in the slow context profile were tailored by means of dense interactions between the top-​down forward prediction and the bottom-​up error regression. After enough training. These findings provide a possible explanation for how different functional roles can be assigned in different regions in brains (i. including connectivity among local regions with bottlenecks and timescale differences in neuronal activities. The observed fluid compositionality that has been metaphorically expressed as “kinetic melody” by Luria should be resulted from this. Can the time constant parameters in the MTRNN be adapted via learning. known as downward causation (Campbell. network topology. Summary This chapter was entirely dedicated to an examination of functional hierarchy by exploring the potential of the MTRNN model. 2011) which denotes causal relationship from the global to local parts. 1974. Such assignments in the brain may not be tailor made by a genome program.. It can be said that the functional hierarchy emerges by means of the upward causation in terms of collective neural activity both in the for- ward activation dynamics and the error back-​propagation which are con- strained by the downward causation in terms of timescale difference. whereas detailed patterns of behavior primitives are generated in the subnetworks consisting of units with fast and intermediate dynamics.3. or do they have to be set by the experimenters as in the current version? Hochreiter and Schmidhuber (1997) proposed the “long-​term and short-​term memory” RNN model.e. Readers may ask a crucial question. This can be accounted by a well-​k nown conception in complex adaptive systems. and environmental interaction. Reflective selves of robots may start from this point. Bassett & Gazzaniga. but result as a consequence of self-​organization via develop- ment and learning under various structural constraints imposed on the anatomy of the brain. the PFC for creating abstract actional plans and the parietal cortex for com- posing sensory–​motor details).216 216 Emergent Minds: Findings from Robotics Experiments 9. The exper- imental results suggest that sequences of primitives are abstractly rep- resented in the subnetwork consisting of units with slow dynamics. . We can conclude that a sort of “fluid compositionality” for smooth and flexible generation of actions is achieved through self-​organization of a functional hierarchy by utilizing the timescale differences as well as the structural connectivity among different levels in the proposed MTRNN model. It was also shown that the capability of abstraction through hierarchy in MTRNN can provide robots with competency of self-​narrative for own actional intention via mental simulation. . As the result of end-​to-​end learning for vari- ous combinations of the gesture patterns and the corresponding motor outputs for grasping different shape of objects. pro- vided that a bottleneck connection is prepared between them. 2015. Hasson and colleagues (Hasson et  al. Actually. In response to this concern.  217 Development of Functional Hierarchy for Action 217 which is characterized by its dynamic memory mechanism imple- mented in “memory cells.. Simulation experiments on robot naviga- tion learning using this model showed that a functional hierarchy for navigation control of the robot was developed by evolving slower and faster dynamics structures between two levels of the subnetworks. If the memory cells were allocated in multiple levels of subnetworks. 2015.” A memory cell can keep its current dynamic state for arbitrarily long time steps without specific parameter setting by means of its associated adaptive gate opening–​closing mechanisms learned via the error back-​propagation scheme. 2016) has recently shown that a spatio-​temporal hier- archy can be developed successfully in a neurodynamic model referred to as multiple spatio-​temporal neural network (MSTNN) for the rec- ognition as well as generation of compositional human action sequence patterns. 2005). it would be interesting to examine whether a functional hierarchy can be developed by organizing the long-​term memory in the higher level and the shorter memory in the lower level. MSTNN and MTRNN has been integrated in a simulated humanoid robot platform (Figure 9. we observed that actions could be generated compositionally depending on . 2008)  suggested development of spatio-​ temporal hierarchy in human visual cortex. Some may consider that brains should involve also a spatial hierarchy as having been evidenced in the accumulated studies on the visual rec- ognition pathway (see section 4. Furthermore. it was found that the intentions for grasping different objects can be developed in the PFC subnetwork characterized by its slowest time scale in the whole network. our group (Jung et al. as represented in pixel level video images when both spatial and temporal constraints are applied to neural activation dynamics in multiple scales for different levels. Going back to the robotics experiment using the MTRNN.8) by which the simulated robot becomes able to generate object manipula- tion behaviors corresponding to visually demonstrated human gesture via end-​to-​end learning from the video image inputs to the motor out- puts (Hwang et al. the MTRNN model was originally developed with a time-​constant adaptation mechanism by using a genetic algo- rithm (Paine & Tani.1). Also.. Choi & Tani. as addressed in section 4. 2015).  A simulated humanoid robot learns to generate object manipulation behaviors as specified by human gesture demonstrated to the robot by video image. the initial states of the intention units. this naturally poses the question of how the initial state is set (Park & Tani.8.3. (a) Task space and (b) the integrated model of MSTNN for video image processing and MTRNN for dynamic motor pattern generation. .218 218 Emergent Minds: Findings from Robotics Experiments (b) Higher level (PFC) Very slow In t e re ntio gestu n to MSTNN uman MTRNN man Slow Slow ipul ing h ate s goriz pec Cate ified MSTNN MTRNN Fast obje (a) Fast ct Motor output Dynamic vision Attention control Inout (VI) time Figure 9. However. The next chapter explores this issue by examining the results from several syn- thetic robotics experiments while drawing attention to possible corre- spondences with the experimental results of Libet (1985) and Soon and colleagues (2008). Is there any way that the initial state representing the intentionality for action could be self-​determined and set autonomously rather than being set by the experimenter? This issue is related to the problem of the ori- gin of spontaneity or free will.   219 10 Free Will for Action and Conscious Awareness We first explore how intentions for actions can be generated spon- taneously in higher cognitive brain areas by reviewing our robotics experiments. a result later confirmed by Soon and colleagues (2008). I  only notice later that I actually added sugar when I take the first sip.  A Dynamic Account of Spontaneous Behaviors Although we may not be aware of it. I usually add milk and then either add sugar or not. Then frequently. and this is where I can see 219 . which is rather unconsciously determined.3. something we are all likely to be very familiar with. 10. After I’ve put a spoonful of coffee granules in my mug and have added hot water. Some parts of these action sequences are defined and static―I must add the coffee granules and hot water―but other parts are optional. our everyday life is full of spon- taneity.1. Let’s take the example of the actions involved in making a cup of instant coffee. Later sections investigate this problem by clarifying causal relationships shared by free will and consciousness. Libet (1985) demonstrates that awareness of intention is delayed. As I wrote in section 4. Saffran et al. as I  have mentioned many times already. voluntary actions might originate from neural activities in the supplementary motor area. and C) has been entirely conscious. Baldwin et al. prefrontal cortex.. Klahr et al.g. where musical phrases or body movement patterns are created freely and on the spot in an unpredictable manner. even though one might believe that the choice of a particular action from among multiple possibilities (e. at all. such as pouring hot water into a mug or repeating musical phrases.. “junctions. Actually. the term “chunk structures” denotes repeatable patterns of action sequences as unified “chunks” and takes into account the probabilistic state transi- tions between those chunks. primitives A. so there is at least the apparent potential for freely chosen action in such instances. It seems that spontaneity appears not within a chunk but at junctions between chunks in behavior streams.220 220 Emergent Minds: Findings from Robotics Experiments spontaneity in the generation of my own actions. or parietal cortex.. Thus. and indeed free will seems not so freely determined. provided that past expe- rience defines equal probabilities for A and B. A similar comment can be made about improvisations in playing jazz or in contemporary dance..” One question essential to the problem of free will arises. and in no case are these activities accompanied by awareness. which are presumably acquired through practice and experience. Here. Kirkham et al. psychological observations of child development as well as adult learning have suggested that chunk structures can be extracted through statistical learning with a sufficiently large number of perceptual and behavioral experiences (e. it is plausible that either of the primitives might be enacted.. B.3. Our MTRNN model can account for these results by assuming that neural activities preceding apparently freely chosen actions are represented by the initial states of the intentional units located in the . 1983. Junctions between behavior primitives are weaker relationships than within primitives themselves because junctions appear less frequently than primitives in repeated behavioral experiences. 2008). 2002. How is it that subsequent chunks or behavior primitives can be considered to be freely selected if one simply follows a learned statistical expectation? If we consider someone who has learned that the next behavior primitive to enact in a certain situation is either A or B. However.g. in fact this apparently conscious decision has been precipitated by neural activity not subject to awareness. Chunks are behavior primitives. following the studies by Libet (1985) and Soon and colleagues (2008) discussed in section 4. 1996. and the experimenter repeated primitive actions that consisted of picking up the object. center. The following experimental results highlight the role of cortical itinerant dynamics in generating spontaneity.2 (and in Yamashita and Tani.0 Right to Center (50%) Left to Center (50%) Vertical Center to Right (50%) Center to Left (50%) –1. this explanation generates fur- ther questions:  (1)  how are the values of the initial states set for ini- tiating voluntary actions.0 –1.1 Experiment A humanoid robot was trained to imitate actions involving object manipulation though direct guidance by an experimenter. and (2) how can conscious awareness of the decision emerge with delay? To address these problems. (2011) with PLoS Creative Commons Attribution (CC BY) license.  Object manipulation actions to be imitated by a Sony humanoid robot. The target task to be imitated included stochastic transitions between primitive actions. This process generated 24 training (a) Right to Left (50%) (b) 1. 2008).1. my colleagues and I conducted some neurorobotics experiments involving the statisti- cal learning of imitative actions (Namikawa et al. or right). The target actions to imitate are shown in Figure 10. (b) Trajectory of the center of mass of the object as observed by using the robot’s vision system.  221 Free Will for Action and Conscious Awareness 221 network with slow dynamics. Adopted from Namikawa et al. The object was located on the workbench in one of three positions (left. 2011). and releasing it by guiding the hands of the robot while deciding the next object position randomly with equal probability (50%).1.. However.0 Left to Right (50%) Figure 10.0 Horizontal 1. . The setup used for the robot and the way its movements were guided were the same as in our experiment described section 9.1. 10. moving it to one of the other two possible positions. (a) The task consists of stochastic transitions between primitive actions: moving an object to one of two possible positions with equal probability after reaching and grasping it. as motor imagery is generated . After the offline training of the network.0 for units with slow. Although the trained primitive action sequences were repro- duced exactly during the initial period consisting of several primitive action transitions. could extract the statistical structures (chunks) with their corresponding transition probabilities from these sequences.222 222 Emergent Minds: Findings from Robotics Experiments sequences. The inter- mediate subnetwork controlled the gate opening by the outputs. however. An analysis of the sequences produced by the trained network for each case showed that the transition prob- abilities of the reproduced actions mostly followed the target ones. one might assume that the prevailing opinion is that spon- taneity is simply due to noise in the (external) physical world.0. Statistical analysis conducted on the transition sequences generated over longer periods showed that the probabilities with which the transitions between the primitive actions were reproduced were quite similar to the ones to which the robot was exposed during the training period. amounting to about 2. It is noted that in this experiment the lower level was assembled with a set of gated RNNs (Tani & Nolfi. This turns out to be quite important because. The following experiment. Let’s now examine the main issue in this context. The time constants of the employed MTRNN were set to 100. it turned out that the answer is affirmative.5%. When the transition probabilities for some of the target actions were changed to 25% and 12. the same proportion of corresponding sequences were newly generated in each case. with deviations of only a few percent.500 time steps of continuous visuo-​ proprioceptive sequences. intermediate. respectively. in examining whether the same statistical repro- duction could also be observed in the case of motor imagery rather than actual motor action.0. These results imply that the proposed model. shows that this is not the case. and 2. and fast dynamics. although unable to learn to imitate the long visuo-​proprioceptive sequences exactly. This was considered to be due to the sensitivity to the ini- tial conditions in the trained network. the robot was tested on imi- tating (generating) each training sequence by setting it to the acquired initial state. 1999) that interacted directly with the visuo-​proprioceptive sequences. the sequences gradually started deviating from the learned ones. Furthermore. each of which consisted of 20 transitions between primi- tive actions. The same analysis was repeated for cases of different transition probabilities of the target actions. 20. Here. which induces transitions between primitive actions represented by different attractors. namely the ori- gin and indeterminacy of spontaneity in choosing subsequent primitive actions. R C R L C R L C Primitive action 1 Activation Vision –1 1 Activation Proprioception –1 1 Unit ID Intermediate dynamics network 30 1 Unit ID Slow dynamics network 30 0 Time steps 1000 Figure 10. respectively. the observed stochastic properties should be due to some internally generated fluctuations rather than noise-​induced perturbations. Plots in the third and fourth panels show. Capital letters shown in the first panel denote primitive actions executed (R: moving to right. with different shades of gray. .2.  223 Free Will for Action and Conscious Awareness 223 deterministically in offline simulation without any contamination from external sensory noise. In other words. the spontaneity observed at junctions between chunks of action sequences seems to arise from within the robots and by way of processes perfectly consistent with results from Libet and Soon. (2011) with PLoS Creative Commons Attribution (CC BY) license. the activities of 30 neural units in the subnetworks with intermediate and slow dynamics. respectively. let’s look at the neural activation sequences in units with different timescales associated with the visuo-​proprioceptive sequences during the generation of motor imagery. as shown in Figure 10.  Time evolution of neural activities associated with visuo-​ proprioceptive sequences in motor imagery. Plots in the first panel and in the second panel show predicted vision and proprioception outputs. To gain some insight into this phenomenon. L: moving to left and C: moving to center).2. Adopted from Namikawa et al. and to center was generated in the case of the “lesion” in the slow dynamics subnet- work.2. Readers may see that this result corresponds exactly with the aforementioned idea (illustrated in Figure 6. which are stored in the lower level as well as with what Braitenberg’s Vehicle 12 predicted in section 5. If the largest component of this vector is positive. this indicates that chaos is generated by means of the stretching and folding mechanism described in section 5. A complex trajectory wandering between the three object positions was generated in the case of the intact network. The Lyapunov exponent is a multidimensional vec- tor that indicates the rate of divergence of adjacent trajectories in a given dynamic system. On the other hand. whereas a simple tra- jectory of exact repetitions of moving to left. Therefore. that chaos in the higher level network can drive compositional generation of action primitives. This implies that the lesion in the subnetwork with slow dynam- ics deprived the network of the potential to spontaneously combine primitive actions. In the plot of intermediate neural activity.2).3. The results were repeatable for different runs of training of the network. To clarify the functional role of each subnetwork. . generating pseudostochastic transitions between primitive action sequences. we next conducted an experiment involving a “lesion” artificially created in one of the sub- networks. a dynamic measure known as the Lyapunov exponent was calculated for the activity of each subnetwork.224 224 Emergent Minds: Findings from Robotics Experiments It can be seen that the neural activities in the subnetworks with intermediate and slow dynamics develop with their intrinsic timescale dynamics. neither of such apparent regularity nor apparent repeated patterns of activity can be observed. it can be seen that its dynamic pattern repeats for the same action primitive generated. it was calculated that the maximum Lyapunov exponent was positive for the subnetwork with slow dynamics and negative for the subnetworks with intermediate and fast dynamics. In the analysis. To examine the dynamic characteristics of the networks.3). implying that chaos emerged in the subnetwork with slow dynamics but not in the other subnetworks. to right.1. in the plot of slow dynamics one. deterministic chaos emerging in the subnetwork with slow dynamics might affect the subnet- works with intermediate and fast dynamics. The trajectory of the manipulated object generated as visual imagery by the original intact network was compared with the one gen- erated by the same network but with a lesion in the subnetwork with slow dynamics (Figure 10. 3. This agrees well with Freeman’s (2000) speculation that intentionality is spontaneously generated by means of chaos in the prefrontal cortex. Trajectories of the manipulated object (a) generated as visual imagery by the original intact network and (b) generated by the same network but with a “lesion” in its subnetwork with slow dynamics. It is assumed that deterministic chaos emerging in the sub- network with slow dynamics.1.  225 Free Will for Action and Conscious Awareness 225 (a) (b) 1 1 –1 –1 –1 1 –1 1 Figure 10. namely free selection and combination of actions. He con- sidered that multiple alternatives can be regarded as accidental generations with spontaneous variation from memory consolidating various experi- ences. as was illustrated in Figure 3. possibly corresponding to the prefrontal cortex. such isolation of chaos in the higher level of the organized functional hierarchy in the brain might afford the estab- lishment of two competencies essential to cognitive agency. (2011) with PLoS Creative Commons Attribution (CC BY) license. .  Comparison of behaviors between an intact network and a “lesioned” network.4. and their robust execution in an actual physical environment. The isolation of chaos into the prefrontal cortex would make sense because the robust- ness of generation of physical actions would be lost if chaos governs the whole cortical region. might be responsible for spontaneity in sequencing primitive actions by destabilizing junctions in chunk structures.2  Origin of Spontaneity The results of the robotics experiments described so far suggest a pos- sible mechanism for generating spontaneous actions and their images in the brain. Our consideration here is analogous to William James’ consideration for the mechanism of free will. Also. 10. Adopted from Namikawa et al. in which one alternative is eventually selected as the next action. Here. The unique- ness of the current model study lies in the fact that deterministic chaos emerges in the process of imitating probabilistic transitions of action primitives. Also.2. We might be justified in asking why models of deterministic dynamic systems are considered to be more essential than models of stochas- tic processes. Furthermore. This result can be understood as a reverse of the ordinary way of constructing the symbolic dynamic in which deter- ministic chaos produces probabilistic transitions of symbols as shown in ­chapter 5. complete free will without any prior causality may not exist. But. might facilitate the spontaneous generation of actions and images.1). such as Markov chains (Markov. What James referred to as intermittent transi- tions between these perches and flights might also be due to the chaos-​ based mechanism discussed here.5. I’d like briefly to discuss the issue of deterministic dynamics versus probabilistic processes in modeling spontaneity. Such fluctuations in neuronal activity. it may feel as if free will exists when one has a limited awareness of underlying casual mechanisms.226 226 Emergent Minds: Findings from Robotics Experiments Chaos present at a higher level of the brain may account for this “acciden- tal” generation with spontaneous variation. as described in section 7. readers may remember the experimental results of Churchland and colleagues (2010) showing that the low-​dimensional neural activity during the movement prepara- tory period exhibits greater fluctuation before the appearance of the tar- get and a more stable trajectory after its appearance. one thing to be noted is that wills or intentions which are spon- taneously generated by deterministic chaos are not really “freely” gener- ated because they are generated by following the deterministic causality of internal states. If we observe action sequences in terms of categorized symbol sequences. 1971). A  fundamental . they turn out to be probabilistic sequences as explained by symbolic dynamics (see section 5. possibly due to chaos originating in higher levels of organization. Now. provided that sufficient training sequences are used to induce generalization in learning. They may look as if generated with some randomness. his metaphoric refer- ence to substantial parts as “perchings” and transient parts as “flights” in theorizing the stream of consciousness might be analogous to the chunk structures and their junctions apparent in the robotics experiments described in section 3. The mechanism is also analogous to what we have seen about the emergence of chaos in conflicting situations encountered by robots. because the true internal state is not consciously accessible. Mathematically speaking. When inputs with unexpected labels arrive. and the output of the MTRNN was fed into the Kohonen network to reconstruct the predicted image of the pixel pattern. The pixel pattern received at each step was fed into the Kohonen network as a two-​dimensional topological map and the low-​dimensional winner-​ take-​all activation pattern of the Kohonen network units was input to the MTRNN. The substates are assigned nodes with labels. The intrinsic fuzziness in represent- ing levels. 2012).3  Creating Novel Action Sequences My colleagues and I investigated the capability of MTRNNs in generat- ing diverse combinatorial action sequences by means of chaos developed via the tutored learning of a set of trajectories. In contrast. and the possible state transitions between those states are denoted by arcs. primitives.. dynamical system models can avoid such catastrophic events as their dynamics develop autonomously. such as MTRNN.. at the very least. In such a discretization scheme. The only difference with an FSM is that arcs represent transition probabili- ties rather than deterministic paths. as in the case of a finite-​state machine (FSM). a Kohonen network model was used for preprocessing of the pixel level visual pattern similar to the model described in section 7. Markov chain models just halt and refuse to accept the inputs. as argued in previous sections. 2009). we often observed that MTRNNs generated novel movement patterns by combining prior learned segments in mental simulation as well as in actual behaviors (Arie et al. On the other hand. In one such humanoid robot experiment involving an object manipu- lation task. which are the most popular schemes for mod- eling probabilistic processes.. employ discrete state representations by partitioning the state space into substates. and intentions in dynamical system models. we employed an extended MTRNN model that can cope with dynamic visual images at pixel levels (Arie et al.  227 Free Will for Action and Conscious Awareness 227 reason for this preference is that models of deterministic dynamical systems more closely represent physical phenomena that take place in continuous time and space. In such experiments. 2009. 10. In training for the object manipulation task. Arie et al.2. could develop robustness and smoothness in interactions with the physical world.1. the robot was tutored on a set . In this exten- sion. Markov chain models. even a slight mismatch between the current state of the model and any inputs from the external environment can result in a failure to match. The tutoring was repeated while the posi- tion of the object in the initial condition was varied. or hallucinations. It was observed that the model could generate diverse visual imag- ery sequences. Then the object was either moved to the right or was put on the small table. Adopted from Arie et al. The tutored sequences were started from two different ini- tial conditions in which one initial condition was set as an object (a small block) that stood on the base in front of a small table (a large block).4). . The other initial condition was set as the same object laid on the base in front of the table. (2009) with permission. After learning all tutored sequences.4. From this initial condition.  A humanoid robot tutored on five different movement sequences starting from two different initial conditions of a manipulated object. the network model was tested on the generation of visual imagery as well as actual action. both for physically possible ones and impossible ones depending on the initial slow context state Time Object standing (1) Move standing object to right on base (Initial condition 1) (2) Put standing object on table (3) Lay down standing object Object laid on base (Initial condition 2) (4) Move laid object to right (5) Put laid object on table Figure 10. the standing object was either moved to the right side by pushing.228 228 Emergent Minds: Findings from Robotics Experiments of movement sequences for manipulating a cuboid object by utilizing the initial sensitivity characteristics in the slow dynamics context units (Figure 10. put on the small table by grasping. or laid down by hitting. It can be said that these robots using RNNPB or MTRNN generated something novel by trying to avoid simply falling into own habitual patterns. It is noted again that novel images can be found in the deep memory developed with relational structure among experienced ones through long-​term consolidative learning. In the test of actual action generation. the aforementioned physically possible one was suc- cessfully generated as shown in Figure 10.  229 Free Will for Action and Conscious Awareness 229 (representing the intention). the network generated an image of concatenating a partial sequence of laying down the standing object on the base and that of grasping it to put on the small table. Adopted from Arie et al. as for the physically pos- sible case. In the current case using MTRNN.5. An analogous observation has been obtained in robotics experiment using MTRNN on learning to generate composi- tional action sequences corresponding to observation of compositional Time Figure 10. Although it is physically impossible that the lying object suddenly stands up after being put on the table. For example. this strange hallucination appeared because the prior learned partial sequence pattern of grasping the standing object and putting it on the table were wrongly concatenated in the image.  The humanoid robot generated an action by spontaneously concatenating prior learned two-​movement sequences of laying down the standing object on the base and grasping it to put it on the small table. The experimental results described here are analogous to the results obtained by using an RNNPB model. it was shown that various action sequences including novel ones were generated by chang- ing the PB values in the RNNPB. In section 8. On the other hand. (2009) with permission. This impossible case involved laying down the standing object and then grasping the lying down object to put it on the small table standing up.5. . an example of a physically impossible case involved a slight modulation of the afore- mentioned possible case. diverse sequential combinations of movement primitives including novel combinations were spontaneously generated by means of chaos or transient chaos organized in the higher level network.2. ” Indeed. I “either add sugar or not. This understanding.  Free Will.2. moreover. 2015. 10. In the very beginning of the current chapter. The idea is essentially this. The next section reviews this set of robotics experi- ments. why is the awareness of a free decision delayed.230 230 Emergent Minds: Findings from Robotics Experiments gesture patterns (Park & Tani. the last one in this book. had led me to develop a further set of experiments clarifying the structural rela- tionships between the spontaneous generation of intention for action and the conscious awareness of these intentions by way of the results of said actions. as evi- denced by Libet’s (1985) and Soon’s (2008) experiments? Here. I wrote that. Through the analysis of the experimental result. If we consider that the spontaneous generation of actional intentions mechanized by chaos in the PFC is the origin of free will. An important question still remains unanswered. spontaneously generated intentions are not completely free but modified so that the conflict can be reduced. in many situations one’s own intention is only consciously recognized when confronted with unexpected outcomes. To illustrate these processes. let us consider how we recognize our own actions in daily life. * * * This is not the end of the story. and Postdiction This final section explores possible mechanisms accounting for the awareness of one’s own actional intentions by examining cases of con- flictive interactions taking place between the self and others in a robot- ics experiment. which is rather unconsciously determined” and then “only notice later that I actually added sugar when I take the first sip. we attempt to explain why free will can become consciously aware with delay immediately before the onset of the actual action. after adding cof- fee granules and hot water. we conducted a simple robotics experiment. . and it is in this interplay that con- sciousness arises. Consciousness.) It was shown that novel action sequences can be adequately generated as corresponding to observation of unlearned gesture pattern sequences conveying novel compositional semantics after consolidative learning of the tutored exemplar which did not contain all possible combination patterns. In conflicting situations. Under the aforementioned task condition. at the same time. the prediction errors for the visual inputs for l steps in the immediate past were back-​propagated through time for updating the activation values of context units at the –​ lth step in the slow dynamics network toward minimizing those errors.2.” This trial was repeated several times.” It is worth noting here that the chance of conflict is 50% because moving either left or right by the “other robot” is determined randomly.” was controlled by an extended version of MTRNN and the other robot. we examined possible interactions between the top-​down process for spontaneously gener- ating actional intention and the bottom-​up process for modifying the intention by recognizing the perceptual reality by means of the error regression mechanism in the conflictive situation. after the right hand of the “other robot” was settled in the center position for a moment. it had to fol- low the movement of the “other robot” by modifying its own intention when its decision conflicted with the “other robot. the human experimenter commanded the robot to move the hand either to the left “L” or right “R” direction at random (by using a pseudo random generator). The error regression was applied for updating the activation states of context units in the slow dynamics network over a specific time length of the regression win- dow in the immediate past.” the “self robot” was supposed to decide to move its hand either left or right spontaneously at each juncture.  231 Free Will for Action and Conscious Awareness 231 10. Meanwhile the “self robot” attempted to generate the same movement simultaneously by predicting the decision made by the “other robot. the “self robot” was trained to imitate the ran- dom action sequences of either moving left or right demonstrated by the “other robot” through visual inputs. it was expected that the robot could learn to imitate the random action sequences by developing chaos in the slow dynamics network of the MTRNN. However. In the test phase for interactive action generation with the “other robot. This was all done using a . At each trial. two humanoid robots were used in which a robot.. Because this part of the robot train- ing is analogous to the one described in the last section. Specifically. referred as the “other robot” was teleoperated by a human experimenter. 2015). In the learning phase. This update reconstructs new image sequence in the regression window in the immediate past as well as prediction of future sequence by means of the forward dynamics in the whole network.1  Model and Robotics Experiment In this experiment (Murata et  al. referred as the “self robot. Both cases were tested with the same conflictive situation wherein the inten- tion of the “self robot” in terms of the initial state of the slow context units was set so that an action sequence LLRRL was anticipated while the “other robot” actually generated an action sequence RRLLR. one-​step prediction using the error regression was quite successful by generating only a spikelike momen- tary error even at a conflictive decision point (see Figure 10. Now. The profiles of one-​step sensory prediction (two representative joint angles of the “self robot” and two-​d imensional visual inputs representing the hand position of the “other robot”) are shown in the first row. In this situation. the prediction error became significantly large at the decision point. at the point of the fourth decision the “self robot” moved its arm in the direction opposite to that of the other robot (see a cross mark in Figure 10. Although the “self-​robot” seemed to try to fol- low the movements of the “other robot” by using the sensory inputs.6a. Furthermore.2) that can perform regression of immediate past and prediction of future simultaneously in online. as observed in the experiments both without and with using the error regression. because its top-​down intention is too strong to be modified. Figure 10. and the slow context and fast context activity are shown in the third and fourth rows. the online prediction error is shown in the second row. present. In contrast.232 232 Emergent Minds: Findings from Robotics Experiments realization of the abstract model proposed in c­ hapter 6 (see Figure 6. and future as associated with current inten- tion and how such image and intention can be modulated dynamically through iterative interactions between the top-​down intentional process . its movements were significantly delayed. we examine how the neural activity represents the perceptual images of past. respectively. In fact. It was observed that the case of one-​step prediction without using the error regression was significantly poorer as compared with the one with the error regression.6b. The test for robot action generation was conducted in comparison between two conditions. The dotted vertical lines represent the decision points.) It seems that the “self robot” cannot adapt to the ongoing conflictive situation just by means of the sensory entrainment. respec- tively.6a and b show examples of the robot tri- als with open-​loop one-​step prediction. namely with and without using the error regression scheme. the movement of the “self robot” became erratic.) These results suggest that the error regression mechanism is more effective for achieving immediate adaptation of the internal neural states to the cur- rent situation than the sensory entrainment mechanism. The prediction error is shown only for the past. At this moment. and the 227th step from left to right—​in an event when the prediction was in conflict with immediate sensory input. Redrawn from Murata et al. The regression window is shown as a shaded area in the immediate past. respectively.4 . the slow context unit activity. and the fast context unit activity are shown from the first row to the fourth row. the 224th step. Also. Figure 10.8 error error MSQ MSQ 0. there were dynamic changes in the activity of fast context units. This dynamic activity prepares a bias to move the hand to a particular direction. the prediction error. It is noted that although the joint angles of the “self robot” were settled. the pre- diction by the “self robot” was betrayed because the hand of the “other . it can be seen that the error arose sharply in the immediate past when the current “now” was at the 221st step.  233 Free Will for Action and Conscious Awareness 233 (a) (b) R R L R R 1 R R L L R 1 prediction prediction sensory sensory 0 0 –1 –1 0 100 200 300 400 0 100 200 300 400 0.0 .7 shows plots for the neural activity at several now steps—​the 221st step.8 . The plots for the sensory prediction (joint angles and visual inputs). which was the right direction in this case. The hand of the “self robot” started to move once to the right direction around the 215th step after settling in the home position for a moment (see the leftmost panels).  The results of the self-​robot interacting with the other robot by the open-​loop generation without (a) and with (b) the error regression mechanism. (2015). They show profiles for the past and for the future.6. with the current step of now sandwiched between them.4 0. and the bottom-​up error regression process while online movement of the robot.0 0 100 200 300 400 0 100 200 300 400 1 1 activity activity slow slow 0 0 –1 –1 0 100 200 300 400 0 100 200 300 400 1 1 activity activity fast fast 0 0 –1 –1 0 100 200 300 400 0 100 200 300 400 time step time step Figure 10. naturally. 8 0.8 Prediction error 0. the 224th step in the center panels. .0 0. Redrawn from Murata et al.8 0. and the 227th step in the right panels.4 0. Each panel shows profiles corresponding to the immediate past (the regression window) with solid lines and to the future with dotted lines. The current “now” is shifted from the 221st step in the left panels. Profiles of sensory prediction. and activations of slow and fast context units are plotted from past to future for different current “now” steps. 234 Regression Now Now Now window (step = 221) (step = 224) (step = 227) 1 Plan 1 Past 1 Past Plan Past Plan prediction modulated Sensory 0 0 0 Overwritten past 1 –1 –1 180 200 220 240 260 180 200 220 240 260 180 200 220 240 260 0.  The rewriting of future by prediction and past by postdiction in the case of conflict. (2015). prediction error.4 0.0 180 200 220 240 260 180 200 220 240 260 180 200 220 240 260 1 1 1 Slow context units 0 0 0 –1 –1 –1 180 200 220 240 260 180 200 220 240 260 180 200 220 240 260 1 1 1 Fast context units 0 0 0 –1 –1 –1 180 200 220 240 260 180 200 220 240 260 180 200 220 240 260 Time step Figure 10.7.0 0.4 0. 2. that people may notice their own intentions in the specious present when confronted with conflicts that must be reduced. Murata et al. our experiments show how conflicts might arise due to the nature of embodied.2 Interpretation Can we apply the aforementioned analysis to account for the delayed awareness of free will? The reader may assume that no conflict should be encountered in just freely pressing a button as in the Libet experi- ment. Thus. Eagleman & Sejnowski. 2014).. What we have observed here is postdiction1 for the past and prediction for the future (Yamashita & Tani. we can see discontinuity in the profiles of the slow context unit activ- ity at the onset of the regression window. Then. situated cognition. the rewritten window in our model may correspond to the encompassing narrative history as space of time in its thought. This structure reminds us of Heidegger’s characterization of the dynamic interplay between looking ahead to the future for possibilities and regressing to the conflictive past through reflection where vivid nowness is born (see section 7. at this point the robot becomes self-​reflective for own past and future!! Especially. we are led to a natural inference. This modification caused the overwriting of all profiles of the sensory prediction (reconstruction) and the neural activity in the regression window by means of the forward dynamics recalculated from the onset of the window (see the panels of the current “now” at the 224th step. When an intention uncon- sciously developed in the higher cognitive level by deterministic chaos 1. 2015) by which one’s own action can be recognized only in a “postdictive” manner when one’s own actional intention is about to be rewritten. Postdiction is known as perceptual phenomena in which a stimulus presented later affects the perception of another stimulus presented earlier (e.. 2012. Then. 10. .2. Here.) Surely.  235 Free Will for Action and Conscious Awareness 235 robot” moved to the left. the error signal generated was propa- gated upstream strongly and the slow context activation state in the starting step of the regression window was modified with effort. However.) The profiles for future steps were also modified accordingly while the error was decreased as the current “now” shifted to the 224th and to the 227th steps.g. with the effort resulting in con- scious experience. the arm of the “self robot” moved to the left. 2000. Shimojo. Likewise. This interpretation of our . It is like when a locomotive suddenly starts to move. In short. This consciously aware intention is different from the original uncon- scious one because it has been already rewritten by means of postdic- tion. the following freight train cars cannot follow immediately. in terms of the preceding experimental model. when higher levels cannot receive exactly the expected response from lower levels.8. t 2. Motor 4. they are not accompanied by consciousness. which can call for a certain modification of the intention for the movement in the direction of minimizing the error. if actions can be generated automatically and smoothly as intended exactly in the beginning. conscious awareness arises. when they are generated in response to con- flicts arising due to the nature of embodiment in the real world. some prediction error is generated. Embodiment entails certain amount of error. including muscle potential states. and the wheels spin as the system over- comes resistance to new inertia. may not be always ready to initiate physical body movements according to top-​down expectations. Spontaneous Error proprioception generation of intention by chaos Parietal in PFC. it attempts to drive the lower peripheral parts to generate a particular movement abruptly (see Figure 10. However. Figure 10. these actions are accompanied by consciousness. exceeds a certain threshold.8). Here. the engineer may slow the engine speed to optimize the acceleration and get the train going properly. As the wheels spin.236 236 Emergent Minds: Findings from Robotics Experiments 3. when the intention for the movement that has been developed unconsciously is modified. However.  Account for how free will can be generated unconsciously and how one can become consciously aware of it later. Intention drives lower level. Intention modulated signal Proprioception by error conscious Error M1 Prediction of 1. the lower levels may not be able to respond to this impetus immediately because the internal neural activity in the peripheral areas. Criticality.  237 Free Will for Action and Conscious Awareness 237 experimental results is analogous to the aforementioned speculation made by Desmurget and colleagues (2009) (see section 4. the point is that our conscious minds cannot see how they develop deterministically through causal chains in unconscious processes. (4) the intention. referring to Merleau-​Ponty: In reality. prediction error is generated between the intended state and the reality in the external world. On this account.3) that the parietal cortex might mediate error monitoring between the predicted perceptual outcome for the intended action and the actual one. (2) the top-​down intention is spontaneously fluctuated by means of the chaotic dynamics without accompanying consciousness. However. p 506). (1)  deterministic chaos is developed in the higher cognitive brain area. we may say that consciousness is the feeling of one’s own embodied neural structure as it physically changes in adaptation to a changing. then. which has been modified by means of the error regression (postdiction). (3)  at the moment of initiating a physi- cal action as triggered by this fluctuated intention. Freeman (2000) also pointed out that action precedes conscious decision. and Authenticity I explored further possibility for applying the MTRNN model extended with the error regression mechanism to a scenario of incremental and . a pro- cess through which one becomes consciously aware.6) might be right in saying that there is no space left for free will because every “free” action is determined through deterministic dynamics. Therefore. unpredict- able or unpredicted external environment. whereas third-​ party observation of the physical processes underlying its appearance tells a different story. 1962. 10. we feel as if our intentions or wills could be gener- ated freely without cause.3  Circular Causality. the deliberation follows the decision—and it is my secret decision that brings the motives to life (Merleau-​Ponty. the relationship between free will and consciousness can be accounted for in the following way. but only notice that each freeaction seems to pop out all of a sudden without any cause. Thomas Hobbes (section 3. To sum up. my account is that free will exists phenomenologically. becomes consciously noticed as the cause for the action about to be generated. In terms of human cognition.2. If considered as just discussed. When I taught a set of movement sequences to the robot. the robot would suddenly ini- tiate an unexpected movement by pulling my hands. resulting from the enactment of such novel intentions. Such experiences.9a. can be learned successively and can induce further modi- fication of the memory structure in the robot brain. I occasionally interacted with the robot in order to modify its ongoing movement by grasping its hands. Intentions for a variety of novel actions can be generated again from such reconstructed memory structures. and consequently its internal neural state was modified by means of the resultant error regression process. .1. When I  pushed them back in a different direction. This was because the reaction forces generated between the robot’s hands and my hands were transformed into an error signal in the MTRNN model in the robot’s brain.9. In these interactions. What I witnessed is illustrated with a sketch shown in Figure 10.) While the robot generated such actions. they responded with something in another way. I understand that novel patterns of the robot were more likely to be generated when my response conflicted with that of the robot. (a) (b) Re-structuring of memory Memory structure Spontaneous generation of Conscious novel intention experience Unpredicted Novel perception action Environment/other agents Figure 10. the robot gen- erated various images as well as actual actions by spontaneously combin- ing these sequences (this is analogous to the experiment results shown in section 10.  Circular causality.238 238 Emergent Minds: Findings from Robotics Experiments interactive tutoring. (a) Chain of circular causality and (b) its appearance by means of mutual prediction of future and regression of past between a robot and myself. Now. because such a venture looked so fascinating to me. The problems we focused on were how free will for action can emerge and how it can become the content of consciousness. sometimes I noticed that my own next movement image popped out suddenly without my conscious control. Ultimately. creating a rich subjective experience in the exploration of my own consciousness and free will through my online interaction with neurodynamic robots.) When I concentrated on tactile perception for the move- ment of the robot in my grasp. 10. Summary This chapter tackled the problems of consciousness. At the same time. accompanied by spontaneous shifts between conscious and unconscious states of mind after repeated confrontation and reconcilia- tion between the subjective mind and the objective world. I  became sure that the interaction between the robot and me exhibited its “authentic” trajectory. finally. I also noticed that tension between me and the robot rose up to critical level occasionally from where unexpected movement patterns of mine as well as of the robot burst out. First. diverse images.3. free will or free action might be generated in a codependent manner between “me” and others who seek for the most possibility in the shared social situation in this world. Furthermore. Consequently. . and (4)  incremental learning of these new experiences and the resultant reconstruction in the memory structure.  239 Free Will for Action and Conscious Awareness 239 This sketch depicts that there is a circular causality among (1) spon- taneous generation of intentions with various proactive actional images developed from the memory structure. but also to enjoy myself.9a include also me as I insert myself into the cir- cular causality in the robotics experiment described in this section (see Figure 10. our study investi- gated how intention for different actions can be generated spontaneously. intention. Here. Although I may be unable to articulate the mechanics behind such experience in greater detail through unaided introspection. I realized that I had conducted robotics experimental studies not only to evaluate the pro- posed cognitive models objectively.9b. (2) enactment of those actional images in reality. alone. and thoughts can be generated. it is worth noting that the emergent processes described in Figure 10. actions. an open dynamic structure emerges by way of the aforementioned circular causality. and free will through the analysis of neurorobotics experimental results. (3) conscious experience of the outcome of the inter- action. These considerations lead us to conjecture that there might be no space for free will because all phenomena including the spontaneous generation of intentions can be explained by causally deterministic dynamics. It was postulated that. Finally. and successive learning of such experience in the robot–​human interactive tutoring experiment. It was speculated that one becomes consciously aware of one’s own intention for generating action only via postdiction. We enjoy. as the conflict emerges between the higher level unconscious intention for initiating a particular movement and the lower level perceptual real- ity by embodiment. And this inter- pretation accords with experiment results as delivered by Libet (1985) and Soon and colleagues (2008). a robotics experiment simulating conflictive situations between two robots was performed. when the originally generated intention is modified in the face of conflicting perceptual real- ity. because we feel as if freely chosen actions appear out of a clear sky in our minds without any cause. because of this circular causal- ity.. all processes time-​develop in a groundless manner (Varela. The experimental results showed that spontaneously generated intention in the higher level subnetwork can be modified in a postdictive manner by using the prediction error generated by the con- flict. an experience of free will subjectively. embodiment of such intention in reality. as in the experiment by Libet. because our conscious mind cannot trace its secret development in unconscious process. The next question tackled was why conscious awareness of the inten- tion for generating spontaneous actions arises only with a delay imme- diately before actual action is initiated. conscious experience of perceived outcomes. For the purpose of considering this question. et al. however.240 240 Emergent Minds: Findings from Robotics Experiments It was found that actions can be shifted from one to another spontane- ously when a chaotic attractor is developed in the slow dynamics sub- network in the higher levels of the cognitive brain. . the delayed awareness of one’s own intention can be explained similarly. the chapter examined the circular causality appearing among processes generating intention. In the case of generating free actions. The experiment used an extended version of the MTRNN model employing an error regression scheme for achieving online modification of the internal neural activity in the con- flictive situation. This implies that intention for free action arises from fluctuating neural activity by means of deterministic chaos in the higher cognitive brain area. which results in generation of the prediction error. our minds might become ultimately free only when gifted with such groundlessness. And thus. whereby images and actions are generated diversely. The vividness and the authenticity of our “selves” might appear especially at a certain crit- icality under such groundless situations developed through circular causality.  241 Free Will for Action and Conscious Awareness 241 1991)  without any convergence to particular situations. . 242 . only then to face the “problem of interactionism.” that is. asking how symbols considered as arbitrary shapes of tokens defined in nonmetric space could interact densely with sensory–​ motor reality defined in physical and material metric space (Tani. and vice versa.  243 11 Conclusions Now. 2016). Actually. after completing descriptions of our robotics experiment out- comes. this final chapter presents some conclusions from reviewing these experiments. 11. 243 . wherein René Descartes suggested that the mind is a nonmaterial. Taniguchi et al. expound- ing how nonmaterial minds can cause anything in material bodies. today’s symbol grounding problem addresses the same concern. I considered that this problem originated from Cartesian dual- ism.1..  Compositionality in the Cognitive Mind This book began with a quest for a solution to the symbol grounding problem by asking how robots can grasp meanings of the objective world from their subjective experiences such as the smell of cool air from a refrigerator or the feeling of one’s own body sinking back into a sofa. material body. thinking thing essentially distinct from the nonthinking. 2014. the cognitivist’s first assumption is that an essential aspect of human cognition can be well accounted for in terms of logical symbol systems. the substantial strength of which being that they can support an infinite range of recursive expressions. A mathematical study by Siegelmann (1995) and recently . in the first place. inspired by Merleau-​Ponty’s philosophy of embodi- ment. our models have successfully demonstrated what Merleau-​Ponty described metaphorically as the reciprocal inser- tion and intertwining of the subject and the object through which those two become inseparable entities. In everyday situations. subjective. one crucial question is whether or not it is necessary for the daily actions and thoughts of human being to be supported by such an infinite length of recursive compositions. the series of robot- ics experiments described in this book confirm this characterization.244 244 Exploring Robotic Minds In this book. Our multiple timescale recurrent neural networks (MTRNNs) can learn to imitate stochastic sequences via self–​organizing deterministic chaos with complexity of finite state machines. iterative interactions between top-​ down. a human being speaks only with a limited depth of embedded sentences. And. However. but not with that of infinite ones. Instead. It might be still difficult for proponents of cognitivism such as Chomsky to accept such a line of thought. This learning grounds higher level cognition in perceptual reality without suffering the disjunction between lower and higher level operations that is often found in hybrid models employing symbolic composition programs. nonlinear dynamic systems onto robotic platforms. Consequently. in which my colleagues and I  have engineered self-​organizing. As mentioned in ­chapter 2. The second assumption is that sensory–​ motor or semantic systems are not necessary for the composition or recursion taking place in terms of symbol systems. I  attempted to resolve this longstanding problem of mind and body by taking synthetic approaches. and makes action plans composed of only a limited length of primitive behavior sequences at each level. Our central hypoth- esis has been that essential cognitive mechanisms self-​organize in the form of neurodynamic structures via iterative learning of continu- ous flow of sensory–​motor experience. intentional processes of acting on the objective world and bottom-​up recognition of perceptual reality result in the alteration of top-​down intention through circular causality. An infinite depth of recur- sive composition is required in neither case. and therefore may not be essential components of any cognitive systems. The book presents the experimental trials. and that the contents of these compositions can remain naturally grounded in the ongoing flow of perceptual reality throughout this process. This scope may include the daily utterances of children. My work with robots has attempted to model everyday analogical processes of ordinary humans generating behaviors and thoughts characterized by an everyday degree of compositionality. that they can exhibit computational capabilities beyond the Turing limit. sensory–​motor reality that its practical embodiment may require.. prob- lems and questions remain. including recurrent neural networks (RNNs) with external memory for writing and reading.. or even making a cup of instant coffee. 1956). even if an equivalence to such a Turing machine might be constructed in an RNN by chance (Tani et al. 2014). These researchers argue that these grandmother cells might function like symbols. On this count. many electrophysiological researchers have argued for the existence of so-​called grandmother cells based on studies of animal brains in which local firings are presumed to encode specific meanings in terms of a one-​to-​one mapping. who can compose sentences in their mother language with- out explicitly recognizing their syntactic structures. because the corresponding parameters such as connectivity weights can be found only in singular points in the weight space.” On the contrary. Of course. and also include the tacit learning of skilled actions such as the grasping of an object to pass it to others without thinking about it. the construction of such Turing machines through learning is practically impossible. This should be the same for ordinary human cognitive processes that rely on relatively poor working memory characterized by the magic num- ber seven (Miller. A neurophys- iologist once emphatically argued with me. a typical concern people often ask me about is whether symbols really don’t exist in the brain (Tani et al. Our robotics experiments have demonstrated that self-​ organization of particular dynamical structures within dynamic neural network models can develop a finite level of compositionality. Even though we may have created an initial picture of what is happening in the mind. For example. before the age of 5 or 6. I thought it a possibility that this neuron could fire for generating other types of . situated in the noisy. denying the possibility of distributed representations.  245 Conclusions 245 by Graves and colleagues (2014) have proved the potential of analog computational models. Such a parameter-​sensitive system may not function reliably. However. this is far from the end of the story. 2014). saying that “this recorded neuron encodes the action of reaching to pulling that object. or Soar. and so any presumption that something like direct symbolic representations exist in the human brain seems equally to be in error. has recently investigated this problem by extending Soar (Laird. may be internalized and employed through working memory like a “blackboard” in the brain to “write down” our thoughts when we don’t have pen or paper handy. thereby distributing thought through symbolic representations.g. or item- ized statements on paper or in other media utilizing symbols. these cognitive architectures are good at manipulating symbols as they exist outside of brains by utilizing explicit knowledge or rules. The extended Soar contains additional building blocks that are involved in the learning of tacit knowledge about perception and action generation . I  speculate that we humans use discrete symbols out- side of the brain depending on the situation. Human civilization has evolved through the use of outside-​brain devices such as pens and paper to write down linguistic symbols. Actually. Tasks at this level might be solved by cognitive architectures such as Act-​R. we typi- cally compose these plans into flow charts. an animal shape. my argument has been that our brain can facilitate everyday compositionality such as in casual conver- sation or even regular skilled action generation by combining primitive behaviors without needing to (fully) depend on symbol representation or manipulation in the outside-​brain devices. an actress’ face. Mormann and colleagues’ (2008) results from multiple-​ cell recordings of the human medial temporal lobe revealed that the firing of cells for a par- ticular concept is sparse (firing of around 1% cell population) and that each cell encodes from two to five different concepts (e.246 246 Exploring Robotic Minds actions that could not be observed in his experiment setting in which movements of the animals were quite constrained. 2008). schematic drawings. this poses the question of how these symbols outside of the brain can be “grounded” in the neurodynamic structures inside the brain. Indeed. one of the original inventors of Soar. So.” This use of external representation. and a mathematical formula). In this book. more- over. recent devel- opments in multiple-​cell recording techniques suggest that such map- pings are more likely to be many-​to-​many than one-​to-​one.. Indeed. when we need to construct complicated plans for solving com- plex problems such as job scheduling for a group of people in a company or basic designing for building complex facilities or machines. That aside. John Laird. Even though con- cepts are represented sparsely. Still. their representation is not one-​to-​one but distributed. GPS. an aspect of what Clark and Chalmers (1998) have called “extended mind. which is characterized by interactions between explicit processes real- ized by symbol systems and implicit processes by the connectionist net- works under the similar motivation.2.) In this spirit. Next actions are determined by applying production rules to the memory contents in the STM. as . CLARION. Ron Sun (2016) have developed a cognitive architecture. only simpler. 2000. and free will by drawing correspondences between the outcomes of neurorobot- ics experiments and some of the literature in traditional phenomenol- ogy. It should be true that human phenomenology. and underlying brain mechanisms can be understood only through their mutual constraints imposed on the formal dynamical models. The essential questions would be from which level in cognitive process external symbols should be used and how such symbols can be interfaced with sub-​symbolic representation. subjective time. These questions are left for future studies. I  speculate that the introduction of symbolic representa- tions in STM in Soar or in the explicit level in CLARION might be too early. Similar research trials can be seen elsewhere (Ritter et al. 11. They are the systems in ques- tion. Such subsymbolic levels are interfaced with symbolically represented short-​term memory (STM) in next level. and in this way models are not metaphors. consciousness. as I have shown repeatedly in the current book. Phenomenology The current book also explored phenomenological aspects of human mind including notions of self. human behav- ior. 2008). against this I would argue that models capture aspects essential to a phenomenon. because such representations can be developed still in a nonsym- bolic manner such as by analog neurodynamic patterns. St Amant & Riedl..  247 Conclusions 247 without using symbolic representation. Although these trials are worth examining. Although some may argue that such analysis from the synthetic modeling side can never be more than metaphorical. reduc- ing the complexity of a system to only these essential dimensions. Bach. 2001. at least in so far as essential dimensions are indeed modeled and nothing more (see further discussion by Jeffrey White [2016]. and there will undoubtedly be many more we will face. I believe that interdisciplinary discussions on the outcomes of such neurorobotics experiments can serve to strengthen the insights for connecting aspects of robot and human behaviors more closely. However.4. This process is char- acterized by a steady phase of neurodynamic activity. when the pre- diction error is generated. In fact. I proposed that the problem of segmenting the contin- uous perceptual flow into meaningful reusable primitive patterns might be related to the problem of time perception as formulated by Husserl.2. 1987). In this way. An important observation was that these two phases alternated intermittently by exhibiting the characteristics of self-​organized criticality (Bak et  al. the flow is segmented into chunks by means of a parametric bias vector modification with an effort for minimizing the error. compositionality in cognition might be related to the . This is because compositional actions generated by others entail potential unpredictability when such actions are composed of primi- tive acts voluntarily selected by means of the “free will” of the oth- ers. When the continuous perceptual flow can be anticipated without generating error. there is no sense of events passing through time. It was considered that the authentic being might be accounted for by this dynamic structure. it was explained that the subject (carpenter) and the object (hammer) form an enactive unity when all of the cognitive and behav- ioral processes proceed smoothly and automatically. and the “self” comes to be noticed consciously. but rather represent discrete events that can be consciously identified according to the perceptual categories as encoded on our model by the PB vector. we reviewed an experiment involving robot imitation learning that uses the RNNPB model. In the unsteady phase. the passing of time comes to conscious awareness. it was argued that the “self” might come to conscious awareness when coher- ence between internal dynamics and environmental dynamics breaks down. robotics experiments of the sort reviewed in this text afford privileged insights into the human condi- tion. let us review these experiments briefly. when subjective anticipation and perceptual observation conflict. In section 8. To reinforce these insights. For the purpose of examining this thought. From the analysis of these experimental results. the distinction between these two becomes explicit.248 248 Exploring Robotic Minds Varela (1996) pointed out. The segmented chunks are no longer just parts of the flow. By referring to Heidegger’s example about a carpenter hitting nails with a hammer.. With this. In the robot navigation experiment described in section 7. Therefore. it is interesting to see that the observation of compositional actions by others accompanies the momentary consciousness at the moment of segmenting the perceptual flow into a patterned set of primi- tives. it was speculated that “now- ness” is bounded where the flow of experience is segmented. ” In ­chapter 9.2 suggest that conscious awareness of the intention developed by such deterministic dynamics can arise only in a postdictive manner when conflicts arise between top-​down prediction and bottom-​up reality. thus. With the relationship between free will and consciousness thus clarified. I wrote that the capability of abstraction through hierar- chy in MTRNN can provide robots with competency of self-​narrative for own actional intention in mental simulation. c­ hapter 10 was devoted to the relationship between free will and conscious experience in greater depth. By con- sidering possible situations in which the intention to enact a particular movement generated in the higher level conflicts with the sensory–​motor reality as constituted in the lower level. Our minds cannot observe the phase space trajectory of chaos developed in the higher cognitive brain area.1). Though it is true that in our everyday subjec- tive experience we feel as if free will exists. By following this argu- ment. On this account. We are conscious of each intention as if it pops up without any prior cause immediately before the corresponding action is enacted. there might be no space for conscious- ness or for the experience of free will in their “minds. I speculated that reflective selves of robots may originate from this point. this chapter suggested that there might be no space for free will from an objective view because all of the mechanisms necessary for generating voluntary actions can be explained by deterministic dynamics due to causal physical phenomena. From results of robotics experi- ments utilizing the MTRNN model (section 10. through the results of our neurorobotics experiments we can see that this phenomenon may arise simply because our minds cannot see the causal processes at work in generating each intentional action. Results of the robotics experi- ment shown in section 10. Finally. it was proposed that an effort autonomously mechanized for reducing the conflict would bring the intention to conscious awareness. If some animals live only on sensory-​reflex behaviors without the ability to either recognize or generate compositional actions. I proposed that inten- tions for free actions could be generated spontaneously by deterministic chaos in the higher cognitive brain area. as I  have shown in our robotics experiments. we may conclude that free will exists but merely as an aspect of our subjective experience.  249 Conclusions 249 phenomenology of free will and consciousness. I will reiterate once more that the problem of consciousness . This observation was correlated with the account for the delayed awareness of free will reported by Libet (1985). whereas the bottom-​up postdictive recognition of such changes including unexpected ones may induce changes in memory and intention in the subjective mind. . However. Along this line. If consciousness is considered to be the first person awareness of embodied physical processes. for example. In the loop of circular causality. qualia might be a special case of conscious experience that appears when the gap is generated only in the lower perceptual level in which the vividness of qualia may be originated from the prediction error residual at each instance. and that nothing exists outside of them (something “supernatural”). or in terms of the aforementioned gap or prediction error. for example.250 250 Exploring Robotic Minds may not be the hard problem after all. as these two are dependent on each other within the same dynamic structure. The top-​down proactive intention acting on the objective world induces changes on this world. The crucial proposal in the current book is that the circular causality developed between the subjective mind and the objective world is responsible for consciousness and also for an appear- ance of free will. 2008). a more essential issue is to understand the underlying struc- ture of consciousness rather than just a conscious state at a particular moment that is measured post hoc in terms of integrated information (Tononi. these poles are differentiable in terms of the gap between them. and objectivity by the bottom-​up recognition of the percep- tual reality. Friston (2010) would say that it is from the error divided by the estimated variance (uncertainty) rather than the error itself. spontane- ous shifts between unconscious state in terms of the coherent phase and conscious state in terms of the incoherent phase occur intermittently as the dynamic whole develops toward criticality. consciousness at each moment should appear as a sense of an effortful process aimed at minimizing this gap. This stands to reason. Consequently. the stream of consciousness formulated as spontaneous alternation between conscious state and unconscious state by William James (1892). then an exhaustive account of consciousness should likewise appear via the explanation of the relationships between the subjective and the objective. of course. Then. When subjectiv- ity is exemplified by the top-​down pathway of predicting an actional outcome. provided that the whole of this universe is also constituted by these two poles. We have to explain the underlying structural mechanism accounting for. and more specifically. This could result in another emergence of “free” action by means of the poten- tial nonlinearity of the system. this open dynamic structure developed in the loop of the circular causality should account for the autonomy of consciousness and free will. 1991)  in terms of fluctuated interac- tion between the subjective mind and the objective world. 11. critical- ity developed in this open. In ­chapter 9. For example. ­chapter  7 employs these two different approaches in the study of robot navigation learning..3 describes the one-​way imitation learning of the robot to show that the RNNPB model can learn to generate and recognize a set of primi- tive behavior patterns by observing movements of its human partner.  251 Conclusions 251 To sum up. Conclusively. I described .1 described how the RNN model used in mobile robots can develop compositional repre- sentations of the outer environment and how these representations can be grounded. dynamic structure might account for the authenticity thought by Heidegger that generates trajectory toward own most possibility by avoiding just falling into habitual or conventional ways of acting (Tani. One type of my robotics experiment focuses more on how adequate action can be generated based on the learning of a rational model of the outer world. 2009). Section 8.  Objective Science and Subjective Experience The readers might have noticed that two different attitudes in conduct- ing robotics experiments appear by turns in Part II of the current book. I introduced the imitation game experiment in which two-​ way mutual imitation between robot and human was the focus.2 explored characteristics of groundlessness (Varela et  al. Afterward.3. It was observed that some psychologically plausible phenomena such as turn taking of initiative emerged in the course of the imitation game. in this case social. Reflective selves of robots that can examine own past and future possibility should originate from this perspective. Or. it can be said that this open dynamic structure explains the inseparable nature of the subjective mind and the objective world in terms of autonomous mechanisms moderating the breakdown and unification of this system of self and situation. On the other hand. world. whereas the other type focuses more on the dynamic characteristics of possible interactions between the subjective mind and the objective world. reinforc- ing our emphasis on the interaction between the first-​personal subjec- tive and the objective. section 7. Section 7. my research attitude has been shifting between one side of investigating rational models for cognitive mechanisms from an objective view and the other side of exploring subjective phenomena by means of putting myself inside the interaction loop in robotics experi- ments. such an observer is regarded as an internal observer because it is included in the internal loop of the interactions. However. To sum up. each process attempts to change its current relations as if it were expected that the inconsistency will be resolved sometime in the future and as long as the interaction continues (Gunji & Konno. limitation in memory capacity. but his play and other band members’ got tensed . Matsuno (1989) and Gunji (Gunji & Konno. On the other hand. This chapter also reported how novel image and action can be generated both in robot and human sides during interactive tutoring of robots by human tutors. the processes can- not be merely terminated. consists in a set of embodied processes that are physically constrained in various ways such as by imprecision in percep- tion and in motor generation. and so on. Actually. recognition of percep- tual outcomes. They used the term observation as mostly equivalent to the term interaction. 1991). passive observation from observed to observer without any interactive feedback. Such physical con- straints in time and space do not allow the system to be uniquely opti- mized and thus give rise to incompleteness and inconsistency.252 252 Exploring Robotic Minds how the MTRNN model can learn compositional action sequences by developing an adequate functional hierarchy in the network model. Observation. A  brilliant tenor sax player like the late Michael Brecker often started a tune with familiar phrases of improvisation in calm. in our robot experiments. and the learning of resultant new experience. such inconsistencies arise in every aspect of cognitive processes including action generation. When the relationship between the observer and the observed can alter because of the interactions between them. Then. itself. at the moment of encountering such an inconsistency. We can experience something analogous to this when we go to a gig of “cutting-​edge” contemporary jazz. Instead. 1991)  wrote that the former type of research attitude would take a view of the so-​called external observer and the latter of the so-​called internal observer. c­ hapter  10 examined how circular causality can be developed among different cognitive processes for the purpose of investigating the free will problem by using the same MTRNN model. time delays in neural activation and body movement. the external observer assumes only one-​way. generation. The former research can be much more advanced by using the recent results from the booming research programs on machine learning and deep learning in which the connectionist approach with employing the error back propagation scheme has been revived by introducing more elegant mathematics to the models than those in 1980’s. On the other hand. one toward stabil- ity and the other toward instability. However such goal-​d irected attempts always entail instability because of their embodiment as well as potential openness of adopted envi- ronment that resulted in the groundlessness. In the next moment. All processes of recognition. For further advancement of the latter part. likely to break down at any moment. now I  become sure that both research attitudes are equally important for the goal of understanding the mind via synthesis. 2001. 2013) wherein we can find the vividness of a living system. the unbelievable tension of sound and phrase burst out. it is crucial to build rational models of cognition with the goal of optimization and stabilization of each elementary cognitive process. Ikegami & Iizuka. which can be accounted for such as by the prediction error minimization principle employed in our models. These activities are geared toward grounding as shown in some of our robotics experiments. It is interesting to note that cognitive minds appear to maintain two processes moving toward opposite directions. His genuine cre- ativity in such thrilling playing resulted not merely from his outstanding skills for improvising phrases or for perfect control of the instrument but originated from the urgent struggle for enactment of his exploded mental image and intention. By overviewing my research history.  253 Conclusions 253 gradually through mutual responses. Ikegami. it is equally cru- cial to explore dynamic aspects of mind while the optimization is yet to be achieved during the ongoing process of robots acting in the world. as we have witnessed in our other robotics experiments. we need to explore the methodology . On the one side. The coexistence of the stable and the unstable nature does not allow the system state to simply converge but imbues the system with autonomy for generating itinerant trajectory (Tsuda. his play sometimes got stuck for an instant as his body control for blowing or tonguing seemed unable to catch up with his rushed image any more. 2007. and learning can be regarded as goal-​d irected activities. however. At the near peak of the tension. The goal directedness is consid- ered as an attempt to achieve the stability of the system by resolving the currently observed inconsistencies of the system. meditators become able to let their minds present themselves or go by themselves by devel- oping a mood for stepping back. 1991).. I felt more vividness on the robot movement and also experi- enced more spontaneous arousal of kinesthetic image for my own move- ment.. 1991) and in their so-​called neurophenomenology program (Varela. if we attempt to develop ultimately natural. 1996). 1991. Analogously. The more deeply the . It was like floating in the middle way between the two extremes of the subjectivity and the objectivity. Such intensive interaction alternated between a more tensed.2. when I was more absorbed in the robot interaction by concentrating on tactile perception for the movement of the robot in my grasp. The Buddhist mindful awareness tradition starts with practices to sus- pend habitual attitudes granted in everyday life (Varela et al. such as what robots or human should do or should not do. In my own experience of interacting with the robot as described in section 10. spontaneous mindful interactions between robots and human. which have been assumed in the conventional human–​robot interaction frame- work. The ongoing interaction was neither dominated by my subjectivity nor the objectivity of the robot. Varela and colleagues proposed to build a bridge between mind in science and mind in experience by articulating a dialogue between these two tra- ditions of Western cognitive science and Buddhist meditative psychology (Varela et al. Why Buddhist meditation for the analysis of subjective experience? This is because the Buddhist tradition of medita- tion practice spanning more than 26 centuries has achieved systematic and pragmatic disciplines for accessing the human experience.254 254 Exploring Robotic Minds of articulating the subjective experience of the experimenters who are within the interaction loop in the robotics experiment. This exactly follows what Varela and colleagues proposed in the embodied mind (Varela et  al. What we need to do is to enhance further the circular loop between the objective science of modeling cognitive mechanisms and the prac- tice for articulating the subjective experience. By practicing this suspension of the habitual attitude.. Parts of Buddhist meditation disciplines could be applied directly to our problem of how to articulate the subjective experience of the experimenter in the robotics experiment loop. xviii). It is noted that con- tinuance of such subtle interaction depended on how diverse memory patterns were consolidated by developing generalized deep structure in the dynamic neural network used in the robot. as I  already mentioned. we should get rid of arbitrary thinking in the human sub- jects. conflictive phase and a more relaxed one. 4. In the process. including Edelman’s group (Fleischer et  al. 2010). we may also examine how these models correspond with evi- dence from neuroscience. I should emphasize. We still need a good abstraction of the biological reality to build tractable models.  255 Conclusions 255 memory structure develops. We may not need to recon- struct the whole brain by simulating activity of 100 billions of biological plausible neurons interconnected with columnar structure as like aimed by Blue Brain project (see section 5.4). and the other for practicing to achieve truly mindful interaction with the robots. now might be a good time to start trying to reconstruct a global model of the brain so that we can synthetically examine what sorts of brain functions appear locally and globally with both static and dynamic connectivity constraints. . An exciting future task might be to build a large-​scale brain network by using either rate-​coding neural units or spiking neurons for artificial humanoid brains. The sizeable amount of human brain imaging data that have been gath- ered to date has enabled a global map to be created of both static con- nectivity and dynamic connectivity between all the different cortical areas (Sporns. 11. In summary. the more intriguing the generated images become. Thanks to such data. it is highly expected that the goal of understanding the mind can be achieved by making efforts both in the objective sci- ence and the subjective experience. The enhancement of the employed models greatly contributes to realization of sensible interactions between the robots and the human subjects. that large scale does not mean a complete replica of real brains. 2007)  and Eliasmith (2014) by introducing millions of spiking neurons in their models. Such experiments have been started already by some researchers.  Future Directions Although this book has not concentrated on modeling the biological reality of the brain in details recent exciting findings in system-​level neuroscience draw me to explore this area of research more explicitly. True features of the mind should be captured by undertaking such research trials of moving back and forth in exploration of objective science and subjective experience.. however. one for investigating more effective cognitive models assuring for better performance and scalability. and depression.4. autism. Humanoid Brain project would clarify the underlying mechanism on the functional differentiation observed across local areas in our brains in terms of downward causation by the functional connectivity and the multiple spatio-​temporal scales property evidenced in human brains and by embodiment in terms of structural coupling of the peripheral cortical areas with sensory-​motor reality. we can start such an enterprise. Now. 2015. It is also inferred that the known diversity in cell types as well as in synaptic con- nection types can be regarded as biological details which may not con- tribute to primary system level understanding of brain mechanisms such as how visual objects can be classified in brains. 2015). can be explained as compensation for adaptive behavior by means of the . LeCun et al. Another line of meaningful extension in terms of neuro-​phenomenological-​ robotics would be exploration of underlying mechanisms for various psychiatric diseases including schizophrenia.. Yamashita and Tani (2012) proposed that disturbance of self. 1998) developed as inspired by the hierarchical organization of the visual cortex can learn to classify visual images of hundreds object types such as bicycles.) Although this classification accuracy is almost close to that of human (Szegedy et  al. Building a large-​scale brain network model consisting of a dozen of major brain areas in its sub- networks by allocating around 10 million of rate-​coding neural units as the total may not be so difficult even in the current computational envi- ronment of using clusters. cars. referred to as Humanoid Brain project. chairs. It was shown that so-​called the convolutional neural network (CNN.0665 by using 1  million set of static visual image training data (Szegedy et  al. a surprising fact is that the used CNN consisting of 30 layers contains only around a million of almost homoge- neous rate-​coding neural units as opposed to the fact that the real visual cortex contains 10 billion of spiking neurons with hundreds of different morpho-​electrical types (see section 5. which is a major symptom in schizophrenia. gui- tars and so on in diverse views and sizes with error rate of 0.. Actually. which have already shown initial results.4. tables.. it has been recently shown that some connectionist type neural network models using several orders less number of rate-​coding neural units can exhibit human level performance in specific tasks such as visual object recognition.) This implies that activities of 10 thousands of spiking neurons could be represented by that of a single rate coding neural unit as a point mass in connectionist models without degrading their performance level as I presumed in section 5.256 256 Exploring Robotic Minds Interestingly. I and my colleagues have started studies in this direction. Recently. cognitive fragmentation (Perry & Braff.. a certain amount of perturbation was given in the connectivity weights between the higher level and the lower level to represent the disconnectivity. 2015.  257 Conclusions 257 error regression. After the training. This generalization error in predicting com- ing perceptual state could be considered as the main cause of autism from accumulated evidence on the patients’ typical symptom that they are significantly good at learning by rote but lacking capability in struc- tural learning (Van de Cruys et al. as I have analyzed in section 8. In future research. 1994). because subjective experience of time passing can be considered to be associated with prediction error in segmentation points in perceptual flow. 2014. the inner prediction error was generated because of the disconnectivity introduced. between prefrontal and poste- rior brain regions).e. In the neurorobotics experiment (Yamashita and Tani. the experimental result by Yamashita and Tani (2012) suggests a hypothetical account for a schizophrenia symptom. This observation suggests that aberrant modulatory signals induced by inter- nally generated prediction error might be a source of the patient’s feeling that his intention is affected by some outside force. 2014) proposed that hyper-​prior with less tolerance with the prediction error results in failure in generalization in learning which is the primary cause of autism. Furthermore. Van de Cruys and colleagues (Van de Cruys et al.. in which the patients lack conti- nuity in spatiotemporal perception. A neurorobotics model was built as inspired by the dis- connectivity hypothesis by Friston (1998) that suggests that basic pathol- ogy of schizophrenia may be associated with functional disconnectivity in the hierarchical network of the brain (i. This can be intuitively explained that the prediction net- work can generate overfitting problem with generalization error when the top-​down pressure for minimizing the error in learning is imposed on the network too strongly.) Robotic experiment for reconstructing the symptom could be conducted by mod- eling hyper-​prior by implementing estimation of inverse precision used .3.. a humanoid robot was trained for a set of behavioral tasks. It is speculated that such cognitive fragmentation might be caused by frequent occurrences of the inner prediction error. Nagai & Asada. the intention state in the higher level was modulated autonomously by the error signal back-​propagated from the lower perception level. When the robot performed the trained tasks with online error regression. the mechanism for autism could be clarified in terms of another type of malfunction in the predictive coding scheme presumed in the brain. 2012) using an MTRNN model. Consequently. . 2005. recent pragmatic studies of deep learning (Hinton et  al. because brain structures of these patients are known to be not so much different from the normal ones. 1998)  can perform visual object classification with near human level performance by learning (Szegedy et  al. For example. Future studies should examine other psychiatric diseases including attention deficit hyperactivity disorder and obsessive–​compulsive disorder. 2015)  as described previously in this sub- section. some deep learning schemes have recently demonstrated significant advances in perception and rec- ognition capabilities by using millions of exemplar datasets for learning. experienced through iterative interactions of the agents with the world. Such promising results seem to justify some optimism. 2013) have revived aging connectionist approaches supercharged with huge computational power latent within (multiple) graphic process- ing units in standard desktop PCs.. . 2006. The handwriting recognition system using long-​term short-​ term memory by Doetsch and colleagues (2014) demonstrated its almost human-​equivalent recognition performance.. Murata et  al.. and a speech recognition system provided a far better recogni- tion rate given noisy speech signals of unspecified speakers than widely used. Although this should be true for recognition of a single modality of perceptional channel. it should be able to account also for the underlying mechanisms for these common psychia- try pathologies. Although I’d say that the progress made in neurorobotics has thus far been steady. as Van de Cruys and colleagues (2014) rationalized that over esti- mation of the precision under noisy real world circumstance can result in overfitting of the prediction model. a convolutional neural network (LeCun et  al. state-​of-​the-​art commercial speech recognition systems (Hannun et al.258 258 Exploring Robotic Minds in Bayesian predictive coding framework (Friston. that the arti- ficial upscaling to human-​like cognitive capabilities using these meth- ods may not be so difficult. Confronted with this challenge. actually scaling robots to near-​ human level might be very difficult.. it is clear that deep understanding of the world on a human level cannot be achieved just by this. Another crucial question should be how much we can scale the neu- rorobots described in this book. 2015). Already. if a particular neurorobotics model represents a good model of the human mind. In summary. Bengio et al. as I know well that still my robots can work only in toy environments. Optimists may say that these systems can exhibit near human-​level perceptual capabilities. Such understanding should require associative integration among multiple modalities of perceptual flows. 2014).. deep minds near human level might appear as a consequence. induction. 2015. these and other recent advances in deep learning suggest that neurorobotics studies could be scaled significantly with the afore- mentioned large-​scale brain network model if massive training librar- ies are used alongside multimodal. Success in this endeavor should lead to a more general intelligence. because we cannot train robots simply by connecting them to the Internet or to a data- base.. One amazing aspect of human com- petency is that we can perform such a wide variety of tasks like navigat- ing.3. the crucial question becomes how to increase the amount of learning. So empowered. simply to name a few. and the developmental robotics commu- nity has already begun investigating this issue seriously (Kuniyoshi & Sangawa. 2006. imitation.. with online experience actively associated with its own intentional interaction with the world. of course.. Toward this end. Asada et al. When a robot becomes able to develop subjective. dancing.) However. This is not easy. like the one shown by Hwang et  al. what is crucially missing from current models is general intelligence by way of which various tasks across different domains can be completed by adaptively combining available cognitive resources through functions such as inference. high-​d imensional perceptual flow including the pixel level visual stream. Robots must act on the physical environment to acquire their own experiences. simulation. deliberating over mathematical equations. talking with others. designing intricate structures. Oudeyer et al. 2014. proactive self-​images in huge numbers of dimensions along- side its own unique “real-​ time” perceptual flow as it interacts with the world. Ugur et al. painting pictures. Metta et al.  259 Conclusions 259 Regardless. we may approach the reconstruction of real human minds! Attempts to scale neurorobots toward human-​ like being. 2010. olfactory “organs” and so on. cleaning rooms. among many others. improvisation. what our robots can do is merely navigate a given workspace or manipulate simple objects. 2015. Asada. So. researchers must provide a certain developmental . Cangelosi & Schlesinger. (2015) briefly described in section 9. however. and planning. inhibition of habitua- tion. and tactile sensation via hundreds of thousands of points of contact covering an entire “skin” surface. Compared with this. are scientifically fascinating. 2009. So. 2007. likewise for auditory signals. and searching the Internet for information on neurorobotics. working memory retrieval. taking our work one stage further logically involves educating robots to perform multiple domain tasks toward multiple goals with increasing degrees of complex- ity.. and therefore generalization with less amount of tutoring experience becomes possible. higher level functions are entrained alongside foundational perceptual abilities during tutoring. In the aforementioned developmental tutoring process. For this question developmental robotics could provide a possible solution by using the concept of staged development considered by Piaget (1951). 2010). devel- opmental stage would proceed from physically embodiment level to more symbolic level. Based on this conception.. In considering developmental education of robots. as I mentioned repeatedly. as Norm Chomsky (1972) once asked. research on the method for the tutor or educator side may become equally important. in robots with limited amount of tutoring experiences. Rather. This is asking how generalization in learning can be achieved. It could happen that the earlier stages may require merely sensory–​motor level interaction with environment physi- cally guided by tutors whereas the later stages may provide tutoring more in demonstration and imitation style without introducing physical guid- ance. still remained is that how human or artifacts like robots can acquire structural representation of the world by learning through experience under the constraints of “poverty of stimulus”. With scaffolding. . by reflecting own past seriously and also by act- ing proactively for own most possibility that is shared with the tutors. The expectation is that learning in one developmental stage can provide “prior” to the one in the next stage by which dimensionality of the learning can be drastically reduced. it should be an active learner that acts “creatively” for exploring the world. For this purpose. this environment is necessarily more complex than a long series of still photos. and the robot’s cognitive capacities develop from grounding simple sensory-​motor skills to more complex compositional cognitive ones. For implementation of such staged tutoring and development of robots. for example. a robot should not be a passive learner. The very final stage of education may require only usage of virtual environments (like learning from watching videos) or symbolically repre- sented materials (like reading books). And. Trials should require a lengthy period wherein phys- ical interactions between robots and tutors involve “scaffolding”—​guiding support provided by tutors that enables the bootstrapping of cognitive and social skills required in the next stage (Metta et  al. robots should become authentic beings. not merely repeating acquired skills or habits. an essential ques- tion.260 260 Exploring Robotic Minds educational environment wherein robots can be tutored every day for months or possibly for years. as robots must be educated within various task domains. nonliving machines for such long periods. and sympathy. Simultaneously. affectively reinforced education would become possible. Finally. the development of emotions within the robot would be an indispensable aid. he or she experiences happiness in bringing happiness to others similarly embodied. as one human seeks happiness for him-​or herself. human tutors would require emotional responses from the robots.  261 Conclusions 261 Tutoring interaction between such active learner robots and human tutors should inevitably become highly intensive occasionally. the long-​term educational processes of robots by human caregivers should be accompanied by these two codependent channels of development. . robots may start to have “free will.” as I  have postulated in this book. To carry out long-​ term and sometime intensive educational interactions. such robots would generate only good behaviors by inhibiting themselves to generate bad behaviors. Consequently. human beings are motivated to do something “good” for others because they share in the consequences of their actions by means of mirror neurons. to take care of robots like children. However. Although this issue has been neglected in this book. robots can do the same by learning the effects of their own actions on the happiness expressed by others and reinforced through mirroring neural models. a difficult but important problem to be considered is whether artifacts can embody and express moral virtue. Aristotle says that moral virtues are not innate. I would like to prove that robots can be developed or educated to acquire not only sophisticated cognitive competency but also moral virtue. This means that those robots could happen to generate bad behaviors toward others as well by own wills. by which long-​term. Otherwise. emotional empathy. The development of adequate emotional responses should deepen bonds between tutors and robots. if the robots can learn about moral virtue. It is said that an individual becomes truthful by acting truthfully or becomes unselfish by acting unselfishly. many human tutors may not be able to con- tinue cordial interactions with stone-​cold. Such robots would contribute to true happiness in a future human–​robot coexisting society. Nowadays. His crucial premise is that the development of emotion and that of embodied social interaction are codependent on each other. Minoru Asada proposed so-​ called affective developmental robotics (Asada. In principle. The net effect is that. cognitive empathy. 2015) in which he assumes multiple stages of emotional development from a simple stage to a com- plex one including emotional contagion. but they can be acquired through habitual prac- tice. embodiment by Merleau-​Ponty.” In Part I. this is to inquire into the essential. The book was organized into two parts. Part II started with new proposals for tackling open problems through neurorobotics experiments. and stream of consciousness by James. including cognitive science. To start with. Chapter 3 on phenomenology introduced views on the mind from the other extreme. phenomenology. being-​in-​the-​world by Heidegger. the book reviewed how different questions about minds have been explored in different research fields. Part I  started with an introduction to cognitivism. in ­ chapter  2 emphasizing “compositionality.262 262 Exploring Robotic Minds 11. we explored how philosophers have tackled the prob- lem of the inseparable complex that is the subjective mind and the objec- tive world. the potential difficulty in utilizing symbols internal to the mechanics of minds.” considered to be a uniquely human competency whereby knowledge of the world is represented by utilizing symbols.5. dynamical nature of the mind. In essence. Chapter 4 attempted to explain how human brains can support cogni- tive mechanisms through a review of current knowledge in the field of neuroscience. This book sought to account for the subjective experience character- ized on the one hand by compositionality of higher-​order cognition and on the other hand by fluid and spontaneous interaction with the outer world through the examination of synthetic neurorobotics experiments conducted by the author. brain science. however. Summary This final section overviews the whole book once again for the purpose of providing final conclusive remarks. It was also shown that notions of consciousness and free will may be clarified through phenomenological analysis. sensory-​motor reality and context. and synthetic modelling. We once again look at each chapter briefly to summarize them. especially in an attempt to ground symbols in real-​time. psychology. Some representative cognitive models were introduced that address the issues of problem solving in problem spaces and the abstrac- tion of information by using “chunking” and hierarchy. By emphasizing the cycle of perception and action in the physical world via embodiment. This chapter sug- gested. online. namely “Part I—On the Mind” and “Part II—Emergent Minds: Findings from Robotics Experiments. The chapter covered the ideas of subjective time by Husserl. emphasizing direct or pure experiences prior to being articulated with particular knowledge or symbols. we looked at a possible hierarchy in brains . new challenges dis- cussed in ­chapters 7 through 10 concerned the reconstruction of various cognitive or psychological behaviours in a set of synthetic neurorobot- ics experiments. our research focus went . First. top to bottom and bottom to top. It was furthermore speculated that a key to solving the so-​called hard problem of consciousness and free will could be found on close examination of such interactions. these chapters concluded that it is not yet possible to grasp com- plete understanding of the neuronal mechanisms accounting for cognitive functions of our interests due to conflicting evidence and the limitations inherent in experimental observation in neuroscience. It was speculated that human-​ like capabilities for dealing with compositional language-​thoughts or even for much simpler cognitive schemes should emerge as the results of iterative interactions between these two pathways. In these robotics studies. Chapter 5 introduced the dynamical systems approach for modeling embodied cognition both in natural and artificial systems. By following this tutorial. Chapter  6. the chapter described Gibsonian and Neo-​Gibsonian ideas in psychology that fit quite well with the dynamical systems frame- work and also explained how they have influenced the communities of behavior-​based robotics and neurorobotics. The chap- ter began with a tutorial on nonlinear dynamical systems. as the first chapter of Part II. proposed new paradigms for understanding cognitive minds by taking a synthetic approach uti- lizing neurorobotics experiments. the chapter postulated the potential difficulty in clarifying the essence of minds by just pursuing the bottom-​up pathway emphasized by the behaviour-​based approach. Then it was argued that what is missing are the top-​down subjective intentions for acting on the objective world and its iterative interaction with the bottom-​up perceptual reality. rather than just by one-​way processes along the bottom-​ up pathway. We then considered the possibility that two cognitive functions—​generating actions and recognizing perceptual reality—​are just two sides of the same coin by reviewing empirical studies on the mirror neurons and the pari- etal cortices.  263 Conclusions 263 that supports complex visual recognition and action generation. Based on the thoughts described in ­chapter  6. This chapter also examined the issue of the origin of free will by reviewing the experimental study conducted by Libet (1985). Despite the recent accumulation of various experimental findings in neurosci- ence. Some representative neuro- robotics studies were introduced investigating how primitive behaviors can develop and be explained from the dynamical systems perspective. perception. imitation game. The experimental results showed that the compositionality hidden in the topological trajectory in the obstacle environment can be extracted as embedded in a global attractor with fractal structure in the phase space of the RNN model. we also examined the codependent relationship between the subjective mind and the objective world that emerges in their dense interaction for the purpose of investigating the underlying structure of consciousness and free will. we investigated the development of compositionality by reviewing a robotics experiment on predictive nav- igation learning using a simple RNN model. By referring to Heidegger’s example about a carpenter hitting nails with a hammer. prediction. On the one hand. The RNNPB can learn a set of behavior primitives for generation as well as for recognition by means of error minimization in a predictive coding framework. on the other hand. and acting. as shown in the incoherent phase whereby the “self” rises to conscious awareness. In the second half of ­chapter 7.264 264 Exploring Robotic Minds back and forth between two fundamental issues. The RNNPB model was evaluated through a set of robotics experiments including learning of multiple movement patterns. It was shown that the develop- mental learning process during the exploration switched spontaneously between coherent phases and incoherent phases when chain reactions took place among different cognitive processes of recognition. and associative learn- ing of protolanguage and action whereby the following characteristics . we explored how compositionality for cognition can be developed via itera- tive sensory–​motor level interactions of agents with their environments and how these compositional representations can be grounded. It was shown that compositional representation developed in the RNN can be naturally grounded in the physical environment by allowing iter- ative interactions between the two in a shared metric space. On the other hand. In the first half of c­hapter  7. we explored a sense of groundlessness (a sense of not to be grounded completely) through the analysis of another navigation experiment. it was explained that the distinction between the two poles of the subjective mind and the objec- tive world become explicit in the breakdown. learning. We drew the con- clusion that the open dynamic structure characterized by self-​organized criticality (SOC) can account for the underlying structure of conscious- ness by way of which the “momentary self” appears spontaneously. Chapter 8 introduced the RNNPB as a model of mirror neurons that have been considered to be crucially responsible for the composition and decomposition of actions. More specifi- cally. Results showed that a set of behavior primitives were developed in the fast timescale network in the lower level. Furthermore. when a gap emerged between the top-​down intention for acting and the bottom-​up perception of reality. (1) The model can recognize aspects of a continuous percep- tual flow by segmenting it into a sequence of chunks or reusable primi- tives. It was concluded that a sort of “fluid compositionality” for smooth and flexible generation of actions was achieved in the proposed MTRNN model through the self-​organization of a functional hierarchy by adopting neuroscientifically plausible con- straints including timescale differences among different local networks and structural connectivity among them as downward causation. and (3) the model can generate not only learned behavior patterns but also novel ones by means of twists or dimples generated in the manifold of the RNNPB due to the potential nonlinearity of the network. we proposed a dynamic model. One involved its origin and the other the conscious awareness of it. It was also found that the initial neural activation state in the slow timescale network encoded the top-​down actional intention that triggers the generation of a correspond- ing slow dynamics trajectory in the higher level. which again triggers the projection of an intended sequence of behavior primitives from the lower level of the network to the outer world. I proposed that actional intention can be spontaneously generated by means of chaos in the higher cognitive brain areas. Chapter 9 addressed the issue of hierarchy in cognitive systems. For this purpose. the MTRNN that is character- ized by its multiple timescale and examined how a functional hierarchy for action can be developed in the model through robotics experiments employing this model. Chapter  10 also considered two problems about free will. while the whole action plan that sequences the behavior primitives was developed in the slow timescale network in the higher level. It was postulated that intention or will developed unconsciously in the higher cognitive brain by chaos would only come to conscious awareness in a postdictive manner. From the results of experiments employing the MTRNN model. as shown in the experiment on associative learning between protolanguage and actions. (2) a set of actional concepts can be learned with generalization by developing relational structures among those concepts in the neu- ral activation space. the chapter examined the circular causality developed among different cognitive processes in human–​robot interactive tutoring .  265 Conclusions 265 emerged. the intention may be noticed as the effort of minimizing this gap is exercised. Finally. However. provided they share the same metric space for interaction. aiming at the reduction of any apparent conflict between these two processing streams. The robot as well as human at such moments could be regarded as authentic beings. and the sensory–​motor peripheral areas that are assumed to be the neocortical target of the consolidative learning in human or mammals. Such a developmental process should take place in a large network consisting of the PFC. some concluding remarks are shown. the parietal cortex.266 266 Exploring Robotic Minds experiments. Image or knowledge can be developed through multiple stages of learning from an agent’s limited experiences—​ first stage: each instance of experience is acquired. third stage: novel or creative structures can be found in the memory developed with nonlinearity. The argument pre- sented here leads to: 1. or linguistic thoughts may develop by means of the self-​organization of neurodynamic structures through the aforementioned top-​ down and bottom-​up interactions. Structures and functions constituting mechanisms driving higher-​order cognition. second stage: generalized images or concepts are developed by extracting relational structures among the acquired instances in the memory. The mind should emerge via intricate interactions between the top-​down subjective view for proactively acting on the external world and the bottom-​up recognition of the perceptual reality. It was conjectured that free will could exist in the subjective experience of the human experimenter as well as the robot who seeks their own most possibility in their conflictive interaction when they feel as if whatever creative image for next act could pop out freely in their minds. It is presumed that such a compositional cognitive process embedded in neurodynamic attractors could be naturally grounded into the physical world. understood in the end as the inseparability of subjectivity and the objective . concepts. 4. 3. the most crucial aspect of minds is the sense of groundlessness that arises by circular causality. 2. such as for compositional manipulations of symbols. 5. developmental. The exploration of cognitive minds should continue with close dialogue between objective science and subjective experience (as suggested by Varela and others) for which synthetic approaches including cognitive. or neuronal robotics could contribute by providing effective research platforms. This understanding could shed light on the hard problem of consciousness and its relationship to the problem of free will through unification of theoretical studies on SOC of the holistic dynamics evolved and Heidegger’s thoughts on authenticity. .  267 Conclusions 267 world. 268 .   269 Glossary for Abbreviations BPTT back-​propagation through time CPG central pattern generator CTRNN continuous-​time recurrent neural network DOF degree of freedom EEG electroencephalography fMRI functional magnetic resonance imaging IPL inferior parietal lobe LGN lateral geniculate nucleus LIP lateral intraparietal area LSBN largescale brain network LSTM long-​term short-​term memory LRP lateralized readiness potential M1 primary motor cortex MIST medial superior temporal area MSTNN multiple spatiotemporal neural network MT middle temporal area MTRNN multiple timescale recurrent neural network PB parametric biases PC parietal cortex PCA principal component analysis PFC prefrontal cortex PMC premotor cortex PMv ventral premotor area RNN recurrent neural network 269 . 270 270 Glossary for Abbreviations RNNPB recurrent neural network with parametric biases RP readiness potential SMA supplementary motor area SOC self-​organized criticality STS superior temporal sulcus TEO inferior temporal area TPJ temporoparietal junction V1 primary visual cortex VIP ventral intraparietal area VP visuo-​proprioceptive . J. 5(01). Andry. & Toyoda. (2009). Arbib. Motor control (pp. Handbook of physiology: The nervous system. Part A: Systems and Humans. Learning and communication via imitation:  An autonomous robot perspective. A. Creating novel goal-╉ directed actions at criticality:  A  neuro-╉robotic experiment. IEEE Transactions on Systems. Sugano.. 271 . 55(1). Arbib. (2001). Brooks (Ed. Oxford:  Oxford University Press. Chaotic neural networks.. P. Amari. D. Man and Cybernetics. II. 1448–╉1480). R. (1990). P. In V. (1981). S. Mirror system activity for action and language is embed- ded in the integration of dorsal and ventral pathways. (2001). 299–╉307. Hicks. International Journal of Human-╉Computer Studies. J. Bethesda. S. K.. 112. Aristotle. M. A theory of adaptive pattern classifiers. P. T.. Arie. 333– ╉340. Gaussier. R.. (1983). J. MD: American Physiological Society. M. Cambridge. (2010). Endo. T.. Moga. Trans. New York: Oxford University Press.╇ 271 References Aihara. B. Arbib. & Riedl. A perception/╉action substrate for cog- nitive modeling in HCI. M. Arakaki. Anderson... (1907). 3. 15–╉39. H. & Tani. Physics Letters A. The architecture of cognition. 144. 431–╉4 42. St Amant. MA: Harvard University Press. Brain & Language. Banquet. De anima (R. IEEE Transactions on Electronic Computers. Takabe. (2012). How the brain got language:  The mirror system hypothesis. (1967). M. T.. M.). S. Perceptual structures and distributed motor control. O.. J.). New Mathematics and Natural Computation.. & Nadel. 12–╉24. 307–╉334. 31(5). Asada. 10. A. & Sirigu. Consciousness and the prefrontal parietal net- work: Insights from attention. P. IEEE Transactions on Cognitive and Developmental Systems. (2009). Balslev. (2008). M. A. (2000). 12–​34.. D. & Seth. C. Frontiers in Psychology. Badre. P. IEEE Transactions on Autonomous Mental Development. 659–​6 69. & Vincent. 3. A. S. International Journal of Social Robotics. R. Beer. Thought and reference. 7(1).. Ogino.. Artificial Intelligence. Bach. Oxford: Oxford University Press. 15(5). and chunking. 1798–​1828. Nielsen. 91–​9 9. Y. Beer. K. M. (2003). (1987). 239–​245.. Ishiguro. D. Billard. K. M. 63. A. (2005). Learning motor skills by imitation: A biologically inspired robotic model. D’Esposito. J.. & Gazzaniga.. 200–​209. Right tempo- roparietal cortex activation during visuo-​proprioceptive conflict. Understanding complexity in the human brain. (2012). Vehicles:  Experiments in synthetic psychology. 72(1). Bassett. Towards artificial empathy. D. Cerebral Cortex. (2009). (1995b). New York: Oxford University Press. M.. V.272 272 References Arnold. 381–​384. T. (2000). Y.. K. O. Blakemore. 73–​215. A. Berlin: Springer. Self-​organized criticality:  An explanation of the 1/​f noise. Baldwin. Hosoda K. (1987). IEEE Transactions on Pattern Analysis and Machine Intelligence. Braitenberg. Beer. Paulson. Cybernetics and Systems. On the dynamics of small continuous-​time recurrent neural networks. (2011). 35(8). Segmenting dynamic human action via statistical structure. Experimental Brain Research. Bak.. How can artificial empathy fol- low the developmental pathway of natural empathy?. & Wiesenfeld. 166–​169. H.. and Asada. Action prediction in the cerebel- lum and in the parietal cortex. Andersson. (in press).. Dynamical approaches to cognitive science. (1995a). A dynamical systems perspective on agent-​environment interaction. & Law. Nagai. Yoshikawa. R. working memory. Emergence of altruistic behav- ior through the minimization of prediction error.. M. 15(2). Courville. 1382–​1407. & Meyer. & Yoshida. D. J. Saffran. Baraglia. Y. L. Cam­ bridge. M.. Trends in cognitive sciences.. I. Inui. A. (1995). 3(4). D. 106. MA: MIT Press. 59. Bengio. 1(1). (1984). F. (2015). Y.. M. C.. J. Bor.. Adaptive Behavior. 155–​193. 153(2). Physical Review Letters.. 19–​33. 32. Representation learning: A review and new perspectives. Trends in Cognitive Sciences. Bach. R. S-​J. (2013). Kuniyoshi.. D. Random dynamical systems. Tang. B. Principles of synthetic intelligence: Building blocks for an archi- tecture of motivated cognition. 4(3).. S. Asada. . D. Is the rostro-​caudal axis of the frontal lobe hierarchical? Nature Reviews Neuroscience. (2008). 471–​511... D. Cognitive developmental robotics: A sur- vey. Cognition..  7–​19. K... A.. (1998). Clark. M. Snyder. W. T.. & Schlesinger. K. Inferring statistical complexity. J. 369–​378. Macmillan Education UK. Journal of Consciousness Studies. (1972). M. M. D.. M. Ryu. Churchland.. M. Robotics and Autonomous Systems. Clark. Rules and representations. S. Cunningham. A. 58(1). Nature. Cohen. (1989). Being there:  Putting brain. Neural Computation... Campbell. (1989). Lisberger. A. Artificial Intelligence Journal.. H. 200–​219. Hosseini. “Downward causation” in hierarchically organized biological systems. arXiv.. Armstrong.. Intelligence without representation. J.. J. A. D. MA: MIT Press. S. I. D. 487.. 372–​381. T.. I. The extended mind. A.. Developmental robotics from babies to robots. Ryu. J. M. I. M. Analysis. D. Oxford: Basil Blackwell. Chang. Moore. 179–​186)... 73–​110. 2(3). S. T. R.. Journal of Experimental & Theoretical Artificial Intelligence. Predictive coding for dynamic vision : Development of functional hierarchy in a multiple spatio-temporal scales RNN model. N. Clark. P. Facing up to the problem of consciousness. Structure of neural popula- tion dynamics during reaching. M. Priebe. Corrado. action. J. 1. Nuyujukian. 13(3).. 139–​159. M. B. Choi.. B.. N. Kaufman. K. Newsome. Scott. Bradley. L. Clark. P. C.. 105–​108. P. & Young. (2015). Sugrue. (2010). Brooks. P.. Cambridge. R. (1974). & Harvey. 51–​56. Clark. Kohn. J. Stimulus onset quenches neural variability: a widespread cortical phenomenon.org preprint arXiv:1606. Dale. J. & Shenoy. & McClelland.. Ferster. A.01672 Chomsky. V. Adaptive Behavior. (1999). (1990). Servan-​Schreiber. P. S. Churchland. Language and mind. M. J. Santhanam. Cambridge. A. M. . (1980).. 47. W. L. J. R. Finn. Cliff. Crutchfield. D. (1995). A.. (2015). Cunningham. (2005). & Chalmers.. Chalmers. (1993). Nature Neuroscience. Cangelosi. D. N. Elephants don’t play chess. I. M. P.  B. Explorations in evolutionary robotics.. 6. An embodied cognitive science?. New York: Oxford University Press.. Yu.. & Tani. G. (2016). P. Surfing uncertainty:  Prediction.. K.. R. Foster. & Spivey. Cleeremans. G. L. M. New  York:  Harcourt Brace Jovanovich. 317–​342. Sahani. Smith.. From apples and oranges to symbolic dynam- ics:  A  framework for conciliating notions of cognitive representation. MA: MIT Press. G. A. Trends in Cognitive Sciences. Husbands. 345–​351. (2012). and the embodied mind. S. Chomsky.. (1991). Finite state automata and simple recurrent networks.. 63.. In Studies in the Philosophy of Biology (pp. 2(1). A.  273 References 273 Brooks. Physical Review Letters. V. A. A. (1998). J. 17(4).. D. 3–​15. D. & Shenoy. and world together again. body. T. M. Movshon. M. 3(9). 492– ​498. Di Paolo. H. (1988). P.. 429.. J.. 117(1). The embodied mind. T.D. M. (2002). 6). & Hayes. Mikulis. Neural basis of rhythmic behavior in animals. R. In S. 27–​48.). 327–​361). S. 149–​160. I. Reilly. 106. E. Adaptive Behavior. 121–​126. (2000). structural congruence and entrainment in a simulation of acoustically coupled agents.. 13(2). N. Richard.. In IEEE 14th International Conference on Frontiers in Handwriting Recognition (ICFHR) (pp. (1991). (1991). Forward modeling allows feedback control for fast reaching movements. & Grafton. Dautenhahn & C. Doya.). Kozielski. & Poo. D. Mottolese. Science. Rapid BDNF-​induced retrograde synaptic modifica- tion in a developing retinotectal system. Trends in Cognitive Sciences. K. L. Dreyfus. A. Fast and robust training of recur- rent neural networks for offline handwriting recognition.. Nature.. Memorizing oscillatory patterns in the ana- log neuron network. Dennett. 279–​284). (1989). (2014). (2000). MA: Addison-​Wesley. 878–​8 83. Downar. M. 277–​283. H. Proceedings of the 1989 International Joint Conference on Neural Networks. (2009). 4(11). Reading. & Yoshizawa.). 210. Y. Crawley. Cambridge. 15– ​43. Devaney. Imitation in animals and artifacts (pp. 8(1). & Davis. L. Movement intention after parietal cortex stimulation in humans. Diamond. 3(3).. An introduction to chaotic dynamical systems (Vol. Rosch (Eds. 423–​431. Adaptive Behavior. The epigenesis of mind: Essays on biology and knowledge (pp. (2005). S. P. D. Daedalus. K. E. Cambridge. Thompson and E.. American Journal of Psychology. J. & Uchibe.. . R. In K. E. Du. 811–​813. F. (1989). Imitation as a dual-​route process featur- ing predictive and learning components: A biologically plausible compu- tational model. M. Hillsdale. 67–​110). Being-​in-​the-​world: A commentary on Heidegger’s Being and Time. (1993). Gelman (Eds. M. NJ: Erlbaum. & Dreyfus.. Doya. & Ney. Carey.J.. (1980). Desmurget. Desmurget. (2000). Dreyfus. A. A. E. G. Demiris. A. A. MA: MIT Press. Science. Review of F. K.. & Sirigu. Nehaniv (Eds.. Doetsch. A multi- modal cortical network for the detection of changes in the sensory environ- ment. Varela. 324. K. Behavioral coordination. MA: MIT Press. (2004). H. 27–​32. L. C.L. Szathmari.. Neuropsychological insights into the meaning of object concept development. Making a mind versus model- ing the brain:  artificial intelligence back at a branch point. Nature Neuroscience.274 274 References Delcomyn. The cyber rodent project:  Exploration of adaptive mechanisms for self-​preservation and self-​reproduction. S.. Propagation of activity depend- ent synaptic depression in simple neural networks. Oxford: Clarendon Press. & Assad. G. 439– ​4 48. Fleischer.. R. Z. T. Inc. Science. McCarthy. (2007). Proceedings of the National Academy of Sciences of the USA. (1987)... J. P. (2008). & Spencer. 2036–​2038. 14.. Cognition. R.. Finding structure in time.. I. A theory of cortical responses. J. 3556–​3561. 213–​228. 2978–​2986. H. (2000).. J. & Poo. How to build a brain: A neural architecture for biological cognition. 308. Chersi. Fried. B. (1988). Nature Neuroscience. Elman. S. A. P. Fogassi. (1990).. E. (2005). 662– ​6 67. Katz. G. J. G. Cognitive Science. Retrospective and prospective responses arising in a modeled hippocampus during maze navigation by a brain-​based device. The varieties of reference. Journal of Neuroscience. A. Dissociation of visual. 360(1456). & Forssberg. Motion integration and postdic- tion in visual awareness. 115–​125. 7(2– ​3). and grammatical structure. Edelman. (1991). G.. Fitzsimonds. (1999). Eskandar. M. S. Nakanishi. & Krichmar. Matsubara. 90.  275 References 275 Eagleman.. K. Neural Darwinism: The theory of neuronal group selec- tion. 388. D. & Pylyshyn. D. Friston. L. Distributed representations. J. 3–​71. Friston.. W. New York: Basic Books.. 104(9). Evans.. The disconnection hypothesis.. Journal of Neurophysiology. New York: Oxford University Press. J. 815–​836. Gally. D. L. Fodor. & Sejnowski. M. Philosophical transactions of the Royal Society B: Biological Sciences. C. 11. K. Eliasmith. Song. (2003). 3656– ​3666. & Rizzolatti. 28. (1997). motor and predic- tive signals in parietal cortex during visual guidance. Ferrari. Johansson. 2. Sass. J.. Learning CPG-​based biped locomotion with a policy gradient method:  Application to a humanoid robot. Parietal lobe: from action organization to intention understanding. Connectionism and cognitive architec- ture: A critique. Fagergren.. How brains make up their minds? New York: Columbia University Press. F. J. simple recurrent networks. K. (2014)..... Science. J. Nature. Ehrsson. 179–​211. Evidence for the involvement of the posterior parietal cortex in coordination of fin- gertip forces for grasp stability in manipulation. G. Freeman. (1982). 30(2). J. Edelman. (2005). & Cheng.. T.. (1991). M. 287(5460). Elman. J. L. Gesierich. 88–​93. 27(2). Functional organization of human supplementary motor cortex studied by electrical stimulation. Endo. J.. Rozzi. Williamson. H. G. Morimoto. H. Spencer.. 195–​225. Machine Learning.. Schizophrenia Research. (2000). S. (1998). The International Journal of Robotics Research. . The free-​energy principle: A unified brain theory? Nature Reviews Neuroscience. Moga.. (1990). Complex movements evoked by microstimulation of precentral cortex. Harnad. S. Prenger... (2007). Artificial life with autonomously emerging boundaries. J. J. Gibson. & Ng. A. Case. (2000). 14–​21. Casper. P.org preprint arXiv:1410.276 276 References Friston. Yamaguti. (2002). E. Y. Haggard. An ecological approach to perceptual learn- ing and development. (2000).. S. & Pick. Gibson. Goodale.. Human volition: towards a neuroscience of will... Journal of Cognition and Development. S. Milner. G. G. 2.. A. & Moore. E.. A. The ecological approach to visual perception. & Frith. M. C. A. I.. K.. Georgopoulos. (2012). & Kuroda. Physica D.. 127–​138. 42. Nature Reviews Neuroscience. Trends in Cognitive Sciences.. Elsen. (1991). (1983)... Boston: Houghton Mifflin. A. (1982).. 305–​316. Neuron. Graves. & Massey. Advanced synergetics. 2.5567. Gunji.. (1998). arXiv. A.. . (2004). (2014). Taylor. 154–​1546. Frith. B. S. Haken. Wayne.. I. 335–​346. L. Mirror neurons and the simulation theory of mind-​reading. M. N. Applied Mathematics and Computation. S. V. A. C. C. (2010). From perception-​ action loops to imitation processes: A bottom-​up approach of learning by imitation. H. (2014). Coates. Berlin: Springer. T. Gallese. & Banquet. Y.. P. (1991).org pre- print arXiv:1412. Quoy.. 5. 841–​851. 34. & Carey. Annual Review of Psychology. Spatial clustering property and its self-​ similarity in membrane poten- tials of hippocampal CA1 pyramidal neurons for a spatio-​temporal input sequence.5401. M. D. J.. J. Kalaska. 349(6305). Gershkoff-​ Stowe. P.. 63.D. D. Neural turing machines. A. M. P. F. Y. On the relations between the direction of two-​dimensional arm movements and cell discharge in primate motor cortex. Philosophical conceptions of the self: Implications for cognitive science. Jakobson. Y. Hannun. & Danihelka. Nature. 12(7-​8). Tsuda. A. & Goldman. 934–​946.J. The symbol grounding problem. U. Mechanisms of social cognition.. 287–​313.. 271–​298. 11. (2008). P. J. Applied Artificial Intelligence. Graziano. Diamos. & Konno. 701–​727. S. 43.. Tsukada. 1. DeepSpeech:  Scaling up end-​to-​end speech recognition. Gaussier. Catanzaro. & Thelen. 1527–​1537.. A neurological dissociation between perceiving objects and grasping them. 11–​36. New York: Oxford University Press. 493–​501. D. U-​ shaped changes in behav- ior: A dynamic systems perspective. R. L. 9(12). Sengupta. Fukushima.. R. The Journal of Neuroscience. (1986). arXiv. Trends in Cognitive Sciences. Cognitive Neurodynamics. E. Satheesh. T. J.. Caminiti. (1998). Gallagher. 4(1). S.   277 References 277 Harnad, S. (1992). Connecting object to symbol in modeling cognition. In A. Clarke, & R. Lutz (Eds.), Connectionism in context. Berlin: Springer Verlag. Haruno, M., Wolpert, D. M., & Kawato, M. (2003). Hierarchical MOSAIC for movement generation. In International congress series (Vol. 1250, pp. 575–​590). Amsterdam: Elsevier. Harris K. (2008). Stability of the fittest:  Organizing learning through ret- roaxonal signals. Trends in Neurosciences, 31(3), 130–​136. Hasson, U., Yang, E., Vallines, I., Heeger, D. J., & Rubin, N. (2008). A hier- archy of temporal receptive windows in human cortex. The Journal of Neuroscience, 28(10), 2539–​2550. Hauk, O., Johnsrude, I., & Pulvermuller, F. (2004). Somatotopic representa- tion of action words in human motor and premotor cortex. Neuron, 41(2), 301–​307. Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002). The faculty of lan- guage: What is it, who has it, and how did it evolve? Science, 298(5598), 1569–​1579. Heidegger, M. (1962). Being and time (J. Macquarrie, & E. Robinson, Trans.). London: SCM Press. Molesworth, W. (1841). The English works of Thomas Hobbes (Vol. 5). J. Bohn, 1841. Hinton, G., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527–​1554. Hochreiter, S., & Schmidhuber, J. (1997). Long short-​term memory. Neural Computation, 9(8), 1735–​1780. Husserl, E. (1964). The phenomenology of internal time consciousness (J. S. Churchill, Trans.). Bloomington, IN: Indiana University Press. Husserl, E. (1970). Logical investigations (Vol. 1). London: Routledge & Kegan Paul Ltd. Husserl, E. (2002). Studien zur arithmetik und geometrie. New  York: Springer-​Verlag. Hyvarinen, J., & Poranen, A. (1974). Function of the parietal associative area 7 as revealed from cellular discharges in alert monkeys. Brain, 97, 673–​6 92. Hwang, J., Jung, M., Madapana, N., Kim, J., Choi, M., & Tani, J. (2015). Achieving “synergy” in cognitive behavior of humanoids via deep learning of dynamic visuo-​motor-​attentional coordination. In Proceeding of 2015 IEEE-​ RAS 15th International Conference on Humanoid Robots (pp. 817–​824). Iacoboni, M., Woods, R. P., Brass, M., Bekkering, H., Mazziotta, J. C. & Rizzolatti, G. (1999). Cortical mechanisms of imitation. Science, 286, 2526–​2528. Ijspeert, A. J. (2001). A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander. Biological Cybernetics, 84, 331–​348. 278 278 References Ikeda, K., Otsuka, K. & Matsumoto, K. (1989). Maxwell-​Bloch turbulence. Progress of Theoretical Physics, 99, 295–​324. Ikegami, T. & Iizuka, H. (2007). Turn-​taking interaction as a cooperative and co-​creative process. Infant Behavior and Development, 30(2), 278–​288. Ikegami, T. (2013). A design for living technology:  Experiments with the mind time machine. Artificial Life, 19(3– ​4), 387– ​400. Ikegaya, Y., Aaron, G., Cossart, R., Aronov, D., Lampl, I., et  al. (2004). Synfire chains and cortical songs:  Temporal modules of cortical activity. Science, 304, 559–​564. Iriki, A., Tanaka, M., & Iwamura, Y. (1996). Coding of modified body schema during tool use by macaque postcentral neurones. Neuroreport, 7(14), 2325–​2330. Ito, M. (1970). Neurophysiological basis of the cerebellar motor control sys- tem. International Journal of Neurology, 7, 162–​176. Ito, M. (2005). Bases and implications of learning in the cerebellum—​adaptive control and internal model mechanism. Progress in Brain Research, 148, 95–​109. Ito, M., & Tani, J. (2004). On-​line imitative interaction with a humanoid robot using a dynamic neural network model of a mirror system. Adaptive Behavior, 12(2), 93–​115. Jaeger, H., & Haas, H. (2004). Harnessing nonlinearity:  Predicting cha- otic systems and saving energy in wireless telecommunication. Science, 308, 78–​8 0. Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U. (2007). Optimization and applications of echo state networks with leaky-​integrator neurons. Neural Networks, 20(3), 335–​352. James, W. (1884). The dilemma of determinism. Unitarian Review (Vol. XXII, p.193). Reprinted (1956) in The will to believe (p.145). Mineola, NY: Dover Publications, p.145. James, W. (1892). The stream of consciousness. World: Cleveland, OH. James, W. (1918). The principles of psychology (Vol 1). New  York, NY: Henry Holt. Jeannerod, M. (1994). The representing brain:  Neural correlates of motor intention and imagery. Behavioral and Brain Sciences, 17, 187–​202. Johnson-​P ynn, J., Fragaszy, D. M., Hirsh, E. M., Brakke, K. E., & Greenfield, P. M. (1999). Strategies used to combine seriated cups by chimpanzees (Pan troglodytes), bonobos (Pan paniscus), and capuchins (Cebus apella). Journal of Comparative Psychology, 113(2), 137–​148. Jordan, M. I. (1986). Attractor dynamics and parallelism in a connectionist sequential machine. In Proceedimgs of Eighth Annual Conference of Cognitive Science Society (pp. 531–​546). Hillsdale, NJ: Erlbaum.   279 References 279 Jung, M., Hwang, J., & Tani, J. (2015). Self-​organization of spatio-​temporal hierarchy via learning of dynamic visual image patterns on action sequences. PLoS One, 10(7), e0131214. Kaneko, K. (1990). Clustering, coding, switching, hierarchical ordering and control in a network of chaotic elements. Physica D, 41, 137–​72. Karmiloff-​Smith, A. (1992). Beyond modularity: A developmental perspective on cognitive science. Cambridge, MA: MIT Press. Kawato, M. (1990). Computational schemes and neural network models for formation and control of multijoint arm trajectory. In T. Miller, R. S. Sutton, & P. J. Werbos (Eds.), Neural networks for control (pp. 197–​228). Cambridge, MA: MIT Press. Kelso, S. (1995). Dynamic patterns: The self-​organization of brain and behavior. Cambridge, MA: MIT Press. Kiebel, S., Daunizeau, J., & Friston, K. (2008). A hierarchy of time-​scales and the brain. PLoS Computational Biology, 4, e1000209. Kimura, H., Akiyama, S., & Sakurama, K. (1999). Realization of dynamic walking and running of the quadruped using neural oscillator. Autonomous Robots, 7(3), 247–​258. Kirkham, N., Slemmer, J., & Johnson, S. (2002). Visual statistical learning in infancy: Evidence for a domain general learning mechanism. Cognition, 83, B35–​B 42. Klahr, D., Chase, W. G., & Lovelace, E. A. (1983). Structure and process in alphabetic retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition, 9(3), 462. Kolen, J. F. (1994). Exploring computational complexity of recurrent neural net- works. (PhD thesis, The Ohio State University). Kourtzi, Z., Tolias, A. S., Altmann, C. F., Augath, M., & Logothetis, N. K. (2003). Integration of local features into global shapes: monkey and human fMRI studies. Neuron, 37(2), 333– ​346. Koza, J. R. (1992). Genetic programming: On the programming of computers by means of natural selection. Cambridge, MA: MIT Press. Krichmar, J. L. & Edelman, G. M. (2002). Machine psychology: autonomous behavior, perceptual categorization and conditioning in a brain-​ based device, Cerebral Cortex, 12, 818–​830. Kuniyoshi, Y., Inaba, M. and Inoue, H. (1994). Learning by watch- ing: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics Automation, 10, 799–​822. Kuniyoshi, Y., Ohmura, Y., Terada, K., Nagakubo, A., Eitoku, S. I., & Yamamoto, T. (2004). Embodied basis of invariant features in execution and perception of whole-​body dynamic actions—​k nacks and focuses of Roll-​a nd-​R ise motion. Robotics and Autonomous Systems, 48(4), 189–​201. 280 280 References Kuniyoshi. Y., & Sangawa, S. (2006). Early motor development from par- tially ordered neural-​body dynamics—​experiments with a cortico-​spinal-​ musculo-​skeletal model. Biological Cybernetics, 95, 589–​6 05. Laird, J. E., Newell, A., & Rosenbloom, P. S. (1987). Soar: An architecture for general intelligence. Artificial Intelligence, 33, 1–​6 4. Laird, J. E. (2008). Extending the Soar cognitive architecture. Frontiers in Artificial Intelligence and Applications, 171, 224. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-​ based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–​2324. Li, W., Piëch, V., & Gilbert, C. D. (2006). Contour saliency in primary visual cortex. Neuron, 50(6), 951–​9 62. Libet, B. (1985). Unconscious cerebral initiative and the role of conscious will in voluntary action. Behavioral and Brain Sciences, 8, 529–​539. Lu, X., & Ashe, J. (2005). Anticipatory activity in primary motor cortex codes memorized movement sequences. Neuron, 45, 967–​973. Luria, A. (1973). The working brain. London: Penguin Books Ltd. McCarthy, J. (1963). Situations, actions and causal laws. Stanford Artificial Intelligence Project, Memo 2. Stanford University. Markov, A. (1971). Extension of the limit theorems of probability theory to a sum of variables connected in a chain. Dynamic Probabilistic Systems, 1, 552–​577. Markram, H., Muller, E., Ramaswamy, S., Reimann, M. W., Abdellah, M., Sanchez, C. A., … & Kahou, G. A. A. (2015). Reconstruction and simula- tion of neocortical microcircuitry. Cell, 163(2), 456–​492. Matarić, M. (1992). Integration of representation into goal-​d riven behavior-​ based robots. IEEE Transactions on Robotics and Automation, 8(3), 304–​312. Matsuno, K. (1989). Physical Basis of Biology. Boca Raton, FL: CRC Press. Maturana, H. R., & Varela, F. J. (1980). Autopoiesis and Cognition. Netherlands: Springer. May, R. M. (1976). Simple mathematical models with very complicated dynamics. Nature, 261(5560), 459–​467. Meeden L. (1996). An incremental approach to developing intelligent neural network controllers for robots. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 26(3), 474–​485. Merleau-​Ponty, M. (1962). Phenomenology of perception (C. Smith, Trans.), London: Routledge & Kegan Paul Ltd. Merleau-​Ponty, M. (1968). The Visible and the invisible: Followed by working notes (Studies in phenomenology and existential philosophy). Evanston, IL: Northwestern University Press. Meltzoff, A. N., & Moore, M. K. (1977). Imitation of facial and manual ges- tures by human neonates. Science, 198(4312), 75–​78.   281 References 281 Meltzoff, A.N. (2005). “Imitation and other minds: The ‘like me’ hypothe- sis.” In S. Hurley and N. Chater (Eds.), Perspectives on imitation: From cogni- tive neuroscience to social science (pp. 55–​77). Cambridge, MA: MIT Press. Metta, G., Natale, L., Nori, F., Sandini, G., Vernon, D., Fadiga, L., et  al. (2010). The iCub humanoid robot: An open-​systems platform for research in cognitive development. Neural Networks, 23(8–​9), 1125–​1134. Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81. Morimoto, J., & Doya, K. (2001). Acquisition of stand-​up behavior by a real robot using hierarchical reinforcement learning. Robotics and Autonomous Systems, 36(1), 37–​51. Mormann, F., Kornblith, S., Quiroga, R. Q., Kraskov, A., Cerf, M., Fried, I., & Koch, C. (2008). Latency and selectivity of single neurons indicate hierarchical processing in the human medial temporal lobe. Journal of Neuroscience, 28, 8865– ​8 872. Mulliken, G. H., Musallam, S., & Andersen, R. A., (2008). Forward esti- mation of movement state in posterior parietal cortex. Proceedings of the National Academy of Sciences of the USA, 105(24), 8170–​8177. Murata, S., Yamashita, Y., Arie, H., Ogata, T., Sugano, S., & Tani, J. (2015). Learning to perceive the world as probabilistic or deterministic via interac- tion with others: A neuro-​robotics experiment. IEEE Transactions on neu- ral Networks and Learning Systems, [2015 Nov 18; epub ahead of print], DOI: 10.1109/​T NNLS.2015.2492140 Mushiake, H., Inase, M., & Tanji, J. (1991). Neuronal activity in the pri- mate premotor, supplementary, and precentral motor cortex during visu- ally guided and internally determined sequential movements. Journal of Neurophysiology, 66(3), 705–​718. Nadel, J. (2002). Imitation and imitation recognition: Functional use in pre- verbal infants and nonverbal children with autism. In A. N. Meltzoff, & W. Prinz (Eds.), The imitative mind: Development, evolution, and brain bases (pp. 42–​62). Cambridge University Press. Nagai, Y., & Asada, M. (2015). Predictive learning of sensorimotor informa- tion as a key for cognitive development. In Proceedings of the IROS 2015 Workshop on Sensorimotor Contingencies for Robotics. Osaka, Japan. Namikawa, J., Nishimoto, R., & Tani, J. (2011). A neurodynamic account of spontaneous behavior. PLoS Computational Biology, 7(10), e1002221. Newell, A., & Simon, H. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-​Hall. Newell, A., & Simon, H. A. (1975). Computer science as empirical inquiry: Symbols and search. Communications of the ACM, 19(3), 113–​126. Newell, A. (1990). Unified theories of cognition. Cambridge, MA:  Harvard University Press. 282 282 References Nicolis, G., & Prigogine, I. (1977). Self-​organization in nonequilibrium systems. New York: Wiley. Nishida, K. (1990). An inquiry into the good (M. Abe & C. Ives, Trans.). New Haven: Yale University Press. Nishimoto, R., & Tani, J. (2009). Development of hierarchical structures for actions and motor imagery: A constructivist view from synthetic neuroro- botics study. Psychological Research, 73, 545–​558. Nolfi, S. & Floreano, D. (2000). Evolutionary robotics: The biology, intelligence, and technology of self-​organizing machines. Cambridge, MA: MIT Press. Nolfi, S., & Floreano, D. (2002). Synthesis of autonomous robots through artificial evolution. Trends in Cognitive Sciences, 6(1), 31–​37. Ogai, Y., & Ikegami, T. (2008). Microslip as a simulated artificial mind. Adaptive Behavior, 16(2–​3), 129–​147. Ogata, T., Hattori, Y., Kozima, H., Komatani, K., & Okuno, H. G. (2006). Generation of robot motions from environmental sounds using intermo- dality mapping by RNNPB. In Sixth International Workshop on Epigenetic Robotics, Paris, France. Ogata, T., Yokoya, R., Tani, J., Komatani, K., & Okuno, H. G. (2009). Prediction and imitation of other’s motions by reusing own forward-​inverse model in robots. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation (pp. 4144–​4149). Kobe, Japan. O’Regan, J. K., & Noe, A. (2001). A sensorimotor account of vision and visual consciousness. Behavioral & Brain Sciences, 24, 939–​1031. Oudeyer, P. Y., Kaplan, F., & Hafner, V. V. (2007). Intrinsic motivation sys- tems for autonomous mental development. Evolutionary Computation, IEEE Transactions on, 11(2), 265–​286. Oztop, E., Kawato, M., & Arbib, M. (2006). Mirror neurons and imita- tion: A computationally guided review. Neural Networks, 19(3), 254–​271. Paine, R. W., & Tani, J. (2005). How hierarchical control self-​organizes in artificial adaptive systems. Adaptive Behavior, 13(3), 211–​225. Park, G., & Tani, J. (2015). Development of compositional and contextual communicable congruence in robots by using dynamic neural network models. Neural Networks, 72, 109–​122. Pepperberg, I. M., & Shive, H. R. (2001). Simultaneous development of vocal and physical object combinations by a Grey parrot (Psittacus eritha- cus): Bottle caps, lids, and labels. Journal of Comparative Psychology, 115(4), 376–​384. Perry, W., & Braff, D. L. (1994). Information-​processing deficits and thought disorder. American Journal of Psychiatry, 15(1), 363– ​367. Pfeifer, R & Bongard, J. (2006). How the body shapes the way we think—​A new view of intelligence. Cambridge, MA: MIT Press. Piaget, J. (1951). The child's conception of the world. Rowman & Littlefield. F. R. G. Cognitive Brain Research. 233–​242.).. (1999).. . An equation for continuous chaos. S. L. & Blakeslee. (1995). G. (1986). Aslin. Ritter. V. 1926–​1928. H. Rumelhart. Schaal. Sakata.. (2004)... Murata.. The induction of dynamical recognizers. R. Pfeifer. D. G. L. 274. E. Y. Supporting cog- nitive models as users. 2.. D. Rumelhart. & F. 429–​438. Infants’ emerging ability to represent object motion. Learning internal representations by error propagation. & Ballard. 3. E.  283 References 283 Piaget. Kuniyoshi.. M.. Machine Learning. Hinton.).. Annual Review of Neuroscience. (1991).. R. 57A(5).. Embedded neural net- works: Exploiting constraints. Physics Letters. Rizzolatti. (2001). 131–​141. Gallese. Cambridge.. 661–​670. O. 1–​22. Pulvermuller. R. E. G. (1986). 11. E. D. J. J. 79–​87. Nature Neuroscience. & Mine. L. Rizzolatti. Statistical learning by 8-​month-​ old infants. (2004). MA: MIT Press. ACM Transactions on Computer-​Human Interaction. Pollack. Neural Networks.. Rumelhart. J. & Craighero. Fadiga. Mclelland (Eds. & von Hofsten. Cerebral Cortex. Play. (1998).. Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences. 7.. 169–​192. Jones. G. Fogassi.. & Young. Taira. 227–​252. C. Cognition. Brain mechanisms linking language and action. Rosander. 27. 5(5). Phantoms in the brain: Probing the mysteries of the human mind. Neural mechanisms of visual guidance of hand action in the parietal cortex of the monkey. Nature Neuroscience. 2. Baxter.. MA: MIT Press. D. 141–​173. Hodgson. 91. (1962). Nature Review Neuroscience. E. Saffran. J. L. 76–​82. Gattegno. J. Neurophysiological mecha- nisms underlying the understanding and imitation of action. 397– ​398. E. Rizzolatti. S. Cambridge. Rao. The mirror-​neuron system. Parallel distributed processing:  Explorations in the microstructure of cognition. F. A.. & Fogassi. G. New York: William Morrow. New York: Norton. M. V. McClelland. (1996). & J. Scheier. (2000). & Williams. Trans. (1976). and imitation in childhood (G. 3. S. Rössler. Premotor cor- tex and the recognition of motor actions. Ramachandran. (1998). 1551–​1596.. (1999). (1996)... M. R. Science. E. C.. V. (2005). & Newport. S. R. 7(2). Predictive coding in the visual cortex: A func- tional interpretation of some extra-​classical receptive-​field effects. 6(5). & Gallese. B. L. In D. & the PDP Research Group. dreams. Parallel distributed processing: Explorations in the microstructure of cognition. L. . L. K. J. Shibata. C. 5. K. 543–​545. L. Nature Neuroscience. & Thelen. Pillon. MA: MIT Press. Neural Computation. 545–​548. hind- sight.. J. A. 84. L. extended sequences using the principle of history compression. & Haggard.. Y. B.. Soon. 4 (2). 11. (2000). 113(2). A. K. M. E. (1995). B. 433–​4 49. The mental representation of hand movements after parietal cortex dam- age.. Ciancia. Science. 234–​242.. Nighoghossian. 7. & Agid. 80. Shima. Wales. Cambridge. S. S.. (2014). E. Retrograde amnesia and memory consoli- dation: A neurobiological perspective. 239.. A cognitive architecture that combines internal simula- tion with a global workspace. Posada. . (1992).284 284 References Schmidhuber. G. O. 273–​299... 7(8). Smith. Journal of Neurophysiology. Shanahan. J. P. Neuronal activity in the supplementary and presupplementary motor areas for temporal organization of multiple movements. 1716–​1720). and sense of agency. Networks of the brain. & Okabe. Shima. (1969). Dynamic pattern generation in behav- ioral and neural systems. (1995). & Kelso. Reinforcement learning when visual sen- sory signals are directly given as inputs. Duhamel. Science.. R. Shimojo. Giraux. New  York:  Oxford University Press. 5. 80–​8 4. & Tanji. Consciousness and cognition. (1988). MA: MIT Press. N. Cambridge.. The sciences of the artificial (2nd ed... 343–​348. Squire. 3. A.. M. 268(5210). Cohen. Development as a dynamic system. Sporns. 2148–​2160. H.. (1981).). P. (2008). & Haynes. In Proceedings of IEEE International Conference on Neural Networks (Vol. Schöner. Trends in Cognitive Sciences. Psychological Review. The continuity of mind. Nature Neuroscience. (2003). Laws of form. (2006). Altered awareness of voluntary action after damage to the parietal cortex. (2007). Using dynamic field theory to rethink infant habituation. 196. Journal of Neurophysiology. R. H. 169–​177. Y. Brass. Learning complex. T. G. M. J. Daprati. P. H. 3247–​3260. S. Simon. A. Spencer-​ Brown. Heinze. Science. 1564–​1568. Spivey. E. 15(2). UK:  George Allen and Unwin Ltd. Frontiers in Psychology. pp. Sirigu. 273(5281). & Tanji. (1996). Computation beyond the Turing limit. J. Siegelmann. Both supplementary and presupplementary motor areas are crucial for the temporal organization of multiple move- ments. Unconscious deter- minants of free decisions in the human brain. (2006). (1997). Sirigu. (1998). J.. Schöner. Postdiction:  Its implications on visual awareness. Dubois.. (2010). (2003). G. 1513–​1539. A. & Thelen. Current Opinion in Neurobiology. & Alvarez. 421–​436.. Y. (1996). Sugita. Anatomy of mind:  Exploring psychological mechanisms and processes with the CLARION cognitive architecture. Haschke. 421–​4 43. and Cybernetics. & Haykin. J.. & Tani. 33–​52. New  York:  Oxford University Press. J. 10(1). A. . 586–​6 05. Tani. Röthling. H. Journal of Consciousness Studies. Autonomy of “self” at criticality: The perspective from syn- thetic neuro-​robotics. 11(9). 1747–​1752). Self-​organized control of bipedal locomotion by neural oscillators in unpredictable environments. & Fukumura N. Szegedy. Adaptive Behavior. J. Neural Networks.. F. 47(2). (2004). D. Reed. 5(5. 5–​24. 262. Going deeper with convolutions. J. Yamaguchi.. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. Biological Cybernetics. 685– ​6 88. (2015). 1–​9). J. 516–​542. Tani. Man. (1997). 13(1). Sermanet. (1991). Proceedings of the IEEE. Tani. (2014). (2003). & Ritter.. An interpretation of the “self” from the dynamical systems perspective:  A  constructivist approach. 16. The dynamical systems accounts for phenomenology of immanent time: An interpretation by revisiting a robotics synthetic study. 129–​141. Tani. (2005). Tani. (1998). R... Y. … & Rabinovich. H. N. & Shimizu. 153–​159. Jia. (An experiment by the mobile robot YAMABICO). Y. Tani. R. Taga. Part B.. (2009). & Fukumura. S. Learning goal-​d irected navigation as attrac- tor dynamics for a sensory motor system. 26(3). Robotics and Autonomous Systems. W. J.. S. Proceedings of the 1993 International Joint Conference on Neural Networks (pp. 606– ​6 07. J. (2014). Situated robot learning for multi-​modal instruction and imitation of grasping. Journal of Consciousness Studies. J. K. 11–​23.. J.. Tani. P. 102(4).. Tanaka. J.. Science. J. J. Self-​Organization and compositionality in cognitive brains: A neurorobotics study. Self-​ organizing internal representation in learning of navigation:  a physical experiment by the mobile robot YAMABICO. 17(5). Model-​based learning for mobile robot navigation from the dynamical systems perspective. Learning semantic combinatoriality from the interaction between linguistic and behavioral processes. Anguelov. Tani. (2016). G. K. Neural Networks. C. Neuronal mechanisms of object recognition. Friston.. (2004).​6 ). Sun. Liu. Learning to generate articulated behavior through the bottom-​ up and the top-​down interaction process.. (1993). (1993). 4(102). 65. IEEE Transactions on Systems. Self-​organization and composition- ality in cognitive brains [Further thoughts]. Tani.  285 References 285 Steil. 147–​159. Proceedings of the IEEE. Adaptive Behavior. J. S. & Asoh. Gallese. & Smith. The physics of consciousness. I. H. F. D. 78. Thelen. Philosophical. 12(7). 2(4). Rizzolatti. & Nolfi. F. J. Role for supplementary motor area cells in pla- nning several movements ahead. Consciousness and Cognition. 365–​370. & Bahlmann. S. 17. J. Taniguchi. Advanced Robotics. Embedding a grammatical description in deterministic chaos: an experiment in recurrent neural learning. The Biological Bulletin. J. Nakamura.. Tani.. (2005).. J. Nagai. K. 215(3). 1–​11. M. 72(4). (2004). N. Behavioral and Brain Sciences. 216–​242.1080/​01691864. . Neural Networks. (1995). Tomasello. Constructing a language: A usage-​based theory of lan- guage acquisition. T. Tani. (2008). M. Toward an interpretation of dynamic neural activity in terms of chaotic dynamical systems. K. M. V. Listening to action-​related sentences activates fronto-​parietal motor circuits. Tokimoto. (1997). T. & Fukumura. 255–​261. J. (1998). Uddén. Self-​organization of distributedly repre- sented multiple behavior schemata in a mirror system: Reviews of robot experiments using RNNPB. 162–​190. Tsuda. Cambridge.. (2001). Iwahashi. Consciousness as integrated information:  A  provisional manifesto. S. A dynamic systems approach to the development of cognition and action.. A. Y. Tani.. 367. (2002). MA: MIT Press. MA: Harvard University Press. Tettamanti.. Neural Networks.. 24(5). 1273–​1289.. Buccino. (2004). G. H. Transactions of the Royal Society of London Series B-​Biological Sciences. Tokyo: Keiso-​shobo. & Shima. & Perani. 1131–​1141. L. T. N.1164622.. Körner... (2009). A rostro-​caudal gradient of structured sequence processing in the left inferior frontal gyrus. T. DOI: 10. Cambridge. Self-​organization of modules and their hierarchy in robot learning prblems: A dynamical systems approach. (2012). Tani. J..2016. (1994). 51–​71. Tanji. Ogata. & Okanoya. Tsuda.286 286 References Tani. (1999). Sugita. and Nolfi. & Shimizu. Learning to perceive the world as articulated: An approach for hierarchical learning in sensory-​ motor systems. J.. C. Memory dynamics in asynchro- nous neural networks.. E. P. 273–​281.. 371.. J. I. Cortical movement preparation before and after a conscious decision to move. Symbol emergence in robotics:  A  survey. 413–​416. Journal of Cognitive Neuroscience. & Miller. 2023–​2032. Fazio. Tononi. N. 17(2).. Progress of Theoretical Physics. Ito. Cappa. M. (2016). M. G. Danna. G.. (1994). Nature.... Scifo. (1987). E. 11(2). System Analysis for Higher Brain Function Research Project News Letter. Spontaneous construction of “Chinese boxes” by Degus (Octodon degu):  A  rudiment of recursive intelligence? Japanese Psychological Research. 793–​810. T. Trevena. 46. Saccuman. Biological Cybernetics. (1996). R. J. e37843. L. Harvard University). B. L. White. E. T. (2007). MA: MIT Press. R. Wolpert. A Learning algorithm for continu- ally running fully recurrent neural networks. 649. J. self-​extinction. E. J. Preparation for grasping an object: A developmental study. . 6 (2-​3). Nagai. P.. F. F. 171–​190. & Wagemans.. de-​ Wit. Tokyo. Experience and Awareness:  Exploring Nishida Philosophy (English Translation from Japanese)... 1. J. AI & Society. (1994). 121(4). (2012). e1000220. & Gray. M. 1(4). Werbos. Sahin. Journal of Neurophysiology... and philosophy in the service of human civilization. P.. (PhD thesis. E. IEEE Transactions on Autonomous Mental Development. (2016). 7(2). Emergence of functional hierarchy in a multiple timescale neural network model: A humanoid robot experiment. C. Yen. Cambridge. J. Werbos. Williams. S. Evers. M.. Staged development of robot skills:  Behavior formation. Science. Beyond regression: New tools for prediction and analysis in the behavioral sciences.. Williams. A. C. (2014).. Present-​ t ime consciousness. Wilson. Y. Ugur. 1326–​1341.. & Tani. 11. F. Varela. (1999). Neural Computation. Van der Hallen. M. (1991).  287 References 287 Ueda. & Rönnqvist. (1974). Varela. The embodied mind: Cognitive science and human experience. B. & Oztop. C. 1317–​1329. Generalization of backpropagation with application to a recurrent gas market model. 31(2). London and New York: Routledge. PloS One. von Hofsten. (1994). 265. J. Yamashita. J.. Japan: Iwanami Shoten. 97. Boets. Simulation. Neurophenomenology: A methodological remedy to the hard problem. & McNaughton. J. Keiken to jikaku–​Nishida tetsugaku no basho wo motomete. J. Neural Networks. (2014). E. L. Precise minds in uncertain worlds: predic- tive coding in autism. 7(5).. Journal of Experimental Psychology: Human Perception and Performance. & Tani. 270–​280. L. (1988). K. Journal of Consciousness Studies. Descartes:  The project of pure enquiry. 610–​621. 111–​140. & Rosch. Neural Networks. E. Baker. Thompson. Iwanami-​shoten. Y. B. 14. Spontaneous prediction error generation in schizophrenia. (2008). Multiple paired forward and inverse mod- els for motor control. PLoS Computational Biology. S. (1998). D. Varela. (1989). Heterogeneity in the responses of adjacent neurons to natural stimuli in cat striate cortex. (2015). 330–​350.. Yamashita. 339–​356. 119–​139. 4(11). 676– ​679. Journal of Consciousness Studies. Y.. Van Eylen.. S. & Kawato. 3. affordance learning and imitation with motionese. (1988). Reactivation of hippocampal ensemble memories during sleep. Psychological Review. D.. & Zipser. J. Van de Cruys. Adaptive Behavior. 7.. M. . Zhong. 185–​199. (2002). S. Frontiers in Behavioral Neuroscience. Cangelosi. 22. 10(3/​4). (2014). Neuromodulation of reactive sensorimotor mappings as a short-​term memory mechanism in delayed response tasks. T. J.. & Wermter. A. & Thieme. Toward a self-​organizing pre-​ symbolic neural model representing sensorimotor primitives.288 288 References Ziemke. 49–╉50. 233f. 215 conscious decision preceded transitive. 225–╉26 for. 200f. 149 goal-╉directed actions. 201f. 190–╉96. 44–╉6 8 neurodynamics generating tasks hierarchical mechanisms for. 50f 56–╉61. 214f 50f.╇ 289 Index Note: Page numbers followed by “ f ” and “t” denote figures and tables. 52f parietal cortex meeting of. 265 learning from. 178f primitives. encoding. predictive 203f. absolute flow level. 28. 151–╉73 intransitive. 219–╉41. 213–╉15. 60–╉61 175–╉98. of. 65 action generation. through hierarchy. with. 49 categories. 59f perception's role in. 215 external inputs perturbing. See also complex actions. 39 289 . actional consequences. abstract sequences. 145–╉48. 228f. 223f. 229f. respectively. 192f brain. 196 training influencing. 238f accidental generations with functional hierarchy developed spontaneous variation. 39 free will for. 237 unconscious generation of. 147f action-╉related words. 215 as voluntary. 197–╉98 language bound to. 234f. SMA 225f. 52–╉53 236f. 55 perceptual reality changed sensory-╉motor flow mirroring. 199–╉218. intentions subjective mind influenced by. 67 recognition's circular causality actions. 65 by. 221f. by. 22. See also behavior model.. 171 Gibsonian approach Ballard. 85f. 31–​32. 204–​6 Aristotle. 131–​32. 158–​59. Michael. 207f robotics. 9–​10. 14 (BPTT). 30 Rössler. Shun-​ichi. authentic agent. 93–​94.. 126–​28 behaviors. See animals. 160 active learner. 205–​6 175. 10. H. Arbib. limit cycle attractors as reactive. 200f Arie. 52–​53 embedding of. arm robot. 143.. 143. 90. J. 113. 82. Minoru. 227–​30. 99f 200f.290 290 Index action sequences. 84. 227–​30. 84–​85. 228f MTRNN. Dona. 31–​32. 128f alternative images. 71 behavior attractor. 207f affordance. 63–​6 4 behavior primitives. 126f. back-​propagation scheme 104f. 50–​51 127–​28. 4. 238f. 180–​82. 91–​97. 219–​20. Randall. 203f. 206–​7. J. 229f autorecovery. 191–​9 6 attractor. 91f beings. 99f as imitative. 131f 201–​2. 171–​72 as compositional.. 130–​31 alternative thoughts. 141–​42 as global. recursion-​like behaviors also chunks exhibited by. D. 34 types of. 219–​30 invariant set. 229f autism. 103–​9. 24f functional hierarchy development. 191 localist scheme. 107f ambiguity. 158 being-​in-​t he-​world. 60 agent. 48. 10–​11 compositions. 31–​32 148. 157. 13. 100–​102. 98–​100. MTRNNs generating. 24–​25. 252 authenticity. 58. 98–​100. 200–​202. 200–​201. 37 Amari. 102f attractors. 267 228f. 121. 181f attractive object. P. 158f Being and Time (Heidegger). 29–​32. 201f appearance. 105f. 155. 256. 237–​39.. 31–​32. 261 PB vector value assigned to. 206–​7. also chunking 148. 143 active perception. 158f as spontaneous. See also error behavior-​based robotics. alien hand syndrome. 71 behavior-​based approach. 261 Bahlmann. 171–​72 . 261 distributed representation Ashe. 158–​59. 66. 24–​25 attunement. 66. 31–​32 Beer. 229–​30. 130–​31 back-​propagation through time Act-​R. See also skilled behaviors Asada. 257–​58 as novel. 39 as authentic. A-​not-​B task. 15. 116f affective developmental Badre. 201f artificial evolution. See also Bak. See authentic being.  63 by. 191. 207–​8. 52f bodies. bottom-​up pathway. Valentino. 59f. 33–​34. 120 Brecker. 49 66–​67. 164–​65. 44–​54. 18–​19 chemical plant as. See also brain science. S. 206–​8. 207f. 116f 47f. 128f . 73f calculus of indications. brains. Rodney. 149 FLN component in. 43–​79. 44–​6 8. dynamics in. 49. 125. 52f catastrophic forgetting. 42. 196 through time Brooks.. 52f. 106. on linguistic competency. 153f boys. 81–​83. 40–​41 blind man. 59f. 40–​41 36f. 103–​6. 65f. 55–​68. 142–​43 outcomes monitored by. 255 46f. 6 Cantor set. 70f. 119. 44–​54. spatio-​temporal hierarchy of. 133f 193f. overview. 50f. 16. See back-​propagation Broca's area. 107–​8. 158–​6 0. 59–​6 0. 108–​9 197–​98. 217 56–​57. 65f button press trial. 192f. 103. 30–​31. 43–​79 bonobos. 11 future directions of. 43–​79 meaning of. 165 human language-​ready. 65f bimodal neurons. 61 visual recognition. 31 overview. 195f Yamabico. 191 cells. (cascaded RNN). 51–​53. 109–​12 as inauthentic. 158f cognitive competencies carpenter.. See also neural network 107f. 30–​31 models. Blue Brain project. 58 two-​stage model mechanized Blakeslee. 248. 132–​34. 50f. specific brain structures Buddhist meditation. 207f 205.  291 Index 291 of equipment. 215 MTRNN correspondence. 60–​61 man reflecting on. 4–​5. 45f. 63–​6 4. S. 60–​61 central pattern generators (CPGs). 255–​61 bottom-​up error regression. 7. 262–​67 Cartesian dualism brain and. 60–​61. 57. 69–​71. 145 models. 33–​34. 47f. 208 symbols in. 30 recognition in. See also intention adjustment mechanisms neurons employed by. 254–​55 action generation. 263 Braitenberg. cats. 10 Cartesian dualism. 32–​37. 252–​53 BPTT. 266 branching bound learning. 190–​91 MTRNN. 4–​6 126–​28. 70f brain science and. 62f. 119f. 107. 245–​47 Blakemore. 264 hosted by. bottom-​up recognition. 11 cascaded recurrent neural network hierarchical mechanisms. 53–​54. 191–​9 6. mind originating in. Michael. 152–​6 0.-​J.  265–​67 complex actions authenticity and. 87–​9 0. 23. context. 13–​15. 12f MTRNN. 50–​51 circular causality. 208. 107–​9. chemical plant. 265 codevelopment process. 209–​15 criticality and. 197–​98 15–​18. 10–​12. 46f. 243–​47 CNN. 49 76–​77. 262. 47f Induction On-​line complex objects. 46–​47. 48 Clark. 7. 213 generalization and. 72–​73. 60 cognitive competencies. chaotic itinerancy. Noam. 149. 240–​41. 225–​26 collision-​f ree maneuvering. 266 closed-​loop mode. 159 compositionality. 209–​15 238f. 202. 248–​49.292 292 Index cerebellum. 178 in cognitive mind. 147f . 237–​39. 175–​76. 18–​19 244. 225–​26 symbol systems. 108. 94–​95. 194–​9 6. 75. 15. 175.. 216. 209f. 9–​20 of language in narrow sense recursion. 260. 238f developmental training of. Andy. A. 226 columnar organization. 63. 220. 172. 17f junctions. 13–​15. circuit-​level mechanisms. 214f CLARION. 145–​48. 264 network as fluid. 195f cogito. 168–​6 9. 35 cognitivism. 204–​6. 152–​61. 9–​13. 6 embodiment chiasm. 57 cognitive fragmentation. David. Chalmers. 209–​15. 225–​27 cognitive models. 9–​13. 45. 190–​91. 10–​11 composition. 252 Cleeremans. faculty overview. 246 complex visual objects. 262 symbol grounding problem.. 218f See also being compositions. 257 CFG. 12f. See context-​f ree grammar cognitive minds. 84. 253 chaos. (classical AI). 212f. 266. 244–​45. MTRNN. 155. 58. 73f structures. See Connectionist complex object features. 14t language in broad sense. 149–​50. 10 cerebral hemorrhage. 217–​18. 246 243–​47. 107f. 25. 161 179. 262 chimps. 169–​72 QRIO. 91f. 12f chunking. combinatory explosion problem. 106 229–​30. 78–​79 comb. Learning with Adaptive Rule 46f. 29–​32. See also faculty of models. 220. 152 Churchland. 186–​87 collective neurons. 12f Chomsky. 222–​23 coherence. 7. 72. 169f See also embodied cognition. experiments. 198. M. 14t chaotic attractor. 9–​13. 85f cognitive processes. 46. 250–​51 QRIO. 46f classical artificial intelligence compositional action sequences. chunks. See convolutional neural development of. 170–​72. 237–​39. 4–​5. D'Esposito. 237 consolidation. 238f CPGs. 75. 70f. 249–​50. 164–​6 9. 11 consciousness. 37–​39 Desmurget. 237–​39. 16. 259 263. contexts. 11 postdiction and. 200–​202. 29. 48 concepts. 158f developmental training. 253–​54. 200–​201.. 127. 10–​12 free will for. 248 counting. 209–​15. 75 death. 83–​8 4 . 146 cogito problem concerning. 25. 73f. 33–​34 (CNN). 69–​75. 72 conscious awareness. 267 degu. 230–​ creative images. 73–​74. 9–​13. A. 214f. 207f 197. 259–​61 continuous-​t ime recurrent neural Diamond. 236f. 256. 122f. 243. deterministic dynamics. 219–​41. 200–​201. Y. 9–​13.. 75. continuity of minds. 206–​7. R. 204–​5. See continuous-​t ime preceding. 230–​39. 121 cortical song. 47. Daniel.. 12f developmental psychology. 201f multiple-​t imescale recurrent cognitivism. 238f Dennett. 250–​51 conscious decision. 157. 172 Descartes. 172n2 also Cartesian dualism conscious states. 12f neural network localist scheme. Demiris. action CTRNN. 120–​25. 238f criticality. See also 200f. 95f Adaptive Rule Induction On-​ cortical electrical stimulation line (CLARION). 238f. 226–​27 11. 172. M. 236f. 76. 171–​72 free will and. 100 209f. 123f. 200f 236f. 12f contours. See central pattern generators intentions. 99f 158–​6 0. 197 39. 48. 28 Dale. 250. 256 structure of. 258 Connectionist Learning with corridor. 200f continuous-​t ime systems. 3–​4 depression. 90 in symbol systems. 27–​28 cup nesting. 94.. 187. 34 easy problem of.  293 Index 293 behavior primitives. difference equation. 237 recurrent neural network conscious memory. 167f. 73–​74 connection weight matrix. See also cursor. 225–​26 deterministic chaos. 31. 226–​27 context-​f ree grammar (CFG). 238f deep learning. 236f. 258–​59 hard problem of. 57 streams of consciousness absolute flow of.. 247 study. 97–​100. 18–​19. René. 211 network (CTRNN). 169f. 230–​39. deep minds. 108 questions about. M. 246 convolutional neural network concrete movements. See surprise quantifying. 29–​32 Dasein. 212f.  263 dimension of. 160f. 169f end-​to-​end learning. 7 emergence through synthesis. 28–​29. 81–​137. 107f. C. 261 dynamic learning. 181f. 142 electrophysiological dreaming. 89 domain specificity. 92f distributed representation framework. 215 emotions. 67 during. 118–​20. 90–​93. 90 embodied cognition. 118–​20. 78. 4–​5 dynamic conflict resolution. Jeffrey. 129f. entrainment. 89f 86f. 97–​98 discrete time system. 167f. 142–​43 A Dynamic Systems Approach to direct reference. 92f embodiment. 85f. intermittency end effectors. 119f nonlinear dynamical systems Elman net. 180–​82. 257 and Action (Thelen and discrete movements. 258 edge of chaos. 256–​57 Edelman. dynamical systems approach. 88f. 88f. 126f. 145 “Elephants don't play chess” dynamical structure. See suspension of disbelief . 254 91f. 135f modeling. 22–​23. 107–​9. 41. 85f structural stability. 124–​25 disturbance of self. 70f double intentionalities. 160. 83 dynamic closure. 36f recurrent neural network with direct experiences. See also Elman. 85–​9 0. do-​re-​mi example. 79. 201–​2 echo-​state network. 94–​95 81–​137. 245 Eliasmith.. 82 125–​36. 135f. L. 23f. 236–​37 79. See also epoché. 128f. 119f continuous-​t ime. parametric biases 26–​28. 26–​28 69–​71. 75 196–​97. 85–​9 0. 165 experiments. 42. Gibsonian approach and. 177. H. 107f. 83–​8 4 See also dynamic neural discrete time. 133f. 88f. 95–​9 6. 61. 86f. 240 self-​organization applied by. 34 the Development of Cognition disconnectivity hypothesis. 106–​8. 35–​36. 83–​93. 130f. 133f. 235–​36. 89f. 91f. 86f. 166–​6 8 emergency shutdown. 86–​87.. easy problem. 154–​56 137. 126f prediction error generated by. 255 dynamical systems. embodied mind. 255 Doetsch. G.294 294 Index dimension. 106 36. 36f. 178f. 56–​57 Dreyfus. 166–​6 9. 180. definition of. 217 dynamic neural network models.. 176–​79. difference equation. P.. 32–​37. 89f network models neurorobotics from perspective of. 180f Smith). 132–​ (Brooks). 35–​36 embodied cognition modeled by. 29 electroencephalography (EEG). dynamical systems approach 131f. dynamic systems theory. 12f. 204. extended mind. 206 223f. 42 first-​person experience. 257 fluid compositionality. fingers. 203f. 84. 263. 202. 221f. 238f external observer. stream of consciousness and. 230–​39. 225–​26 feed-​forward network model. 206–​7. 39 Feynman. 237 of selfhood. experiences. 216. 206 FLN. 81. 108 . 161 experience. 26–​29 236f. 234f. 106–​8. 17. 107f vehicle possessing. 78. See also direct focus of expansion (FOE). 26–​28 Freeman. 161. 160. fixed point attractor. 9–​10 fMRI. Gareth. 152. See faculty of language in retrograde axonal narrow sense signaling mechanism Floreano.. 69–​75. 215 for action. of subjective error regression. 94 113–​16. 130f overview. 246 261. 19 definition of. 126–​32 resonance imaging experiences. 34–​36 204–​5. 12. 223f. 113f. 204–​5 broad sense perceptual sequences acquired by. 96f 236f. 221f. 220–​22. fast dynamics. 248–​51. 88–​8 9. 10. See functional magnetic evolution. 225–​26 (FLN). perception dependence of. 58.  295 Index 295 error back-​propagation scheme. first-​person forward model. 39 fallenness. frame problem. 231–​39. 230–​39. (FLB). 39–​41. 116f. 55. 236f. 153. 227 37–​41. model for. 16. 234f. pure experience. 219–​41. 11–​13. 177–​78 subjective experiences frame system. 95f experiences. 119f. See faculty of language in CTRNN application of. 96–​97. D. Walter. flesh. 225f. 130–​31 implementing. 59. 238f faculty of language in broad sense consciousness and. 10. 123f. 32 experiments. 85. 72–​73. 94. 252 for conscious awareness. 39 free will. 29 continuous flow of. 113f. 23–​42 225. 225f at M1. 221f 129–​30. 49 James considering. 218. Richard. QRIO. 207f intention correlates. 210 70f. 221–​25. 205. 219–​41. 229f. 228f. facial imitation. 238f. 69–​75. 238f finite state machine (FSM). 238f faculty of language in narrow sense consolidation. 40f. 257 FLB. 265–​67 external inputs. 236f. 265 Evans. 73f feature representation. 266. 101 233f. in MTRNN model. 233f. 85f. 103 postdiction and. 207–​8 flow. 40f 112–​20. 16 236f.  200f Gibsonian approach. gated recurrent neural networks 96–​97.. 212f. Steven. H. 200f . 162. 240–​41. 125 67. 217 harmonic oscillator. 93–​95. 200. 142–​43. 42. See General Problem Solver Recognition. 68 Hierarchical Attentive Multiple Goodale. 131f. 124 of whole Haruno. 251. K. 258 General Problem Solver (GPS). 257 grandmother cells. 132 264. 267 genetic algorithm. 157f on future. 245 71–​73. 98.. 264. 170. I.. Mel. 50–​51 hallucinations. See structuring processes Harris. 39. 65f frontopolar part of prefrontal cortex. 190–​91 95f. 30–​31. 96f (RNNs). 158f 248. hands. 172n2.. O. 184f. 194–​9 6. 60. 143 Hauk. Martin. 200f Gibson.. 67. 264 gated local network models. 93–​95. 17. Fukumura. 93 Georgopoulos. 49–​50. 267. 74. 54 Harnad. 54 FSM. 253 hermeneutics. Karl. S. Y. 65. 11. Hebbian learning. Naohiro.. 66–​67 Gibson. A. 67. 202 metronome in synchrony with. Vittorio. 189f Gaussie. 107.. 156–​57 263. 200–​201. 249–​50. M. 245–​46 frontal cortex. 73f Graziano.. on past. 63 200. See finite state machine groundlessness. 64–​6 6. 200–​201. 95f Hayes. Alvin. 250. 58. 34–​35. 65–​6 6. 68 248. 16.. 158–​59.. See also goal-​d irected action plans. M.. 13–​15. 253. G. 252 functional magnetic resonance imaging (fMRI). 207 grasping neurons. 214f. 179. 200–​201. global attractor. Gibsonian approaches 60. generalization. Eleanor. 185f 195f. 21.296 296 Index Fried. A. Graves.. 14t. 29–​31 Goldman. 41–​42. 263. 267–​6 8 Fukushima. 61. 60–​61. 12–​13. being-​in-​t he-​world 156–​57. See also Neo-​ Heidegger.. Gallese. 235 210–​15. 108. 187–​9 0. 172. 75–​76 grammar. 170 hammer. 66. 12f Friston. 70–​71 hair. 170. Y. 75. 159–​6 0 Gunji. 55. 13. 56 Models for Execution and GPS. 222 QRIO imitating. 251.. 182 QRIO predicting. hard problem. See also symbol Gershkoff-​Stowe. 99 grounding problem Gestalt. J. 92. 228–​29 Gallagher. Haas. 235 goal-​d irected actions. L. 257–​58 handwriting recognition system. 183–​87. 49–​50. 61. See imitative behaviors. 22–​23. 47f . 44–​54. 46–​47. See symbol grounding inferotemporal cortex. 47f. 23f. visual imagery (MOSAIC). parietal cortex damage in. 102f. 24f. preverbal. 143. 256 imitative actions. 189f. 102f. 170 object used by. 169–​72 language-​ready brains. 211 Hume. 200f imitation. 192f. 237 for humans. 176. 221–​22. Takashi. Hwang. 102f area. 107–​9. 200–​201. problem 46f. 96–​97. 191–​9 6. 46f. 46f. 71–​72. 191 index fingers. 32 inferior parietal lobe (IPL). 228f.. 50f. 216–​17 manipulation. 56. 101–​2. 31–​32 imitation for. 72–​73. 265 131f. 182. 100–​102. 189f Humanoid Brain project.  297 Index 297 hierarchical mechanisms. 187–​9 0. 73f. 47f hybrid system. 221–​22. hierarchy. self robot imperative sentences. 182–​9 0. 182 184f. 266. 27 direct experiences for. 106. 187–​9 0. 65 by mental state reading. 57 97–​100. 39. 251 Hobbes. 41. 63 QRIO. 102f 145. David. 164. ideational apraxia. 96f linguistic competency of. S. 186–​87. 109 hierarchical Modular Selection images. 65 incoherence. 142–​43. 75–​76 intentionality possessed by. 142–​43 inauthentic agent. 190 inauthentic being. 26–​29 inferior temporal area (TE) (TEO).. 248–​49 inferior frontal cortex. 18–​19 66–​67. statistical learning humanoid robots. humans 193f. 185f. 52f ideomotor apraxia. 107f impression. 229f. 26–​28 inferior parietal cortex. 221f holding neurons. 165 prediction error influencing. 57 hierarchical mixture. 195f cogito level had by. Thomas. 65–​6 6 on time perception. 72. 221–​22. 101–​2. of. 137 200–​201. 57 45f. 198 how pathway. Ikegami.. Edmund. 131–​32. 221f 227– ​30. 190 Hochreiter. 262. 200f Ikegaya. 50f. Husserl. 61 on direct experiences. 221f. See also and Identification for Control motor imagery. 211 24–​25. 109. 259 46– ​47. 65. 142–​43 intransitive actions of. 100–​102. 165 game. 188 hippocampus. 66–​67. 99f presupplementary motor imitation in. 66 developmental psychology. 21. Y. 183 temporality notion of. 189f Hopfield network. indexing. 66 also other robot. homeostasis principle. 190–​91 infants mirror systems in. 197. J. 257. 202. Z. 250. 57 top-​down subjective. 70f. 71–​72. 58. Kelso. 238f Jaeger. 211. H. spontaneous. J. 218 structures. 201. See inferior parietal lobe 225. 225–​26 organization of. See also streams of intention switched to from. S. A.. 69–​75. 70f. Scott. 48 . 59 from PFC. 236f. 19 parietal cortex. 58. 76 Jeannerod. 96f 220–​22. 59f. 142. 223f. 73f. 266 intermediate dynamics. 207f interactionism. 210 VP trajectories. 129–​30. 65 “Intelligence without representation” intraparietal sulcus. 69–​75. 129f. 166–​6 9. 73f 42. 209f. William. 68 free will consideration of. 62f intentions. 152 conscious awareness. 252 instrumental activities.. 61–​62. 95–​97. 37–​41. 133f. 69–​75. 208. 162. 204 free will neural correlates. 170. during dynamic information processing. 78 Ito. 211 mapping. 73f 182. 74–​75 momentary self spoken of by. kinetic melody. 171 parietal cortex as involved in. 116f. 11 prediction error generated by. 263 intention-​to-​perception Karmiloff-​Smith. 158f intentionalities. 130f interaction. Mitsuo. IPL. 203f. 168 212f. 89. 106 invariant set. M. 210 QRIO. 206–​7.. 169f initial states. 224 Kourtzi. 238f joystick task. Kohonen network. 125. 56–​61. 207–​8 information bottlenecks. intermittent chaos. 73f network (Jordan-​t ype RNN). 69. 62 (Brooks). initiation of. 226 of intention units. 221f internal contextual dynamic setting. 57. 21. 243–​47 knowledge. 229f MTRNN. 221f Khepera. 202 intermittency. 214f intermittent transitions. See also subjectivity Iriki. 28.. 70f. 213–​15. 252–​55 Kiebel. 230–​39.. 216 16. 218. 158–​59. James. 210 information hubs. 84. 204–​6. 70f. 200–​201 learning. 204. 133f. 135f inner prediction error. 71–​75. 69–​75. 40f.298 298 Index infinite regression. 132–​36. 73f. 102f intransitive actions. 144 Johnson-​P ynn. 236f. 144 Kawato. 227–​30. 220–​22. 134 230– ​40. problem of. 144 consciousness mirror neurons coding. 214f information mismatch. 240 Jordan-​t ype recurrent neural rising of. 102. 167f. 208–​15. 161. Atsushi. Masao. 164. 152 intention units.. 205 228f. 257 internal observer.  217 Lu. 170–​71 200–​201. 218. 127–​28 Lateralized Readiness Potential. John. 46 locomotive controller. 197–​98 M1. 33. bound. 147f like me mechanism. 132. 92–​93 Massachusetts Institute of locomotion evolution with. 84. 89f. 235. Markov chains. (RNN) model.. of QRIO. limit attractor language-​ready brains. 200f language. 28 consolidation. 85. 214f 249. 88f. 145–​48. 17f. 141 90. See primary motor cortex in RNNPB. 220. imitation. 221–​22. 224 as latent. 193f. Laird. 85f 167f. 190 228f. 202. 190–​91 landmark-​based navigation. mobile LIP. 31. 70 logistic maps. 85f linguistic competency. W. 191–​9 6. 212f. 263 symbol. 221f Mach. 85–​8 9. 177–​82. 45f. 63–​6 4 Libet. 181f macaque monkeys. Alexander. 240. periodicity of. 85. 108 . 190–​91 Luria. 84.  299 Index 299 Kugler. deep learning. tutored sequences. 56–​57 limit cycle attractors. 86f. 223. 129f. 216 of imitative actions. 175–​76 limit torus. 128–​30. 69–​71.. 227–​30. 103 126–​28. 48 manipulation. 187. landmarks. 246–​47 66–​67. 162–​72. 183. 195f 154–​57. 216–​17 predictive learning look-​ahead prediction. Technology (MIT). 95–​9 6 in MTRNN. long-​term and short-​term memory dynamic learning. imitation. local representation framework. 213 Kuniyoshi. 224. 126–​28. 33–​34. 66–​67.. See also longitudinal intentionality. 221–​22. 177 190–​9 6. error back-​ recurrent neural network propagation scheme. 155f. 209–​15. M. 128f Matarić. 70f. 191 evolution. 178f. 101. 163f. 169f. 226–​27 85f. 221f 219. 62–​63. 46 as statistical. 15. 192f locomotion. 225f man. 221–​22. 84. See latent learning. 108 learning. Benjamin. 192f. 248 localist scheme. 157f as end-​to-​end. Y. action bound to. See lateral intraparietal area robot performing. Yamabico. Ernst. 259–​61. local attractors. 229f limbs. learnable neurorobots.. 22–​23. 221f Lyapunov exponent. 230. 52–​53 Hebbian. 166–​6 8 130f. 61 Li. 17–​18. 196–​97. 161 also walking lateral intraparietal area (LIP). X. 73–​74 of visual objects. 209f. 128f. 161 offline processes. 23f lesion. Maurice. See also logistic maps recurrent neural network with meanings. 202 middle temporal area (MT). intention coded by. 163f. 32–​37. 195–​9 6 parametric biases medial superior temporal area dynamic neural network model (MST). See also miming. 36f. subjective mind 16 –​18. 154. 78. mental states. 195f by reading. See Massachusetts Institute of middle way. 201. 259 with vision. 5–​6. 57 as extended. David. 63 May. of monkeys. 117. 169–​72 Matsuno. 132 mirror box. imitating others 193f.. 183. 46f. 128. 178f medial temporal lobe. mirror systems. 67 monkey–​banana problem. 17f deep. minimal cognition. 14t . 188 minimal self. 128f Milner. 189f overview. 193f 42. 45f. Marvin. 126 14–​15..300 300 Index matching. 182–​9 0. 65. 60–​61. 65f melody. 244. in humans. 156 model. 65 237. J. 61–​6 4. 169f. 16–​18. 65f Meltzoff. 65 memory cells. 68 performed by. 57 Yamabico mind/​body dualism. 191–​9 6. 45–​46 for. 76. K. 101. 29 Maturana. 246 evidence for. 262–​67 momentary self. tearing. 68 167f. 47f overview. 177 32–​37. 21. 64–​6 8. 17f dualism landmark-​based navigation mind-​reading. See also embodiment. 169f IPL. 264 theory of. consciousness. 176–​79. 252 Minsky. 42. 16–​18. 144. 26–​28 grasping. 254–​55 Technology Miller. 261. 64–​67. embodied in office environment. 76 185f. 246 modularity. 36f. 254 models. 173. minds. 56 mobile robots. 171. Robert. A. cognition. 173. 100 problem. 67–​6 8 mental rehearsal. 3–​8. 66 Schneider mismatches. 177–​79. See also mirror neurons. in parietal cortex. See also cognitive 163f. 85. 25. 162–​72. 45 MIT. 192f. See Cartesian example.. 217 implementation. embodiment of. 164–​6 9. H. 17f. 248 minds. 17f continuity of. 55–​56. 44–​49. 64–​6 6. 167f. 193–​9 6. 184f.. 162–​72. 65–​6 6 mental simulation. 187 holding. 170. 70 mixed pattern generator. 16–​18. 65f Merleau-​Ponty.  229–​30 212f.. See middle temporal area navigation. MSTNN. 72 motor imagery generated by. 53–​54 189f. 102f. behavior primitives. 229f parietal cortex of. 223f perceptual sequences. 221f. 183 227– ​30. 211. 233f patterns. 234f Mormann. 187–​9 0. 217–​18. motor neurons of. 48. 213–​15. 61– ​6 2. mobile robot recurrent neural networks dynamical structure in. F. 203f. 124 parietal cortex. 246 free will in. G. 59. M. 225f MT. 204–​6 motor neurons. 252. 135f . 132–​36. 233f. 51–​52. 230–​35. moral virtue. 218f inferior parietal cortex of. 204–​6 62f. 65–​6 6 networks (MTRNNs). 183 multiple-​t imescale recurrent neural IPL of. J.. 46. 222–​23 primitive movements of. 32 limit-​cycle attractors in. 238f discrete. 204–​6. 261 214f. 212f. 75–​76 chunks. 220–​22. 101 experiment. multiple spatio-​temporal neural 53–​54. Mushiake. 203–​8. Jacqueline. See medial superior temporal area Nadel. presupplementary motor 206–​8. 73 187–​9 0. 188 temporal neural network Namikawa. 209f. 207f. 101–​2. 215 PMv controlling. 218f Moore. 208. See multiple-​t imescale based navigation. 181f. mirror neurons of. 183 recognition performed by. 206 motor cortex. 214f top-​down forward motor schemata theory. 228f. PMC. 76 257. 206 motor programs. 207f movements tutoring. PMC of. 52f. 223f. 50. 207–​8. of monkeys. 57 network (MSTNN). 207–​8. 175–​76 top-​down pathway. 222–​23 overview. 209f.. 180–​82. 217. 189f MST. 180f Mu-​ming Poo. 76 bottom-​up error regression. RNNPB as analogous to. A. 218f 222–​24. 237–​39..  301 Index 301 monkeys. 65f brain science correspondence. 213 motifs. motor imagery. H. 64–​65. 215 9–​10. See multiple spatio-​ 131. 180. See also landmark-​ MTRNNs. prediction. 206. 265 motor cortex of. 214f mutual imitation game. 208–​15. H. 73–​74 Murata. 208–​15. 57 133f. Mulliken. 251. 207f area. 75–​76 compositionality. 208 207f. 221f mortality. 216–​18.. 208 action sequences generated by. 45f.. 24f. 124 subjectivity as mirror of. types of.. 133f. nonlinear dynamics. 117. 21. 229f 157–​59. 124 from. 15. 208 Nishida. 186–​87. 257– ​59 96f. 78–​79 nonlinear dynamical systems. 214f objectification. 89. 101 Newell. 112–​25. 135f from dynamical systems Yamabico experiments. 135f Neo-​Gibsonian approaches. 76–​77. 149f PMC. 223f newness. 128f. neurodynamic structure. collective. 49. 72–​73. 56–​57. 227–​30. 48 structures in. 107f neurophenomenology program. 132–​36. 228f. 95–​97. 73f 36f. 132–​36. 255–​56. 132–​36. 147f objective science. Nishimoto. perspective. 130–​31. 73f subjective mind's distinction postsynaptic. 40f presynaptic. 27–​28. 23–​42. 212f 133f. 22–​23. 72 148–​50. 172 . tangency in. See also neural activation sequences. 126f. 116f. 263 neuroscience. 235 neurodynamics with timescales. 116f. neuro-​phenomenological-​ 133f. 13. 131f. 144.) as spiking. 113f.. 107–​9. as motifs. R. model. 158f nowness. 171–​72. 147f novel action sequences. 89f 122f. 251–​55. 209f. 135f robotics. 113f. 208 phenomenology. 145f. 7. S. See also mirror experience and. 254 self-​organization in. 23–​42. 267 neurons. 7. Kitaro. 92f. 26–​29 neurodynamic system. 24f. 25 neural circuits. 40f hard problem. motor neurons. 53–​54. 46. 106 views in. 213–​15. 119f. 27. 72–​73. 90–​93. 256–​57 problem. 143–​48. 255–​56 internal contextual dynamic V1. subjective neurons. neurorobotics 133f. See structural stability of. General Problem Solver 222–​24. 135f Nolfi. 125–​36.302 302 Index navigation (Cont. 38–​39 network models objective world. Allen. 63. 130f. neural network models. 200f neural correlates. 83– ​93 122f. See also logistic maps overview. 248–​49 neural activation state. 266–​67 bimodal. subjective (nouvelle AI). 119f. 75 subjective mind as tied to. 200–​201. 123f nonlinear mapping. 145–​48. 36f. 109–​10. neural objective time. 123f nouvelle artificial intelligence neurodynamic models. 132–​36. 153–​62 129f. See brain science neonates. 112–​25. also feed-​forward network model 91f.  10–​11 192f. 215 action generation role of. See Parallel Oztop. 36 intermediate dynamics. 57 57–​61. offline look-​ahead prediction. 207–​8 subject's unified existence with. 154–​55. 35–​36. 178 of. 197–​98 perceptual outcome meeting of. tools. 73–​74 as two-​d imensional. See operating system PB. 204 counting of. 59f look-​ahead prediction perceptual structures in. 177–​82. 145f cells. 183–​87. 254–​55 activations. 73–​74 online prediction. prediction error. See parametric bias other robot. 27 OS. 248 of monkeys. predictive model in. 98 PDP Research Group. See also active Parallel Distributed Processing perception. 233f. 191–​93. 231–​35. 78. 36f bimodal neurons in. 36f overview. action changing reality of. E. 61–​62. damage to. 208 subject as separated from. 59f perception of. 58 Distributed Processing Research Group palpation. 79 parrots. 102f. 232 stimulation of. 144 perception. 60–​61 181f. 55. 22–​23 intention involvement of. 250. 46f 201f. 144–​45. 143. 56–​61. 11 optical constancy. 34–​35. complex. 266. 10–​12 parietal cortex. what pathway. mirror neurons in. 57 skilled behaviors for manipulating. 185f. 177 25. 76 as three-​d imensional. 198 visual objects self-​organization. 98–​100. See 56–​61. 195f. 201–​2. 55 . 36. 191–​93. See also manipulation.  303 Index 303 objectivity. 235 optical flow. 196 where pathway parametric bias (PB). 59f. 234f PCA. 244. 94. 208 shaking. 62f. 192 as attractive. 144 one-​step prediction. 95f past. 33–​37. 59f offline learning processes. 59f as information hub. 178f. 76. 192f objects. 76 subject iterative exchanges. 56–​57 operating system (OS). 35–​36. See principal component outfielder. 36f movements. 95f pastness. 237. See features. 153–​54 visual objects involvement open-​loop mode. (PDP) Research Group. action intention meeting of. 94–​95 analysis overregularization. 68 155f. chimps and. 46f also precuneus infants using. 101–​2.. 99f vectors. 197. 194–​9 6. 211 56–​61. 234f. 36f. 24f polarity. 58. 23–​42. 24f. 167f. 36f. 153–​57. 59f Poincaré section. 23f. 21 top-​down. 195f objectification. 153f. 206. 153–​54 Pfeifer. 32–​37. 192f. 176. 25 36f. 40f 163f. See also one-​step perchings. 25 of time. 159 perception-​to-​action mapping. 197–​98 being-​in-​t he-​world. 90. 143 of objects. 260 of. 144 prediction. 230–​39.. 36f pilots. self-​ objective world. 107 236f. 167f. embodiment of mind. 60–​61. 144 postdiction. 155f. 204–​6 193. 102. 23–​42. 23f predictive coding. 24–​25. 26–​29 predictive dynamics. See prefrontal cortex as online. 169f overview. Anne. 192–​93. See ventral premotor area 56–​61. poles. 7. 59f perceptual sequences. 29–​32 Yamabico. phenomenology. 153–​57. 95–​97. 248–​49 intention altered by reality Piaget. 238f perceptual constancy. 48. 129f. 166–​72. perseverative reaching. 128–​30. 71–​73. 24f.) subjective mind. 20 164–​65. 59–​6 0. 154 PFC. PMv. 26–​29 consequences. 26–​29.304 304 Index perception (Cont. 226 prediction periodicity. 153f. 144 155f. 177–​79. 233f. 24f. 56–​61. 157f phenomenological reduction. time perception. phase transitions. 240. 124 perceptual flows. 236. 73f cortex. 257–​58 98–​100. 33–​37. 33. 96f. 21–​42. Jean. 178f. 26–​29. consciousness and. cogito as separate from. 203f. 36 186–​87. 91f of square. 98–​101. 151–​73 . 166–​6 8 198. 23–​42. 24f. 36f 193f. predictive learning 40f. 157f direct experience in. 94 outcome. 95 postsynaptic neurons. 161–​72. in parietal precuneus. 22–​23. 247–​51 from actional subjective experiences. 55. 60–​61 Pick. 7. See premotor cortex parietal cortex meeting of. attractors. 106. 191–​9 6. 62–​63 of sensation. 186 phantom limbs. poverty of stimulus problem. 130f RNNs as responsible for. 176. J. R. 260 perceptual structures. perception-​to-​motor cycle. of limit cycle errors. 99f. 169f. 99f as offline. 40f experience as dependent on. 40f 186–​87. 248–​49 Pollack. 38. 258 posture. 231–​32. 207–​8. 59f PMC.. 36f.  38–​39 of perceptual sequences. 59f. 189f movements. 209–​15. 73 complex actions. direct manipulation of. 207f phase. 77–​78 Quest for cuRIOsity (QRIO). 210 presupplementary motor area. F. of monkey. prefrontal cortex 26 –​2 8. transitions between. 169f 16. rats. 184f. 167f. 211 bottom-​up. 31–​32 fast dynamics.. 72–​73. problem of interactionism. 69–​71. 52f. 51–​52. 59f. 73f 212f. 209f. 101–​2. 214f present. hippocampus of. recognition 53. 214f presynaptic neurons. Ramachandran. 212f. 22.  305 Index 305 about world. 170–​71 private states of MTRNNs performing. 243–​47 predictive model. 208 184f. 141–​42 primitive actions. 66–​67 206–​7. 26–​27. 185f preempirical time. 48 reactive behaviors. 59f. 206–​7. 60–​61. 144. 102f rake. 160f. 52f. 210 presentness. 52f. 226–​27 reconstruction. 74–​76 212f. 50f. 37–​39 of landmarks. 75–​76 action's circular causality principal component analysis with. 22 . 60 faster dynamics at. 153f. 124 slow dynamics. 207f rapid eye movement (REM) sleep SMA and. 197–​98. 165 primary visual cortex (V1). 57–​6 4. 221f recognition. stochastic Readiness Potential (RP). 77–​78 Rao. 214f role of. 191 See also frontopolar part of pure experience. 27 intermediate dynamics. 151–​73. 163f. 209f. 157f. 207f.. 209–​15. 158f. stimulations of. 49–​54. 55–​6 8. 225–​26. 183–​90. neurons. 73f 44–​45. 49–​53. 61. 266 Principles of Psychology in brain. 39 premotor cortex (PMC). 48. Rajesh. 206 consciousness. proprioception. 26–​27 protention. 179. protosigns. 54. 209–​15. 68 183–​87. 266. 210 pretend play. 149 (PCA). 62f. 60. 101 preverbal infants. 198 prefrontal cortex (PFC). 70f 221–​22. 59–​6 0. 73 209f. 72–​73. stimulation of. 63 50f. 22–​2 3. 206–​7. 206 probabilistic processes. 62 primary motor cortex (M1). 186. Pulvermuller. See also visual primitive movements. 65f (James). 155f. 62f. 76 developmental training. V.  152 RNNPB. self robot parametric bias Cartesian dualism freedom of. behavior-​based robotics. 129–​30. 204. 106. 186–​87 . C. 41–​42 refusal of deficiency. as self-​narrative. 261. 201f. 28. 149 characteristics of. 91f segmentation of. 129f. 193f. 207f system flow of. 249 201–​2. 153–​61. RNNs. 179–​82. 205. 229–​30. 128f MTRNN as analogous to. 116–​20. 257 characteristics in. 130f frame problem avoided by. 150. 186–​87 rostral-​caudal gradient. Jordan-​t ype recurrent Rizzolatti. 141–​43 248–​49. 181f reflective selves of. 157f. 229–​30 Rössler attractor. 153f. with subjective views. 216 models. 177–​82. 178f Rössler system.. See also arm robot. 178f. 107–​9. 177–​79. 33–​34. 160f neurorobotics recurrent neural network with robots. 5. 9–​13.. See recurrent neural 202. 176–​79. Hideo. See also retrograde axonal signaling cascaded recurrent neural mechanism. 90. 186 robotics. 192f. 12f. 167f. 129f. retrograde axonal signal. 107f learning in. 25. 183 see-​ers. See recurrent neural as gated. D. 124. 130f representation. J. 14t reflective pattern generator. 182–​83 as forward dynamics model. 127 Rumelhart. 164–​6 9. 116f. 197 Khepera. 124.. 57 rehearsal. 128–​30. 11. 108. 26–​27. 260 sleep phase Scheier. 207–​8 network. See Readiness Potential recursion. 64–​6 6. 126–​28. See also error reflective selves. 178f RP. 113. 14–​15. 12f rules. 65f. 228f. 264–​65 walking of. mobile robots. 181f as humanoid. 169f sand pile behavior. G. 216. 147f Schmidhuber. 221–​22. 158 overview. 56 understanding meaning. 124 111–​12. 221f. 195f. behavior-​ parametric biases (RNNPB). 222 network with parametric biases models. based robotics. 33 Sakata. See also other robot.34–​35 retention.306 306 Index recurrent neural networks (RNNs). 229f. 216–​17. 198 segmentation.. 264 networks prediction responsibility of. 256–​57 145–​48. 90. 19. 206. 245. 186. 216–​17 response facilitation with Schneider. 206–​7. neural network 76. 171 REM phase. 5–​6. 155f. 216 back-​propagation scheme refrigerator. schizophrenia. 177–​78 navigation problem. 227–​ distributed representation 30. See rapid eye movement scaffolding. 176. See also Yamabico. 191–​98. 119f.  195f model. 203–​4. 223f. 144–​45. 7. 224 semantically combinatorial language at PFC. sequential movements. 191–​9 6. sentences. 220. 185f 167f. 118–​20.. 112. 171. 130f. 210 sensationalism. 57–​61. 175–​98. 230. range of. 258 PMC. 234f single-​unit recording. 76. 206 203f. 135f. K. 192f of dynamical structure. 247 in navigation. 255–​56. 24 SMA. 14. 173. 52f. 169–​72 skilled behaviors. spoken grammar. Linda. 109–​10. 264 sinusoidal function. See also Soar. See also intention units as reflective. A. recursive structure of. 12f 133f. 11. 244–​45 PB. 203f. 74. See also General self-​organized criticality (SOC). 205. 163f. 50–​51. 188–​9 0. C. M. G. 264. 59.. 46 selves. 132–​36. 211 153–​57. 145 QRIO. Problem Solver 171–​72. 192 Simon. Herbert.  307 Index 307 self-​consciousness.. prediction of. 207f short-​term memory (STM). 70–​71. 13. 264 slow dynamics. 135f Siegelmann. 59f momentary. 157f. 128–​32. 155f. 226–​27 . 58. 98. 231–​35. 244–​45 Elman net generating. 33 206. 267 simulation theory. 146 action generation mirrored by. 245 sequence patterns. 207f of thought. 145f multiple timescales. 18–​19 sensory-​motor coordination. 97–​98. 170. 129f. 12–​13 175–​98.. in speech recognition system. 190 202. 68 self robot. See self-​organized criticality sensory aliasing problem. See dynamical systems approach also recurrent neural network applying. 76 as minimal. 92 disturbance of. 53–​54 203f. 256–​57 Sirigu. 218. 233f. 53. 207f shaking. 248. 132–​36. 206–​7. articulating. 15. 178f spontaneity. Hava. 169–​70. spiking neurons. 53–​54 Spencer-​Brown.. 7 with parametric biases of functional hierarchy. 131f See also neural network models sensory-​motor flow Spivey. 219. 191–​9 6 self-​organization. 133f. 100. in bound learning process. 246–​47 synesthesia SOC. 130. Smith. 203–​8. 153f. 67. 203–​8. 178f. 177–​79. 169f sensory-​motor sequences selfhood. Shima. 240 sensory-​g uided actions. sensory cortices. 216 MTRNN. 161–​72. 119f 194–​9 6. 51–​52. 63–​6 4 223. 134 Soon. 39 model. See supplementary motor area sensations. Keiji. steady phase. 70f. emergence through. 266–​67 230– ​40. 254–​55. 22–​23 synchronization. 40f 150. 168. 79. 37–​39 17f. 91f. See primitive actions. 38. 18–​20 object as separated from. 50f. 34 147f. 69–​75. 219–​30 subjective views. 36f. 238f subrecursive functions. (Gestalt). 88f symbolic dynamics. 25 perchings. 49. See also dynamical systems subjective mind approach. statistical learning. 73f. 149f synthetic robotics approach. 195f STM. 188 object iterative exchanges. 218 phenomenology. 90–​93. 236f. 37–​41. 182 surprise. 19–​20. 267 objective world's distinction from. 61. 37–​39 (SMA). 24f. 206–​7. 207f images in. 46 . 191–​9 6. 221f Sugita. 32 Tanaka. 147f spontaneous generation subjectivity. 245–​47 subject symbol systems. 40f tactile palpation. 260 subsumption architecture. 6. synesthesia. 192f. definition of. 244. 49–​54. 102. 172n2 overview. 263–​6 4 148–​50. 36. 15–​18. 9–​13. 37–​41. synthetic modeling approach. 248 synthesis. 141–​48. 247 stochastic transitions between superior parietal cortex. 34. 40f. 108. 87. 170 supplementary motor area characteristics of. 40f suspension of disbelief (epoché). 243 stretching and folding. 250.308 308 Index spontaneous behaviors. 7. 7. 83 subjective experiences. 221–​22. 172. 52f. 226 EEG activity. 37 63–​6 4. 38. 221f precuneus streams of consciousness. Yuuya. 36 synchrony. 23–​42. 170 193f. 42 M1 and. 23–​42. 13 overview. of imitative 107–​9. 24f. 7. Ron. See short-​term memory Sun. synthetic robotics experiments. 77–​78 flights. 107f actions. 141. 145–​48. 251–​55. embodiment actions influencing. 221–​22. 267 82. 34 subjective sense of time. 7. 159–​61. 36f. 12f. 63 25. 226 staged development. 71–​72 substantial parts. 88–​8 9 structural stability. 49 synthetic neurorobotics studies. 131–​32 object's unified existence with. 226 symbol grounding problem. 26–​29. of intention. 145f. 87–​8 8 structuring processes of whole symbols. states. 92f symbolic processes. 169. 160f. 102f. objective world as tied to. 70–​71 free will and.  70 (TPJ). thought segmentation tutoring. 91–​92. 36f usage-​based approach. 207f. of. Tohru. 250 vector field.  309 Index 309 tangency. unsteady phase. 108–​9 top-​down subjective intentions. toy. Francisco. 144–​45 107. 168–​6 9 TEO. 187. 248–​49 V2. 47. 169–​70 35–​36. 39–​41. 197–​98 vehicles. 42. M. 263 Vehicles: Experiments in Synthetic top-​down subjective view. 61 Tsuda. 146 240–​41. 182 transversal intentionality. 23 thinking segmented into. 99f 64–​65. 105f semantically combinatorial Uddén.. 45. See also U-​shaped development. 17f. 168. 171 training. 237–​39. 215. 63. 108 two-​stage model. 57–​61. 52f.. 238f. 46. 53–​54. See inferior temporal area transition rules. 98–​100. 259–​61 thoughts. 190 that which appears. 228f.. subjective sense of. 98–​100. 32 transitive actions. actions influenced by. 186–​87. J.. top-​down pathway. 24f Turvey. S. 92f 164–​65. J. 92f top-​down prediction. 104f. 38 See also developmental training Tanji. 105f. 215 Varela. top-​down projection. 248. 60–​61. 99f objective time time perception phenomenology. See primary visual cortex 26–​29. 35–​36. 36f chaos generating. 211 tutored sequences. 35 ventral premotor area (PMv). 227–​30. 89. See temporoparietal junction VIP. See inferior temporal area Turing limit. 71–​72 two-​d imensional objects. Ester. 229f thinking. 188. 191 time. 28. 164–​65. 222 temporality. 245 Tettamanti. 46 touching. 65 temporal patterns. 91–​92. 51–​52. Tani... 35. See ventral intraparietal area . 97–​98. 99. 117. 28 temporoparietal junction Trevena. 14t tearing neurons. 48 tokens. 207–​8. 266 Psychology (Braitenburg). 24–​25. 40f experiments. 35 ventral intraparietal area (VIP). 112. 95–​9 6 Thelen. 191 three-​d imensional objects. Shizuteru. 89f. 103–​6 touched. 206–​7. 207f language of. 76. 67 turn taking. 65 transition sequences. 103–​5. 206 transient parts. I. 254 172. 32. J. 27. 103–​5. 132. 13 V4. 10. 257–​58 top-​down forward prediction. 226 TE. vector flow. 48 tools. 104f. 145 Ueda. 14–​15. 176. 65f TPJ. 59f Van de Cruys. V1. 146 universal grammar. 56. 153–​61. 133. 131–​32. 213–​15. VP trajectories. 94 visual objects. 157f. 39 158f. 132–​36. Paul. VP flow. 153f. 126–​28. 256–​57. 6 See also multiple-​t imescale Werbos. 45f. See 155f. 44–​54. S. 191 virtue. 63 Wernicke's area. 162–​65. vision 162–​65. 46f. 153f. 190 visual imagery. voluntary actions. 69–​71. branching. 46. Yuuichi. 208. 52f 155f. Claes. 47f words. visual receptive field. 160f. 70f visual alphabets. 46f. 152–​6 0. will. 45f. 58 visual cortex. 67. See visuo-​ 157f. 155f. 47f. 161 visuo-​proprioceptive mapping. 162–​72. Ludwig. 173.. 50f. 50–​53. 228–​29 World War II (WWII). 156–​57. See also free will 167f. 193–​96. 157f walking reflex. 56 w-​judgment time. look-​ahead prediction. 46f. 157f visuo-​proprioceptive flow RNN. 153–​57. 160f walking. 131f 155f. 159–​61. 210 intentionality of. trajectories. 163f mobile robot with. 135f. 155. water hammers. 133f. 145–​48. 153f. 169f. Daniel. SMA. 158f. 160f voluntary sequential movements in neurodynamic structure. 113. 261 what pathway. 47 Wolpert. 193f. 195f Wittgenstein. 45. See also error recurrent neural network back-​propagation scheme Yen. 153f. 98 Yamashita. 143 prediction.310 310 Index virtual-​reality mirror box. 56–​57 visual palpation. 157f. 154–​57. 147f. 18 visual agnosia. 158f von Hofsten. 160f proprioceptive trajectories symbol grounding problem. 52f 157–​59. 155f. 173 visual recognition. 49 . 214f 153–​62. 62 164. 47f. C. 157f visuo-​proprioceptive (VP) navigation experiments with. 45. 176 visuo-​proprioceptive (VP) flow. 163f Merleau-​Ponty on. 4–​5. 34–​35 where pathway. 163f. 144 Yamabico. 128f trajectories of. 44–​49.   311 . 312 .