<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"></head><body dir="auto"><div dir="auto">Yes, the results that LLMs produce are fascinating. Yet there is something missing. It could be the grounding that comes from embeddedness in a real environment.</div><div dir="auto"><br></div><div dir="auto">William James writes in his "Essays in Radical Empiricism":</div><div dir="auto">"Most thought-paths, it is true, are substitutes for nothing actual; they end outside the real world altogether, in wayward fancies, utopias, fictions or mistakes [...] The objective nucleus of every man's experience, his own body, is [...] a continuous percept; and equally continuous as a percept (though we may be inattentive to it) is the material environment of that body, changing by gradual transition when the body moves. But the distant parts of the physical world are at all times absent from us, and form conceptual objects merely, into the perceptual reality of which our life inserts itself at points discrete and relatively rare."</div><div dir="auto">https://brocku.ca/MeadProject/James/James_1912/James_1912_02.html</div><div dir="auto"><br></div><div dir="auto">-J.</div><div dir="auto"><br></div><div><br></div><div align="left" dir="auto" style="font-size:100%;color:#000000"><div>-------- Original message --------</div><div>From: Russ Abbott <russ.abbott@gmail.com> </div><div>Date: 5/29/23 8:01 PM (GMT+01:00) </div><div>To: Jochen Fromm <jofr@cas-group.net> </div><div>Cc: The Friday Morning Applied Complexity Coffee Group <friam@redfish.com> </div><div>Subject: Re: [FRIAM] A distinguishing feature of living entities </div><div><br></div></div><div dir="ltr"><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:#000000" class="gmail_default">I saw that Rodney Brooks video. He claimed that transformer-based software has no semantic content. I think that's an exaggeration. The semantic content is defined by the token embeddings. So many of the explanations of token embeddings overcomplicate the basic idea. Look up <a href="https://www.google.com/search?q=word-to-vec+&tbm=isch&ved=2ahUKEwju7IP4i5v_AhUEK0QIHbXrDycQ2-cCegQIABAA&oq=word-to-vec+&gs_lcp=CgNpbWcQAzIECCMQJzIECAAQHjIECAAQHlD0EVj0EWDpGWgAcAB4AIABWogBrwGSAQEymAEAoAEBqgELZ3dzLXdpei1pbWfAAQE&sclient=img&ei=S-Z0ZK6bDITWkPIPtde_uAI&bih=956&biw=1781&cs=0&rlz=1C1RXQR_enUS1008US1008#imgrc=aSLVbcgBsmbHKM">wordToVec</a> and read some of the articles. Word-to-Vec was around before transformers. But transformers are based on that idea. (One of the keys to transformers is that the embedding space, including the features themselves, is generated as part of the training.) The embedding of all tokens in the GPT embedding space is the semantics. It's amazing the extent to which that idea can be pushed and the results LLMs produce!</div><div><div data-smartmail="gmail_signature" class="gmail_signature" dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><font><u style="color:rgb(33,33,33);font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:20px"><br></u></font></div><div dir="ltr"><font><u style="color:rgb(33,33,33);font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:20px"> </u></font><span style="color:rgb(33,33,33);font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:24.75px"> </span>-- Russ <br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div class="gmail_attr" dir="ltr">On Mon, May 29, 2023 at 10:13 AM Jochen Fromm <<a href="mailto:jofr@cas-group.net">jofr@cas-group.net</a>> wrote:<br></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div dir="auto"><div dir="auto">Yes, Rodney Brooks said something similar. He said "GPTs have no understanding of the words they use, no way to connect those words, those symbols, to the real world. A robot needs to be connected to the real world and its commands need to be coherent with the real world. Classically it is known as the 'symbol grounding problem""</div><div dir="auto"><a href="https://rodneybrooks.com/what-will-transformers-transform/">https://rodneybrooks.com/what-will-transformers-transform/</a></div><div dir="auto"><br></div><div dir="auto">One could argue that this form of connectedness and embeddedness leads eventually to self-awareness. First physical embeddedness, then social embeddedness and finally self-awareness</div><div dir="auto"><br>1. Physical Embeddedness:<br>Agents who are embedded in a physical world are aware of the world and move in it. To be embedded they need to be embodied. Embeddedness leads to a grounding and a unique point of view<br><a href="https://iep.utm.edu/husspemb/">https://iep.utm.edu/husspemb/</a><br><br>2. Social Embeddedness:<br>Agents who are embedded in a world of social actors are aware of other agents in the world and interact with them<br><a href="https://mitpress.mit.edu/9780262518581/the-new-science-of-the-mind/">https://mitpress.mit.edu/9780262518581/the-new-science-of-the-mind/</a><br><br>3. Self-Awareness:<br>Agents who are embedded in two worlds, the physical world and the world of language become aware that they are actors in a world of social actors by creating a link between the embodiment in the physical world (the body) and the embodiment in the world of language (the name or I)<br></div><div dir="auto"><br></div><div dir="auto">-J.</div><div dir="auto"><br></div><div><br></div><div style="font-size:100%;color:rgb(0,0,0)" dir="auto" align="left"><div>-------- Original message --------</div><div>From: Russ Abbott <<a href="mailto:russ.abbott@gmail.com">russ.abbott@gmail.com</a>> </div><div>Date: 5/29/23 7:08 AM (GMT+01:00) </div><div>To: The Friday Morning Applied Complexity Coffee Group <<a href="mailto:friam@redfish.com">friam@redfish.com</a>> </div><div>Subject: [FRIAM] A distinguishing feature of living entities </div><div><br></div></div><div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">While watching my two little dogs run around our house, it struck me that a feature that distinguishes living from non-living entities is the apparent effortlessness with which living ones navigate the world. Imagine how difficult it would be to build a robot that could navigate the world so effortlessly. To make the comparison a bit simpler, imagine how difficult it would be to build a robotic cockroach. </div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">When I asked ChatGPT whether anyone has built a robotic cockroach, it came up with these examples. (I haven't checked to see whether these are real projects.)</div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div class="gmail_default"><ul><li style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px;padding-left:0.375em;white-space:pre-wrap;background-color:rgb(247,247,248)"><p style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px"><font color="#000000" face="verdana, sans-serif">DASH: The Dynamic Autonomous Sprawled Hexapod (DASH) robot, developed at the University of California, Berkeley, was inspired by the rapid locomotion of cockroaches. It has six legs and can move quickly on various terrains using a simple control mechanism.</font></p></li></ul></div></div><div><div class="gmail_default"><ul><li style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px;padding-left:0.375em;white-space:pre-wrap;background-color:rgb(247,247,248)"><p style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px"><font color="#000000" face="verdana, sans-serif">Harvard RoboBee: Although not specifically modeled after a cockroach, the Harvard RoboBee project aims to develop small, insect-like robots. These tiny flying robots are inspired by the mechanics and flight capabilities of insects and demonstrate similar agility and maneuverability.</font></p></li></ul></div></div><div><div class="gmail_default"><ul><li style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px;padding-left:0.375em;white-space:pre-wrap;background-color:rgb(247,247,248)"><p style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px"><font color="#000000" face="verdana, sans-serif">iSprawl: The iSprawl robot, developed at the University of California, Berkeley, was inspired by cockroaches' ability to squeeze through small spaces. It uses a compliant body design and six legs to navigate tight and cluttered environments.</font></p></li></ul></div></div><div><div class="gmail_default"><ul><li style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px;padding-left:0.375em;white-space:pre-wrap;background-color:rgb(247,247,248)"><p style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px"><font color="#000000" face="verdana, sans-serif">VelociRoACH: Developed at the University of California, Berkeley, the VelociRoACH is a fast-running robot designed to mimic the high-speed locomotion of cockroaches. It utilizes a legged design and has demonstrated impressive speed and agility.</font></p></li></ul></div></div></blockquote><div dir="ltr"><div dir="ltr" class="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><font><u style="color:rgb(33,33,33);font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:20px"><span class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"></span></u></font><span class="gmail_default" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"></span><span class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">These mainly explore locomotion. Besides locomotion, cockroaches notice when someone enters an area where they are exposed. They quickly scuttle off to some hiding place. How do they sense the presence of a new being? How do they know where the hiding places are? How do they know how to move in the right direction? How do they know how to avoid small obstacles and fires? Etc. </span></div><div dir="ltr"><span class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></span></div><div dir="ltr"><span class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">One can argue that these capabilities are hard-wired in. But that doesn't make it any easier. These are still capabilities they have, that would be a challenge to build.</span></div><div dir="ltr"><span class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></span></div><div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">I became amazed at how well-connected living entities are to their environments. They quickly and easily extract and use information from their environment that is important to their survival. </div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">Man-made robots have nowhere near that level of embeddedness and environmental integration. </div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">Was it Rodney Brooks who said that we should build that sort of connectedness before worrying about building intelligence into our robots? Today that struck me as an important insight.</div></div><div dir="ltr"><u style="color:rgb(33,33,33);font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:20px"><span class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"></span></u></div><div dir="ltr"><font><u style="color:rgb(33,33,33);font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:20px"> </u></font><span style="color:rgb(33,33,33);font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:24.75px"> </span>-- Russ Abbott <br><span class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"></span>Professor Emeritus, Computer Science<br>California State University, Los Angeles</div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div></blockquote></div>
</body></html>