<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 5/29/23 1:32 PM, Marcus Daniels
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:35451027-076F-4829-88C8-905EA7423537@snoutfarm.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
I believe that Zoox, Cruise, Waymo, and Tesla all use these
techniques.
</blockquote>
I was involved in a project in the 90s where I was introduced to the
degenerate form: Trigram Analysis. It was fascinatingly effective
*and* fairly easy to understand/explain through a few examples
(especially when compressing out white-space and therefore picking
up word endings juxtaposed with word-beginnings ( "ng z" or "ng q"
or "ls a" or "a sl") providing surprisingly good word-word context
which in turn correlates "forward" to limited phrase context (i.e.
such as when "ng" words (verbs?) precede words starting with any
particular unusual consonant).<br>
<br>
<blockquote type="cite"
cite="mid:35451027-076F-4829-88C8-905EA7423537@snoutfarm.com">
<div><br>
</div>
<div>Also my dog does not consider the Roomba to be alive. She
performed some experiments.</div>
</blockquote>
<p>My previous dog ( a little skitchy to start with ) never
liked/trusted our roboVac (Roomba-knockoff) and I would say
treated it as if it were alive. My previous cat (very old, very
experienced, still active hunter at 20) never gave it a second
look. My new dog (puppy who grew up with Babadook (our pet name
for the roboVac)) is almost entirely non-plussed by the roboVac
*except* when I drive it (by remote) to keep following/confronting
him then he wants to scare/play it. My new cat (also 1 year old)
also ignores Babadook even when *I* drive it... but she *does*
simply leave the room (the entire floor) probably because the high
pitched whine is irritating? <br>
</p>
<p>FWIW my dog *does* take interest/exception to things happening on
our 40" TV screen with/without familiar sounds (e.g. barking) but
his interest is fleeting and can't be fooled by bringing back the
same visuals, including images/videos of himself. The cat walks
across the shelf in front of the TV noticing nothing more than
perhaps the warmth emanating from the box? I sit and stare at
for hours, getting deeply involved in the personal lives of the
fictional ensemble characters who show up faithfully week by week
(or hour by hour if I'm binging)...</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:35451027-076F-4829-88C8-905EA7423537@snoutfarm.com">
<div>
<div><br>
</div>
<div>
<div style="display: block;" class="">
<div style="-webkit-user-select: all; -webkit-user-drag:
element; display: inline-block;" class="apple-rich-link"
draggable="true" role="link"
data-url="https://arxiv.org/pdf/2106.08417v3.pdf">
<a style="border-radius:10px;font-family:-apple-system,
Helvetica, Arial,
sans-serif;display:block;-webkit-user-select:none;width:300px;user-select:none;-webkit-user-modify:read-only;user-modify:read-only;overflow:hidden;text-decoration:none;"
class="lp-rich-link" rel="nofollow"
href="https://arxiv.org/pdf/2106.08417v3.pdf" dir="ltr"
role="button" draggable="false" width="300"
moz-do-not-send="true">
<table
style="table-layout:fixed;border-collapse:collapse;width:300px;background-color:#E9E9EB;font-family:-apple-system,
Helvetica, Arial, sans-serif;"
class="lp-rich-link-emailBaseTable" width="300"
cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td vertical-align="center">
<table style="font-family:-apple-system,
Helvetica, Arial,
sans-serif;table-layout:fixed;background-color:rgba(233,
233, 235, 1);" class="lp-rich-link-captionBar"
width="300" cellspacing="0" cellpadding="0"
bgcolor="#E9E9EB">
<tbody>
<tr>
<td style="padding:8px 0px 8px 0px;"
class="lp-rich-link-captionBar-textStackItem">
<div style="max-width:100%;margin:0px
16px 0px 16px;overflow:hidden;"
class="lp-rich-link-captionBar-textStack">
<div
style="word-wrap:break-word;font-weight:400;font-size:11px;overflow:hidden;text-overflow:ellipsis;text-align:left;"
class="lp-rich-link-captionBar-textStack-bottomCaption-leading">
<a rel="nofollow"
href="https://arxiv.org/pdf/2106.08417v3.pdf"
style="text-decoration: none"
draggable="false"
moz-do-not-send="true"><font
style="color: rgba(0, 0, 0, 1);"
color="#000000">arxiv.org</font></a></div>
</div>
</td>
<td style="padding:0px 12px 0px 0px;"
class="lp-rich-link-captionBar-rightIconItem"
width="32">
<a rel="nofollow"
href="https://arxiv.org/pdf/2106.08417v3.pdf"
draggable="false"
moz-do-not-send="true"><img
src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAGwAAABsCAQAAAAlb59GAAANBGlDQ1BrQ0dDb2xvclNwYWNlR2VuZXJpY0dyYXlHYW1tYTJfMgAAWIWlVwdck9cWv9/IAJKwp4ywkWVAgQAyIjOA7CG4iEkggRBiBgLiQooVrFscOCoqilpcFYE6UYtW6satD2qpoNRiLS6svpsEEKvte+/3vvzud//fPefcc8495557A4DuRo5EIkIBAHliuTQikZU+KT2DTroHyMAYaAN3oM3hyiSs+PgYyALE+WI++OR5cQMgyv6am3KuT+n/+BB4fBkX9idhK+LJuHkAIOMBIJtxJVI5ABqT4LjtLLlEiUsgNshNTgyBeDnkoQzKKh+rCL6YLxVy6RFSThE9gpOXx6F7unvS46X5WULRZ6z+f588kWJYN2wUWW5SNOzdof1lPE6oEvtBfJDLCUuCmAlxb4EwNRbiYABQO4l8QiLEURDzFLkpLIhdIa7PkoanQBwI8R2BIlKJxwGAmRQLktMgNoM4Jjc/WilrA3GWeEZsnFoX9iVXFpIBsRPELQI+WxkzO4gfS/MTlTzOAOA0Hj80DGJoB84UytnJg7hcVpAUprYTv14sCIlV6yJQcjhR8RA7QOzAF0UkquchxEjk8co54TehQCyKjVH7RTjHl6n8hd9EslyQHAmxJ8TJcmlyotoeYnmWMJwNcTjEuwXSyES1v8Q+iUiVZ3BNSO4caViEek1IhVJFYoraR9J2vjhFOT/MEdIDkIpwAB/kgxnwzQVi0AnoQAaEoECFsgEH5MFGhxa4whYBucSwSSGHDOSqOKSga5g+JKGUcQMSSMsHWZBXBCWHxumAB2dQSypnyYdN+aWcuVs1xh3U6A5biOUOoIBfAtAL6QKIJoIO1UghtDAP9iFwVAFp2RCP1KKWj1dZq7aBPmh/z6CWfJUtnGG5D7aFQLoYFMMR2ZBvuDHOwMfC5o/H4AE4QyUlhRxFwE01Pl41NqT1g+dK33qGtc6Eto70fuSKDa3iKSglh98i6KF4cH1k0Jq3UCZ3UPovfi43UzhJJFVLE9jTatUjpdLpQu6lZX2tJUdNAP3GkpPnAX2vTtO5YRvp7XjjlGuU1pJ/iOqntn0c1biReaPKJN4neQN1Ea4SLhMeEK4DOux/JrQTuiG6S7gHf7eH7fkQA/XaDOWE2i4ugg3bwIKaRSpqHmxCFY9sOB4KiOXwnaWSdvtLLCI+8WgkPX9YezZs+X+1YTBj+Cr9nM+uz/+yQ0asZJZ4uZlEMq22ZIAvUa+HMnb8RbEvYkGpK2M/o5exnbGX8Zzx4EP8GDcZvzLaGVsh5Qm2CjuMHcOasGasDdDhVzN2CmtSob3YUfg78Dc7IvszO0KZYdzBHaCkygdzcOReGekza0Q0lPxDa5jzN/k9MoeUa/nfWTRyno8rCP/DLqXZ0jxoJJozzYvGoiE0a/jzpAVDZEuzocXQjCE1kuZIC6WNGpF36oiJBjNI+FE9UFucDqlDmSZWVSMO5FRycAb9/auP9I+8VHomHJkbCBXmhnBEDflc7aJ/tNdSoKwQzFLJy1TVQaySk3yU3zJV1YIjyGRVDD9jG9GP6EgMIzp+0EMMJUYSw2HvoRwnjiFGQeyr5MItcQ+cDatbHKDjLNwLDx7E6oo3VPNUUcWDIDUQD8WZyhr50U7g/kdPR+5CeNeQ8wvlyotBSL6kSCrMFsjpLHgz4tPZYq67K92T4QFPROU9S319eJ6guj8hRm1chbRAPYYrXwSgCe9gBsAUWAJbeKq7QV0+wB+es2HwjIwDyTCy06B1AmiNFK5tCVgAykElWA7WgA1gC9gO6kA9OAiOgKOwKn8PLoDLoB3chSdQF3gC+sALMIAgCAmhIvqIKWKF2CMuiCfCRAKRMCQGSUTSkUwkGxEjCqQEWYhUIiuRDchWpA45gDQhp5DzyBXkNtKJ9CC/I29QDKWgBqgF6oCOQZkoC41Gk9GpaDY6Ey1Gy9Cl6Dq0Bt2LNqCn0AtoO9qBPkH7MYBpYUaYNeaGMbEQLA7LwLIwKTYXq8CqsBqsHlaBVuwa1oH1Yq9xIq6P03E3GJtIPAXn4jPxufgSfAO+C2/Az+DX8E68D39HoBLMCS4EPwKbMImQTZhFKCdUEWoJhwlnYdXuIrwgEolGMC98YL6kE3OIs4lLiJuI+4gniVeID4n9JBLJlORCCiDFkTgkOamctJ60l3SCdJXURXpF1iJbkT3J4eQMsphcSq4i7yYfJ18lPyIPaOho2Gv4acRp8DSKNJZpbNdo1rik0aUxoKmr6agZoJmsmaO5QHOdZr3mWc17ms+1tLRstHy1ErSEWvO11mnt1zqn1an1mqJHcaaEUKZQFJSllJ2Uk5TblOdUKtWBGkzNoMqpS6l11NPUB9RXNH2aO41N49Hm0appDbSrtKfaGtr22iztadrF2lXah7QvaffqaOg46ITocHTm6lTrNOnc1OnX1df10I3TzdNdortb97xutx5Jz0EvTI+nV6a3Te+03kN9TN9WP0Sfq79Qf7v+Wf0uA6KBowHbIMeg0uAbg4sGfYZ6huMMUw0LDasNjxl2GGFGDkZsI5HRMqODRjeM3hhbGLOM+caLjeuNrxq/NBllEmzCN6kw2WfSbvLGlG4aZpprusL0iOl9M9zM2SzBbJbZZrOzZr2jDEb5j+KOqhh1cNQdc9Tc2TzRfLb5NvM2834LS4sIC4nFeovTFr2WRpbBljmWqy2PW/ZY6VsFWgmtVludsHpMN6Sz6CL6OvoZep+1uXWktcJ6q/VF6wEbR5sUm1KbfTb3bTVtmbZZtqttW2z77KzsJtqV2O2xu2OvYc+0F9ivtW+1f+ng6JDmsMjhiEO3o4kj27HYcY/jPSeqU5DTTKcap+ujiaOZo3NHbxp92Rl19nIWOFc7X3JBXbxdhC6bXK64Elx9XcWuNa433ShuLLcCtz1une5G7jHupe5H3J+OsRuTMWbFmNYx7xheDBE83+566HlEeZR6NHv87unsyfWs9rw+ljo2fOy8sY1jn41zGccft3ncLS99r4lei7xavP709vGWetd79/jY+WT6bPS5yTRgxjOXMM/5Enwn+M7zPer72s/bT+530O83fzf/XP/d/t3jHcfzx28f/zDAJoATsDWgI5AemBn4dWBHkHUQJ6gm6Kdg22BecG3wI9ZoVg5rL+vpBMYE6YTDE16G+IXMCTkZioVGhFaEXgzTC0sJ2xD2INwmPDt8T3hfhFfE7IiTkYTI6MgVkTfZFmwuu47dF+UTNSfqTDQlOil6Q/RPMc4x0pjmiejEqImrJt6LtY8Vxx6JA3HsuFVx9+Md42fGf5dATIhPqE74JdEjsSSxNUk/aXrS7qQXyROSlyXfTXFKUaS0pGqnTkmtS32ZFpq2Mq1j0phJcyZdSDdLF6Y3ZpAyUjNqM/onh01eM7lriteU8ik3pjpOLZx6fprZNNG0Y9O1p3OmH8okZKZl7s58y4nj1HD6Z7BnbJzRxw3hruU+4QXzVvN6+AH8lfxHWQFZK7O6swOyV2X3CIIEVYJeYYhwg/BZTmTOlpyXuXG5O3Pfi9JE+/LIeZl5TWI9ca74TL5lfmH+FYmLpFzSMdNv5pqZfdJoaa0MkU2VNcoN4J/SNoWT4gtFZ0FgQXXBq1mpsw4V6haKC9uKnIsWFz0qDi/eMRufzZ3dUmJdsqCkcw5rzta5yNwZc1vm2c4rm9c1P2L+rgWaC3IX/FjKKF1Z+sfCtIXNZRZl88sefhHxxZ5yWrm0/OYi/0VbvsS/FH55cfHYxesXv6vgVfxQyaisqny7hLvkh688vlr31fulWUsvLvNetnk5cbl4+Y0VQSt2rdRdWbzy4aqJqxpW01dXrP5jzfQ156vGVW1Zq7lWsbZjXcy6xvV265evf7tBsKG9ekL1vo3mGxdvfLmJt+nq5uDN9VsstlRuefO18OtbWyO2NtQ41FRtI24r2PbL9tTtrTuYO+pqzWora//cKd7ZsStx15k6n7q63ea7l+1B9yj29OydsvfyN6HfNNa71W/dZ7Svcj/Yr9j/+EDmgRsHow+2HGIeqv/W/tuNh/UPVzQgDUUNfUcERzoa0xuvNEU1tTT7Nx/+zv27nUetj1YfMzy27Ljm8bLj708Un+g/KTnZeyr71MOW6S13T086ff1MwpmLZ6PPnvs+/PvTrazWE+cCzh0973e+6QfmD0cueF9oaPNqO/yj14+HL3pfbLjkc6nxsu/l5ivjrxy/GnT11LXQa99fZ1+/0B7bfuVGyo1bN6fc7LjFu9V9W3T72Z2COwN358OLfcV9nftVD8wf1Pxr9L/2dXh3HOsM7Wz7Kemnuw+5D5/8LPv5bVfZL9Rfqh5ZParr9uw+2hPec/nx5MddTyRPBnrLf9X9deNTp6ff/hb8W1vfpL6uZ9Jn739f8tz0+c4/xv3R0h/f/+BF3ouBlxWvTF/tes183fom7c2jgVlvSW/X/Tn6z+Z30e/uvc97//7fCQ/4Yk7kYoUAAABsZVhJZk1NACoAAAAIAAQBGgAFAAAAAQAAAD4BGwAFAAAAAQAAAEYBKAADAAAAAQACAACHaQAEAAAAAQAAAE4AAAAAAAAA2AAAAAEAAADYAAAAAQACoAIABAAAAAEAAABsoAMABAAAAAEAAABsAAAAAC3W2nAAAAAJcEhZcwAAITgAACE4AUWWMWAAAAAcaURPVAAAAAIAAAAAAAAANgAAACgAAAA2AAAANgAABRSxLH/cAAAE4ElEQVR4AexZTWgVVxT+3suPbTXE1KSQ5vmz0o0krbTWgtpuEmk20kb8W1hNWim66FoaKWiNG6VFpVUpSLJo04okC0VdiAoGxIVoREgoBGqjEmhjjNG0Ic30zJk7983MvcOb9+ZO3cy8xdw59zvf+bl37j1zH5BeaQbSDKQZSDOQZiDNQJqBNANpBtIMpBlIM5BmIM1AmoFEMpDBSuxEF85hEA8xCYt+k9QaJEkX9axEJhG7CZK+jnb8jDEOxQ4n7DeGXkIuStATY9RZtKIfM6Gh6EKcIY1WZI35YJyoHJ9iyBfSOC7gMHZgHVagFjX0W0YTcC1JDlPPXz7sEE3OcuM+GSBswbDH0UF04u0Co5AlxFe449EaRosBTwxS5PCrdG8aJ/FWUdxN+AHTUv8sckVpJwhuwxPh1jMcRX2oJfv9CrvqSfOZYHmCTWGw/08+D8dlrrs9Qb2icUEf2AKJrEc35gTbccyT8pfQqMUN4cgomj32f6T9Sp1QusCWYArHPJrNGBWMA7TcvKQrJ9fAy6jz+dBDzh3wSewHXWBdJD3jQ9bhkghtSJMcHzSZhxxGhAMHlApiDfU8QmXAsBpYJW/i7wZwGUqKjbXIgjruAbDpxzqxuM9ij5b6Nrm1OdAzRTL/tZ0kt/wi8bQHs9RnkRX/XNCCzQkrcZPNzirOuzY+p/5r7kPo3X5Dd4X0bhah3VRGPkTBhNhZCeeognCuVxXS+fgdpwpUEhWE+AOqrku2Q6yRJ1xB0vc2Hi2LagvnOoR7mglTFsmNCgVViS2yYukUltoUVAKCBkywuT6xZFRxUaQLrRTjK2iSN0nFDPrY1gQWS1lijbNs6gEVtO5VZyy0DtrVulxavtfgAdu74JMm8LCBzcz5tmPQVmqXsvFGrYbrzWEEa5Zm8aZtTCAaSZkVW3KPlLgNO7T7npLKlUe9r6fFxqIQPtQo2Nu9RZajvbUagsKirWxiCg0aaK1mAdHANKJy2pCdXeu0phdkzd4BLWzT9hoQZui0wjbgfwviEi/DALNadBqyMITMLrwssp7QGclHTP/CaHG6DU9FWBY+CQnLfodfMKo1FBGro5fJv4/F4VWuos8UeySc3zlvl9I+waheRW5AUC2+chuZ6zR68H4s1tX4TQZl0afqmxq2tfgJTt3RyNhpVGtQMUXtTH2XWarE1LiNzzC/BN4s9gVOsnYHWBbgC95CLIyLWvEu228P4Aw8OjXAfsGUw0E8ZlOjBSpC1XQDrrCmOwktXA8sCxX00WP3juJrOZL7WdKn0sWTlIlzjVUemgqq6q76vn89naHNj/FnIKxpLFfQZyj4Nl/KVrHWhOndrIlpx2V5mvekmG3zNTqPyo+T29qXJ5Mt9YQxK84h85WkBMdpdLBDF+NQoJFqEzeY/P0O1Apfb+Yia3foO0uVfsekR0pVp3foS/ytCWsW70TmPML630bGRwKeZ9Lg2hVJlUDv4RdNUPaoHY1KQbjdzHG+CI0IUGex/SACUoW8QUc2A3iuCW2kqM1iPTM4G45qpUSJs7Srq1cUun7a2vvxDf5RQmuJoi4xy1n/kXw20nD+tqstgWsXHaJtwBIspRMS95zXWTq6i2RbxIFNFqlVAO644hQ0+RXt3wJaoHCe0nvkVOUZ7PWM2VjBP/yCdqqFdkGjNuA/AAAA//811nvAAAAFPElEQVTtWW1oVWUc/92r220WrOlGTTdbH4rINUso+zCiQE1DiNBlQqPaxhD6JAV9UCuK5ocGlb1YIZGDYpG1kqkgVAS9SGU6ixiLhMWYWa3mNKdr7PZ7/ue5957Xe55z793ywzkD7zn/5/f7/V+ec543AbMrDfVXJWDrXv07HkJO4lOcQa1GJVCGZ0VHcTeHcJFFZvxUaUsoMQpgQkSXRqEQu5WsB1yc10Sp32U1eVwqzAkTqDnmlIjeZE4g8kacxzsuRgp34SjOIWqJlEyjxDDqUizycUBE74ygUoZvMYKFLkaKqf6JT1xWs8c7JIYBM7Apql9EO03hxD1DxhoXfhm+xjS6UeGymz12SgyFvMR59F8U0e48CGfTbZjC6w5TJd6ixo9Y6bBGeeiWGF6IQgnHtovooXCgICowiJ9xuQ29DsP4l2NiymaLentIYmiPSsuPXy6iY0h6YOUeC7ALF3F71r5Q+uoYbs5awm/meyBJjEkMyz0tRRnmcc5Ss88Km0qKc9EXeMNmsW5X8Tt6Kmu9D6dxAdvhDTUL8bnZi8+wifNe7loh/scxL2cqzV2fCO/QYg3oYsAq1WFXL15Jy3ew+rEG7zHJI1gWMYQKrX2KL2+d5u4Qb30RlQzgbSJsDbYVOCtP36DNM8L18JW5QfQ24nfOZI8VVOMFaMf34mMcl4nacXlqM4g0IqQSkyLdJLzd/G5u9VHYwB7aSvtV2E/057jOB2NuWokePC/wJvE9iUpzsjmyV8RfzUO4Gn/gML+mh/AbJvCo6yXNQwxtekV894biCgKsE/HzqA5k7+ertwYfE3cYDYGo6A3V+Ed83xOdasJI4ITIdwWAO9j6E9eB4+hAIgBTmPk58XuixKq2WDaLg3NYbLNlbq/ly6dGyX4syZgi/y5BjQ9nMYtlttHxIZuZklxRKBd7PfAkB4o0x8MHPS3mhmvwC5dc3tR6xOdgQaOrsfe14mQGq12Mx2nfx7GwmKsax6jiTm01ZsTnvcVIm3DfFzfDejdtMRo5KW80IYdgrNR+sPVaFX4VfwdCmCVoruNmX72OfdlPuZyT8KISKCuJTGoLRC+BD8XXGdSXSD+vTKs4S3P9Z13ehbGym63jvStIldoTWnm79lTMl6ulzH72iMMZtAbCr+Bq782Qz72M64pRWH1jF8rsF1r117XH3ji79ykubNXrOI37AxxtYWv4AYAaSTsCFFqornwcMez7AJmo5hoMidtpbPGlqjOSDa4WFabz2kTLUadJP3XqtIZsA4kvsPTGOpyU1NJ4OjuMZLw0s2XEswPzJlbOVzFt25Ja/AQVFTZND3UZybn8rdfTdRoHXXV9l0E96QnFmxjkANU53dfggE5rcG7GQk+cNFTjSx3EiGPK3sX1Yub8N8fzS6wef+GlHASr2NNWb32VZ7FtI8zWbQov60BmuNDKJeN3wOaXGPReW8VXi7f1OJimqtl0MVt5iW4L/tbJneW2MJec26laygZdtWRau3J1Ut8SBJtrez2shZbqk0nshrXLNo2iiQxrd674+/6/L8s/4Lv1BGB9IcexDbd4RksnM0HENlhnGRZrCGudkEvjaT4eyY6TVqBjHOG6uD5pxvUcCqr418ATq2ZauthinRNayDSZbZ4J4tLIS6JIYj0+4vF2JlyT3yky1pfwfGQWy7GIC6VefTaYL7XTPHfsKNnOYBYTckon+D9aD2MnPuBZyagcx0xx1jqJAVp2sqUx5Bt0qsVPcQXiCsQViCsQVyCuQFyBuAJxBeIKxBWIKxBXIK5AXAHzCvwH9WAIYRB4imoAAAAASUVORK5CYII="
draggable="false"
style="pointer-events:none
!important;display:inline-block;width:32px;height:32px;"
class="lp-rich-link-captionBar-rightIcon" data-unique-identifier=""
moz-do-not-send="true" width="32"
height="32"></a></td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</a></div>
</div>
<br>
<div dir="ltr">
<blockquote type="cite">On May 29, 2023, at 11:01 AM, Russ
Abbott <a class="moz-txt-link-rfc2396E" href="mailto:russ.abbott@gmail.com"><russ.abbott@gmail.com></a> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif;font-size:small;color:#000000">I
saw that Rodney Brooks video. He claimed that
transformer-based software has no semantic content. I
think that's an exaggeration. The semantic content is
defined by the token embeddings. So many of the
explanations of token embeddings overcomplicate
the basic idea. Look up <a
href="https://www.google.com/search?q=word-to-vec+&tbm=isch&ved=2ahUKEwju7IP4i5v_AhUEK0QIHbXrDycQ2-cCegQIABAA&oq=word-to-vec+&gs_lcp=CgNpbWcQAzIECCMQJzIECAAQHjIECAAQHlD0EVj0EWDpGWgAcAB4AIABWogBrwGSAQEymAEAoAEBqgELZ3dzLXdpei1pbWfAAQE&sclient=img&ei=S-Z0ZK6bDITWkPIPtde_uAI&bih=956&biw=1781&cs=0&rlz=1C1RXQR_enUS1008US1008#imgrc=aSLVbcgBsmbHKM"
moz-do-not-send="true">
wordToVec</a> and read some of the articles.
Word-to-Vec was around before transformers. But
transformers are based on that idea. (One of the keys
to transformers is that the embedding space, including
the features themselves, is generated as part of the
training.) The embedding of all tokens in the GPT
embedding space is the semantics. It's amazing the
extent to which that idea can be pushed and the
results LLMs produce!</div>
<div>
<div dir="ltr" class="gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr"><font><u
style="color:rgb(33,33,33);font-family:"Helvetica
Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:20px"><br>
</u></font></div>
<div dir="ltr"><span
style="color:rgb(33,33,33);font-family:"Helvetica
Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:24.75px"></span>--
Russ <br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, May 29, 2023
at 10:13 AM Jochen Fromm <<a
href="mailto:jofr@cas-group.net"
moz-do-not-send="true" class="moz-txt-link-freetext">jofr@cas-group.net</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="auto">
<div dir="auto">Yes, Rodney Brooks said something
similar. He said "GPTs have no understanding of
the words they use, no way to connect those words,
those symbols, to the real world. A robot needs to
be connected to the real world and its commands
need to be coherent with the real world.
Classically it is known as the 'symbol grounding
problem""</div>
<div dir="auto"><a
href="https://rodneybrooks.com/what-will-transformers-transform/"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://rodneybrooks.com/what-will-transformers-transform/</a></div>
<div dir="auto"><br>
</div>
<div dir="auto">One could argue that this form of
connectedness and embeddedness leads eventually to
self-awareness. First physical embeddedness, then
social embeddedness and finally self-awareness</div>
<div dir="auto"><br>
1. Physical Embeddedness:<br>
Agents who are embedded in a physical world are
aware of the world and move in it. To be embedded
they need to be embodied. Embeddedness leads to a
grounding and a unique point of view<br>
<a href="https://iep.utm.edu/husspemb/"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://iep.utm.edu/husspemb/</a><br>
<br>
2. Social Embeddedness:<br>
Agents who are embedded in a world of social
actors are aware of other agents in the world and
interact with them<br>
<a
href="https://mitpress.mit.edu/9780262518581/the-new-science-of-the-mind/"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://mitpress.mit.edu/9780262518581/the-new-science-of-the-mind/</a><br>
<br>
3. Self-Awareness:<br>
Agents who are embedded in two worlds, the
physical world and the world of language become
aware that they are actors in a world of social
actors by creating a link between the embodiment
in the physical world (the body) and the
embodiment in the world of language (the name or
I)<br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">-J.</div>
<div dir="auto"><br>
</div>
<div><br>
</div>
<div dir="auto"
style="font-size:100%;color:rgb(0,0,0)"
align="left">
<div>-------- Original message --------</div>
<div>From: Russ Abbott <<a
href="mailto:russ.abbott@gmail.com"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">russ.abbott@gmail.com</a>>
</div>
<div>Date: 5/29/23 7:08 AM (GMT+01:00) </div>
<div>To: The Friday Morning Applied Complexity
Coffee Group <<a
href="mailto:friam@redfish.com"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">friam@redfish.com</a>>
</div>
<div>Subject: [FRIAM] A distinguishing feature of
living entities </div>
<div><br>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default">
While watching my two little dogs run around
our house, it struck me that a feature that
distinguishes living from non-living entities
is the apparent effortlessness with which
living ones navigate the world. Imagine how
difficult it would be to build a robot that
could navigate the world so effortlessly. To
make the comparison a bit simpler, imagine how
difficult it would be to build a robotic
cockroach. </div>
<div
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default">
<br>
</div>
<div
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default">
When I asked ChatGPT whether anyone has built
a robotic cockroach, it came up with these
examples. (I haven't checked to see whether
these are real projects.)</div>
</div>
<blockquote style="margin:0px 0px 0px
40px;border:none;padding:0px">
<div>
<div class="gmail_default">
<ul>
<li style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px;padding-left:0.375em;white-space:pre-wrap;background-color:rgb(247,247,248)">
<p style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px"><font face="verdana, sans-serif" color="#000000">DASH: The Dynamic Autonomous Sprawled Hexapod (DASH) robot, developed at the University of California, Berkeley, was inspired by the
rapid locomotion of cockroaches. It has six legs and can move quickly on various terrains using a simple control mechanism.</font></p>
</li>
</ul>
</div>
</div>
<div>
<div class="gmail_default">
<ul>
<li style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px;padding-left:0.375em;white-space:pre-wrap;background-color:rgb(247,247,248)">
<p style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px"><font face="verdana, sans-serif" color="#000000">Harvard RoboBee: Although not specifically modeled after a cockroach, the Harvard RoboBee project aims to develop small, insect-like
robots. These tiny flying robots are inspired by the mechanics and flight capabilities of insects and demonstrate similar agility and maneuverability.</font></p>
</li>
</ul>
</div>
</div>
<div>
<div class="gmail_default">
<ul>
<li style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px;padding-left:0.375em;white-space:pre-wrap;background-color:rgb(247,247,248)">
<p style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px"><font face="verdana, sans-serif" color="#000000">iSprawl: The iSprawl robot, developed at the University of California, Berkeley, was inspired by cockroaches' ability to squeeze through
small spaces. It uses a compliant body design and six legs to navigate tight and cluttered environments.</font></p>
</li>
</ul>
</div>
</div>
<div>
<div class="gmail_default">
<ul>
<li style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px;padding-left:0.375em;white-space:pre-wrap;background-color:rgb(247,247,248)">
<p style="border:0px solid rgb(217,217,227);box-sizing:border-box;margin:0px"><font face="verdana, sans-serif" color="#000000">VelociRoACH: Developed at the University of California, Berkeley, the VelociRoACH is a fast-running robot designed to mimic the high-speed
locomotion of cockroaches. It utilizes a legged design and has demonstrated impressive speed and agility.</font></p>
</li>
</ul>
</div>
</div>
</blockquote>
<div dir="ltr">
<div class="gmail_signature" dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr"><font><u
style="color:rgb(33,33,33);font-family:"Helvetica
Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:20px"><span
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default"></span></u></font><span
style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"
class="gmail_default"></span><span
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default">These mainly explore locomotion. Besides
locomotion,
cockroaches notice
when someone
enters an area
where they are
exposed. They
quickly
scuttle off to
some hiding
place. How do
they sense the
presence of a
new being? How
do they know
where the
hiding places
are? How do
they know how
to move in the
right
direction?
How do they
know how to
avoid small obstacles
and fires?
Etc. </span></div>
<div dir="ltr"><span
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default"><br>
</span></div>
<div dir="ltr"><span
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default">One can argue that these capabilities are
hard-wired in.
But that
doesn't make
it any easier.
These are
still
capabilities
they
have, that
would be a
challenge to
build.</span></div>
<div dir="ltr"><span
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default"><br>
</span></div>
<div>
<div
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default">
I
became amazed
at how
well-connected
living
entities are
to their
environments.
They quickly
and easily
extract and
use
information
from their
environment
that is
important to
their
survival. </div>
<div
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default">
<br>
</div>
<div
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default">
Man-made
robots have
nowhere near
that level of
embeddedness
and
environmental
integration. </div>
<div
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default">
<br>
</div>
<div
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default">
Was it Rodney
Brooks who
said that we
should build
that sort of
connectedness
before
worrying about
building
intelligence
into our
robots? Today
that struck me
as an
important
insight.</div>
</div>
<div dir="ltr"><u
style="color:rgb(33,33,33);font-family:"Helvetica
Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:20px"><span
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default"></span></u></div>
<div dir="ltr"><span
style="color:rgb(33,33,33);font-family:"Helvetica
Neue",Helvetica,Arial,sans-serif;font-size:16.5px;line-height:24.75px"></span>--
Russ Abbott
<br>
<span
style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"
class="gmail_default"></span>Professor Emeritus, Computer Science<br>
California
State
University,
Los Angeles</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<span>-. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . /
-.-. --- -.. .</span><br>
<span>FRIAM Applied Complexity Group listserv</span><br>
<span>Fridays 9a-12p Friday St. Johns Cafe / Thursdays
9a-12p Zoom <a class="moz-txt-link-freetext" href="https://bit.ly/virtualfriam">https://bit.ly/virtualfriam</a></span><br>
<span>to (un)subscribe
<a class="moz-txt-link-freetext" href="http://redfish.com/mailman/listinfo/friam_redfish.com">http://redfish.com/mailman/listinfo/friam_redfish.com</a></span><br>
<span>FRIAM-COMIC <a class="moz-txt-link-freetext" href="http://friam-comic.blogspot.com/">http://friam-comic.blogspot.com/</a></span><br>
<span>archives: 5/2017 thru present
<a class="moz-txt-link-freetext" href="https://redfish.com/pipermail/friam_redfish.com/">https://redfish.com/pipermail/friam_redfish.com/</a></span><br>
<span> 1/2003 thru 6/2021 <a class="moz-txt-link-freetext" href="http://friam.383.s1.nabble.com/">http://friam.383.s1.nabble.com/</a></span><br>
</div>
</blockquote>
</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">-. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. .
FRIAM Applied Complexity Group listserv
Fridays 9a-12p Friday St. Johns Cafe / Thursdays 9a-12p Zoom <a class="moz-txt-link-freetext" href="https://bit.ly/virtualfriam">https://bit.ly/virtualfriam</a>
to (un)subscribe <a class="moz-txt-link-freetext" href="http://redfish.com/mailman/listinfo/friam_redfish.com">http://redfish.com/mailman/listinfo/friam_redfish.com</a>
FRIAM-COMIC <a class="moz-txt-link-freetext" href="http://friam-comic.blogspot.com/">http://friam-comic.blogspot.com/</a>
archives: 5/2017 thru present <a class="moz-txt-link-freetext" href="https://redfish.com/pipermail/friam_redfish.com/">https://redfish.com/pipermail/friam_redfish.com/</a>
1/2003 thru 6/2021 <a class="moz-txt-link-freetext" href="http://friam.383.s1.nabble.com/">http://friam.383.s1.nabble.com/</a>
</pre>
</blockquote>
</body>
</html>