<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Mutual information turned up earlier this week in some articles about what Google AI does next, which could be read as an attempt on organization, too.<div><br></div><div><a href="https://www.zdnet.com/article/google-ponders-the-shortcomings-of-machine-learning/">https://www.zdnet.com/article/google-ponders-the-shortcomings-of-machine-learning/</a> this is the article that Google news pushed on me. The shortcoming of machine learning, in the article, is that it isn't general intelligence. And it linked to the following papers.<br></div><div><br></div><div>First <a href="https://arxiv.org/abs/1806.01261">https://arxiv.org/abs/1806.01261</a> <span style="color:rgb(0,0,0);font-family:"Lucida Grande",helvetica,arial,verdana,sans-serif">Relational inductive biases, deep learning, and graph networks</span></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span style="color:rgb(0,0,0);font-family:"Lucida Grande",helvetica,arial,verdana,sans-serif">Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, in part, to cheap data and cheap compute resources, which have fit the natural strengths of deep learning. However, many defining characteristics of human intelligence, which developed under much different pressures, remain out of reach for current approaches. In particular, generalizing beyond one's experiences--a hallmark of human intelligence from infancy--remains a formidable challenge for modern AI. </span></blockquote><div>The proposed solution is to develop deep learning architectures over graph structured data.</div><div> </div><div>Then <a href="https://arxiv.org/abs/1809.10341">https://arxiv.org/abs/1809.10341</a> <span style="color:rgb(0,0,0);font-family:"Lucida Grande",helvetica,arial,verdana,sans-serif">Deep Graph Infomax</span></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span style="color:rgb(0,0,0);font-family:"Lucida Grande",helvetica,arial,verdana,sans-serif">We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised manner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs---both derived using established graph convolutional network architectures.</span></blockquote><div>So mutual information between the whole and the part, the shared purpose as it were, where the residual information of the part -- modulo the mutual information with the whole -- would presumably be the 'function' of the part.</div><div><br></div><div>What struck me most forcefully about this is that deep learning models themselves provide a huge dataset of graph structured data, with varying degrees of generality and success over varying datasets of experience.</div><div><br></div><div>The oil and water never "really mix". The shaking mechanically breaks them into smaller and smaller pieces which intermingle, but they are immediately reforming into their original layers. If you add detergent, not recommended for salad dressing, you can get the emulsified oil to dissolve into the water and they won't separate again.</div><div><br></div><div>-- rec --</div><div> </div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Oct 24, 2018 at 2:56 AM Marcus Daniels <<a href="mailto:marcus@snoutfarm.com">marcus@snoutfarm.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="m_-8619395452988123414WordSection1">
<p class="MsoNormal">Nick,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">It sounds like you are describing mutual information. This is ancient, but a nice overview of related topics:<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><a href="https://www.amazon.com/Information-Theory-Qualitative-Quantitative-Applications/dp/0803921322" target="_blank">https://www.amazon.com/Information-Theory-Qualitative-Quantitative-Applications/dp/0803921322</a><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Marcus<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Friam <<a href="mailto:friam-bounces@redfish.com" target="_blank">friam-bounces@redfish.com</a>> on behalf of Nick Thompson <<a href="mailto:nickthompson@earthlink.net" target="_blank">nickthompson@earthlink.net</a>><br>
<b>Reply-To: </b>The Friday Morning Applied Complexity Coffee Group <<a href="mailto:Friam@redfish.com" target="_blank">Friam@redfish.com</a>><br>
<b>Date: </b>Wednesday, October 24, 2018 at 12:22 AM<br>
<b>To: </b>Roger Critchlow <<a href="mailto:rec@elf.org" target="_blank">rec@elf.org</a>>, Friam <<a href="mailto:Friam@redfish.com" target="_blank">Friam@redfish.com</a>><br>
<b>Subject: </b>[FRIAM] On old question<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">Dear Roger, and anybody else who wants to play, <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">While waiting for my paper, <i>Signs and Designs</i>, to be rejected, I have gone back to thinking about an old project, whose working title has been “<i>A Sign Language</i>.” And this has led me back to Robert Rosen, whose
<i>Life Itself</i> I bought almost 9 years ago and it has remained almost pristine, ever since. In the chapter I am now looking at, Rosen is talking about “organization.” Now, I have been thinking about organization ever since I read C. Ray Carpenter’s early
work on primate groups back in the late 50’s. It seemed to me at the time, and it seems to me reasonable now, to define the organization of a set of entities as related in some way to the degree to which one can predict the behavior of one entity from knowledge
about another. Now the relationship is not straightforward, because neither total unpredictability (every monkey behaves exactly the same as every other monkey in every situation) nor total unpredictability (no monkey behaves like any other monkey in ANY
situation) smacks of great organization. The highest levels organization, speaking inexpertly and intuitively, seem to correspond to intermediate levels of predictability, where there were several classes of individuals within a group and within class predictability
was strong but cross-class predictability was weak. On my account, the highest levels of organization involve hierarchies of predictability. Thus honey bees and ants are more organized than starling flocks, say.
<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">This is where the matter stood at the point that I came to Santa Fe and started interacting with you guys 14 years ago. You-all introduced me to a totally different notion of organization based – shudder – on the second law. But I have
never been able to deploy your concept with any assurance. So, for instance, when I shake the salad dressing, I feel like I am disorganizing it, and when it reasserts itself into layers, I feel like it ought to be called more organized. But I have a feeling
that you are going to tell me that the reverse is true. That, given the molecules of fat and water/acid, that the layered state is the less organized state.
<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Now this confusion of mine takes on importance when I try to read Rosen. He defines a function as the difference that occurs when one removes a component of a system. I can see no reason why the oil or the water in my salad dressing cannot
be thought of components of a system and if, for instance, I were to siphon out the water from the bottom of my layered salad dressing, I could claim that the function of the water had been to hold the water up. This seems a rather lame notion of function.
<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Some of you who have been on this list forever will remember that I raised the same kind of worry almost a decade back when I noticed the drainage of water from a basin was actually
<i>slowed </i>by the formation of a vortex. This seemed to dispel any notion that vortices are structures whose function is to efficiently dispel a gradient. It also violated my intuition from traffic flows, where I imagine that rigid rules of priority would
facilitate the flow of people crossing bridges to escape Zozobra. <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">It’s quite possible that my confusions in this matter are of no great general applicability, in which case, I look forward to being ignored. <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Nick <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Nicholas S. Thompson<u></u><u></u></p>
<p class="MsoNormal">Emeritus Professor of Psychology and Biology<u></u><u></u></p>
<p class="MsoNormal">Clark University<u></u><u></u></p>
<p class="MsoNormal"><a href="http://home.earthlink.net/~nickthompson/naturaldesigns/" target="_blank">http://home.earthlink.net/~nickthompson/naturaldesigns/</a><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</blockquote></div>