[FRIAM] neural operators seem promising

[Santafe]

I _very often_ have the thought that, were the nature of people such that grievance and misanthropy simply didn’t do them any good, and so they simply never engaged in it, so many conversations would go on in such different ways, that we might have to adjust a bit to realize they started from the same query.

One such query is whether the nature of anti-theory people is mainly an aesthetic style of thought (seems very possible), or mainly motivated by a dislike of people they met earlier who (whether with warrant or just to serve other needs of their own) they label as “theory people”.  I would like it if it were mostly the former; that anti-theory people were “born this way”; it would give me a conversation that seems interesting in several dimensions and that I could navigate.  Let’s suppose that such conversations are available somewhere, even if not everywhere.

The start of this went something along the lines of “Faraday locked in electromagnetism by its empirical evidences, and Maxwell put some pretty symbols onto it.”  (The original wasn’t exactly as I just wrote it, and I am over-drawing here to take the direction to its cartoon-simplified limit.  I am also _sure_ I can find some truly anti-theory people who believe this is the absolutely right take on it.  Within Chemistry, where I have the counterpart to this conversation fairly often, I have a good list of names, because it is still the prevalent aesthetic of the field.)

The sort of mind that believes that the former take on Maxwellian electromagnetism is indeed the only real-man’s hard-headed take, is likely (to the extent that it has any patience with formal logical analysis at all as not a priestly self-indulgent waste of time) inclined to think that Popper has a good description of the criteria for scientific meaningfulness and truthfulness.

But then we can do it recursively all the way down.  Is Newtonian gravity just one among an infinitude of data-compressions of Keplerian orbits (since, at the end, everything moving under gravity and approximating away other effects such as friction is on a Keplerian orbit, including apples, so there “isn’t” really anything else).

Let’s not answer, but simply add attested observations:

It was studying Maxwell’s field equations in school that led Einstein to try to construct general relativity within similar concepts.  And presumably the very geometric flux-sphere picture that comes with Newtonian gravity that causes geometry to be retained as the phenomenon for Einstein’s gravitational field theory to be about.

One can go through such idea-chains across the sciences.  In some, people don’t leave pithy accounts of why they believed it occurred to them to do things one way rather than another; in other cases they do leave such trails, at least about their beliefs.  Or philosophers come along later and do forensics and argue that their work shows their reasons to be such-and-such.

A compact representation of the latter collection of asserted-observations is that there is some kind of work that theory is doing as itself, not as a proxy for something else (like description-length shortening for a pile of data-instances).  I remember how it seemed an insightful turn for me when my graduate advisor commented that the particle physicists had felt a sense of liberation when they could throw away the Particle Data Book, with the advent of first Murray’s symmetry classification and eventually the settling in of QCD as a theory in which one could stably compute things, and then the whole symmetry-grouping of all the elementary particles by a few terms. 


Circling back to thermodynamics, Harold’s “Emergence of Everything”, and what is or isn’t substantial in the world of observations and states of mind that we take on in relation to them:

Harold was happy invoking Popper, and didn’t want to sweat a lot over how much Popper was trying to take over a dichotomy from first-order logic,  and the asymmetry between there-exists and for-all, and how much it doesn’t work to press that into service as a formalization for empiricist reasoning.  Harold was, generally, an easy-going guy, and willing for things to be rough, or half-wrong, supposing that if he could intuitively get them half-right, that would be much better than nothing, and there would be time to come back and fix whatever parts may have been wrong.  So he could like Popper as one of his half-right positions, even though it was the inability to deal with being half-right where Popper ultimately undermined himself.   btw., that’s where a very useful study of metaphor in science, along the lines that DaveW gave a definition of it from Quine, can get built up.

Probably likewise with thermo and steam engines.  For the purpose of making a certain point — that theory doesn’t arise in a vacuum or from direct access to the Mind of God — Harold would be happy to overstate the simplicity of this position, and to evangelize for empiricism.

But of course, in the world we live in — and especially the world where I live, which is almost-all thermodynamics almost-all the time, and almost-none of it about steam engines, or even anything having to do with mechanics or energy — we have learned much, much more about nearly-everything, from thermodynamics, than there even was of thermodynamics, to have learned from steam engines.  At the end of the day, the lessons of thermodynamics, when properly understood, constitute the explanation for why there even are stable macro-worlds.  Of more-or-less anything.  In other working conversations, with other aims, Harold would of course have seen that too, and been happy with the statement putting it on record.  Even though that statement would have seemed, to a debaterly-type mind, to have contradicted the earlier one.


I have seen a lot of chat over the years about what is “the nature” of theory as something that can do work that deserves to be called different-in-kind, and not just different-in-cost, than listing data instances, thus making theory particular among data compressions (the latter, as a kind of generic category; obviously theories are, as one of their aspects, compressions of data instances; the question here is whether to say that is “all” they are is as good or as useful an account as we can give).  But at the end, I just hear the same positions reiterated, some of them more rhetorically elegantly (Cris Moore did a very nice job in a tiny soliloquy in one of the SFI public lectures), or more tritely and conventionally.  But I haven’t heard somebody with something really original to say on the question, that makes me stop and think I see things better, for a long time now.  I think the Philosophers of Science (I’ll capitalize both for DaveW) put a lot of time into this.  If I had more time I would probably try to listen to them, and I might find they have interesting things to say. 

Eric





> On Jul 17, 2025, at 2:19, Steve Smith <sasmyth at swcp.com> wrote:
> 
> Anima's presentation reminded me quite nicely of the Numenta/Redwood work of Jeff Hawkins et al?   Cortical columns, etc. 
> Did Harold Morowitz make a strong assertion to the tune: "we learned more about thermodynamics from steam-engines than vice-versa"?    EricS or StephenG might have first-hand knowledge?
> Is this theory/practice dichotomy just another form of meta-scaffolding in evolution (of any system) with the cut-and-try providing the mutation/selection and the theory/formalism binding the "lessons learned" into well... "lessons learned"?
> On 7/16/2025 2:12 AM, Pieter Steenekamp wrote:
>> Both the video of Anima Anandkumar’s Stanford seminar and her scientific paper on Neural Operators really got me excited—the ideas feel fresh and powerful.
>> 
>> The paper is quite technical and digs into the math behind Neural Operators, without talking much about robotics. In her talk, though, she clearly links the work to robots, and it sounds as if robotics is a big focus for her team.
>> 
>> What jumped out at me is how different her style is from Elon Musk’s approach with Tesla’s Optimus robot. Anandkumar begins with deep theory, building firm mathematical foundations first. Musk takes a “just build it” path—make it, test it, break it, fix it, and keep going.
>> 
>> This contrast reminds me of engineering school and the Faraday‑Maxwell story. Faraday was the hands‑on experimenter who uncovered the basics of electricity and magnetism through careful tests. Maxwell came later and wrote the elegant equations that explained what Faraday had already shown.
>> 
>> So I wonder: will the roles flip this time? Will deep theory from researchers like Anandkumar guide the breakthroughs first, with practice following? Or will practical builders like Musk sprint ahead and let theory catch up afterward?
>> 
>> Either way, watching these two paths unfold side by side is thrilling. It feels like we’re standing on the edge of something big.
>> 
>> On Wed, 16 Jul 2025 at 04:11, Jon Zingale <jonzingale at gmail.com <mailto:jonzingale at gmail.com>> wrote:
>>> Even if just for the freedom of scale, learning infinite dimensional function spaces, etc...
>>> 
>>> https://www.youtube.com/watch?v=caZyFlSSKtI
>>> https://arxiv.org/pdf/2506.10973
>>> 
>>> 
>>> .- .-.. .-.. / ..-. --- --- - . .-. ... / .- .-. . / .-- .-. --- -. --. / ... --- -- . / .- .-. . / ..- ... . ..-. ..- .-..
>>> FRIAM Applied Complexity Group listserv
>>> Fridays 9a-12p Friday St. Johns Cafe   /   Thursdays 9a-12p Zoom https://bit.ly/virtualfriam <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fbit.ly%2fvirtualfriam&c=E,1,fqLCfmxD9O9EQWAvVNvceED6l9-CH0x1NbEyF8I6tcaSy_FU4J1BIpk1r8NvZJBSw_X_tH9vSJ6xESIP5HytHQKnnFYA0bh9moSxajzwT4EYdEq9cHxPGzXfcA,,&typo=1>
>>> to (un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.com <https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fredfish.com%2fmailman%2flistinfo%2ffriam_redfish.com&c=E,1,X0Tt91XAOHQTWZG3DGupZBtxQnkXycficvAt9fd5eVt_EjCCANoIQdzMHKjD34hw80pBjuG5m0Ng0pKp6FqbhoVaFQnFtdY4jekmh9Ax&typo=1>
>>> FRIAM-COMIC http://friam-comic.blogspot.com/ <https://linkprotect.cudasvc.com/url?a=http%3a%2f%2ffriam-comic.blogspot.com%2f&c=E,1,DElem1umk50s7v5_n8aVPAcuNV5thmMiUNtEcpFHqLNGxLef3zoSkKJPGL3NZj1fv06LWCojrSEIPlxZ3DM5iqIAP_WjqVD3XisuTSpogyJAvVNW&typo=1>
>>> archives:  5/2017 thru present https://redfish.com/pipermail/friam_redfish.com/ <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fredfish.com%2fpipermail%2ffriam_redfish.com%2f&c=E,1,9A_0i3LLqKl8Z6qXqieHffdNyyR503sa6Oo3V8p-iO1DqKqmX1nf8FqlAot1NLFgXo_-XLkMFSEEDbNfuFH-1h6JhTt5u7hnpxtsKUVRZZ_gmT36_3gyxbGeVw,,&typo=1>
>>>   1/2003 thru 6/2021  http://friam.383.s1.nabble.com/
>> 
>> 
>> .- .-.. .-.. / ..-. --- --- - . .-. ... / .- .-. . / .-- .-. --- -. --. / ... --- -- . / .- .-. . / ..- ... . ..-. ..- .-..
>> FRIAM Applied Complexity Group listserv
>> Fridays 9a-12p Friday St. Johns Cafe   /   Thursdays 9a-12p Zoom https://bit.ly/virtualfriam <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fbit.ly%2fvirtualfriam&c=E,1,bH-hGkqjZ_l5N8cHZOb6yVyDB4zuxxdqcEut-frWgkUiX57a014jPVVX89pFQA2pTCCx3ETeGsgy-Ebng6oiq5XGeWyqEOyU2iOsxdpEx-02r0v6yN-MzQ,,&typo=1>
>> to (un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.com <https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fredfish.com%2fmailman%2flistinfo%2ffriam_redfish.com&c=E,1,CU6C3Y1qd8G69WfNwLFAjYL5YxIpypMnh7WGKCy7sI48bkmbUT5OGSrkkK4qX7Vv4Po78cjfn6PNC8husKj2aNPjP2D6iNPCTJ3WPNnZeponPuUiAOHt&typo=1>
>> FRIAM-COMIC http://friam-comic.blogspot.com/ <https://linkprotect.cudasvc.com/url?a=http%3a%2f%2ffriam-comic.blogspot.com%2f&c=E,1,RIb8FsDgoQ0Vcb-n9EkS2GQFbBeJu8uV2sGMNlPscE_EdDeMj0ahLXajjAvxvMHu6tlbXkQCgWl_MTEcnopJaHk5D9GFnpfrh-GWo_EiGeciXUDkEQ1qbO2Mr88,&typo=1>
>> archives:  5/2017 thru present https://redfish.com/pipermail/friam_redfish.com/ <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fredfish.com%2fpipermail%2ffriam_redfish.com%2f&c=E,1,Nm3_HtIy4HCOuapzpRMF87dtGln0JtxitxkNZTPOqUuGBgGYuL0j80NQuuNnBFx6wvj0oVrBopF85ahUKilzV5PbJGWTirK9CRxxici-LBl04MpaJ-cSRTtxYCGj&typo=1>
>>   1/2003 thru 6/2021  http://friam.383.s1.nabble.com/
> .- .-.. .-.. / ..-. --- --- - . .-. ... / .- .-. . / .-- .-. --- -. --. / ... --- -- . / .- .-. . / ..- ... . ..-. ..- .-..
> FRIAM Applied Complexity Group listserv
> Fridays 9a-12p Friday St. Johns Cafe   /   Thursdays 9a-12p Zoom https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fbit.ly%2fvirtualfriam&c=E,1,-RpDiYiWHxJeffwvdH-CWmpGyK0qha9HoBze7PGtHY6n8MaAys5GjR8LtHvRrVY3_ob0PBC7Txcp6-mRLqxifemdrqWxqW6AWjop9jJg9Es,&typo=1
> to (un)subscribe https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fredfish.com%2fmailman%2flistinfo%2ffriam_redfish.com&c=E,1,CpBNaP1wLHqmBbqmOnbcR_A--YQDoU7HDAojbfqgYdVlMS7XIcuaMOoFfdwqSiDvk41jFiODa1XRUogcyTdjRhEs0OXbJiB8XC-ETzRS&typo=1
> FRIAM-COMIC https://linkprotect.cudasvc.com/url?a=http%3a%2f%2ffriam-comic.blogspot.com%2f&c=E,1,vA2k7dsDzJHRJysWhARmmeSKjKvA3-R9sIlP1XhRgHzWK4Ljmw7rUZ-61GO8xtcXcXekXZMyh7huJAy6BiXByCHabhN7XcV5dpepv3itVq4KL6Xx7cU,&typo=1
> archives:  5/2017 thru present https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fredfish.com%2fpipermail%2ffriam_redfish.com%2f&c=E,1,NO9uwFaMy3n5q3E8Ag4y1Vi-4B1i9hl-FEdQhPpwKjBigxn02sR9IWH4F8NunauqdZjWTPdQBXCQprmuN2odJIEFtpnLmApntWcNXTzBUBRXcvVonrOkS9S9eQ,,&typo=1
>  1/2003 thru 6/2021  http://friam.383.s1.nabble.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20250717/a5f1773d/attachment.html>