[FRIAM] fluid codes revisited

[David Eric Smith]

Jon, hi and thank you,

This note was tremendously helpful in getting me to see where all this was coming from; apologies to be weeks later than I wanted in having a moment to answer.

I should have realized that all this was coming up in a context of Nick’s concern with Fisher’s theorem, since I know that the intensive/extensive adjectives were put on the list in a tread that was also Fisher-themed.  But the two are in my mind so divergent that I didn’t put together that there would be a conversation trying to identify them.  But now I have (I think) a better sense of where we are.

It doesn’t seem impossible to me that “additivity” as it arises in large-deviation theory and thermodynamics has _some_ connection to “additivity” as the geneticists use it for Fisher’s theorem.  But if someone had a gun to my head and forced me to claim what that relation might be, I would probably be dead before I could come up with anything I was comfortable claiming.  Just to alert you that I am starting from a frame of mind in which an overloading of the same term with both uses seems very un-natural.

Let me briefly comment on how the genetics/statistician’s Fisher-additivity seems to me (not claiming how “it is”, but again declaring my own frame).  From much of the rest of our language of biology, one would think that we believe development, molecular biology, physiology, etc. are extremely algorithmic.  We use terms for their function much like those we would use for a computer program.  

So, then:

Suppose someone gave you two large files of N bits.  One of them is a global climate model, and the other is a pornographic movie.  Your job is to assign a probability to the type of each file.  (Here I am treating the two as if they are different types, I know that for some weather-people that might not be so,  but it’s a parable.)  

Now, if I limited you to finding a linear function, which is the binary inner product of some vector with the 1/0 bit values of all the bits in the two files (is this what Vapnik’s “support vectors” are?), you would feel very impaired in solving this problem.  There are 2^N possible programs, while there are N+1 possible inner products of the bit values treated independently.  

A snide person would say “how could geneticists ever think of reducing the algorithmicity of biology to some additive fitness measure?”  But if you are not a snide person, you could say “Suppose we have no theory of what a computer is, or even what computation is, much less any way to encode our knowledge of the particular computer that will read and execute these programs, its operating system, its languages, etc.  There is no way we will every have the statistical power from new instances to resolve 2^N cases.  We will be lucky to have the power to train a linear function to resolve N+1 cases.  So without a system to represent the information in algorithmicity, the only statistics we can do that is not prior-biased would be something like additivity.”  That, I think, is where the statistical geneticists found themselves in the 1930s and 1940s, trying to work out the most-basic integration of Darwin and Mendel, at the same time trying to empirically validate any of what they were doing.

That doesn’t change, of course, our expectation that an additive classifier should do a horrible job with anything that really is algorithmically structured.

It also makes us aware that the fascinating question is why _anything_ in genetics gives _any_ performance at all.  I think the answer is an evo-devo answer, that selection is a weak filter, so life has succeeded ty trying to pigeonhole its algorithmicity into separable chunks as much as possible to make any use of selection.  That is Leslie Valiant’s theme in the book Probably Approximately Correct.

If, however, we want to discuss the whole rest of the universe of features where the algorithmicity doesn’t boil down into simple bit-wise additive functions, we can do exactly the thing you say, in the obvious way.  The reduction of 2^N distinct binary genotypes into additive sets is just the conversion of a direct-product representation to a direct-sum representation.  First we take all N+1 independent linear functions.  Then the set of binary cumulants, subtracting out the information already in the linear functions.  Then the ternary cumulants subtracting out the linear and binary, etc., until we end up with the N-th irreducible cumulant.  In genetics, of course, we will barely-ever have statistical power to resolve almost-all of those terms.  But at least we have framed the problem in such a way that it is clear what needs identifying.  We have also now defined a search process for the rare higher-order cumulants that may be identifiable while we pass over lower-order ones.  A much better decomposition, of course, is not to use structureless sets like cumulants, but rather to find representations of algorithmic architectures that can be put into a frame of statistical identification.

The simple version of this (cumulant expansion or its equivalent) is what geneticists term “interactions” (as you already know), and the fact that additivity includes higher-order interaction terms and is not limited to linearity, exactly as you say below, is emphasized by Steve Frank in:
  author = "Frank, Steven A.",
  title = "The Price equation, Fisher's fundamental theorem, kin selection, and causal analysis",
  journal = "Evolution", 
  volume = "51",
  pages = "1712--1729",
  year = "1997"


So, that was genetics.  To contrast that whole framework with “additivity” as it applies in large-deviation theory (and being made conceptually coherent within that frame, to thermodynamics), and with the context-dependent notions of extensivity and its dual intensiivity:  All that categorization only exists is aggregation causes the projections of joint probability distributions to attract onto some low-dimensional manifold in which scale separates from structure.  Such a condition presumably requires that the real world (pace DaveW) really does modularize in some way that creates this renormalization-like flow onto low-dimensional manifolds.  I _assume_ (a better person than me would know the answer and would know how to derive it) that such a condition is extremely restrictive, in something like the way the non-random numbers are rare relative to the random ones (Chaitin) or the rationals are measure zero relative to the reals.  But still there can be infinitely many ways to realize that condition.


Now, back to whether the two have interesting overlaps.  I find it very interesting to ask whether, in systems that are algorithmically organized, there might be kinds of aggregation that lead to dimensional reductions of the kind that large-deviation theory describes for the simple salient modularizations.  That looks like a hard and interesting question to formulate.  Dave Ackley’s work on scaling of avalanches in library rewriiting for Linux looks like an empirical investigation to look for patterns that might be developed.  I don’t know what-all might be in the algorithmic literature, particularly following from the work of people like Risannen.  There must be an enormous literature in IEEE, which I almost-never see.


Hope this is helpful, 

Eric




> On May 11, 2022, at 4:26 AM, Jon Zingale <jonzingale at gmail.com> wrote:
> 
> EricS,
> 
> Thanks for getting back to me with a reference[1]. I printed a couple of pages of it, which I then scrape out time to read.
> 
> In a rare moment of straightforward computation, NickT walked me through how one performs an analysis of variation. Now I am reading a little bit of "Beyond Versus" about the history of the nature-nurture debate. There, the author discusses Fisher's notion of *interaction* as a deviation from additivity. This got me thinking again about my question regarding additivity and your comment about finding correct *aggregation operators*. To what degree does this *wider class* of operators correspond to instances of coproducts in context-appropriate categories? My wondering begins with the observation that *interactions* can often be represented by (hyper?)-arrows (or at least *(hyper?)-edges) and that such structured things can often have their own notions of *additivity* (following from the notion of coproduct and especially in linear categories, aka, additive categories). Does this jive with your understanding?
> 
> [1] https://redfish.com/pipermail/friam_redfish.com/2022-April/092364.html <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fredfish.com%2fpipermail%2ffriam_redfish.com%2f2022-April%2f092364.html&c=E,1,3m7P1uspuRvXk-LFegpJygquNikHuEjhWL4rxBX0KSosfqxvXxLvKyQaUyAB5PpzWk4hMDKF3_GMgl3lJ-OQ_s8JbX_J0uRKqX6doJWJW74GZ_U,&typo=1>-. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. .
> FRIAM Applied Complexity Group listserv
> Fridays 9a-12p Friday St. Johns Cafe   /   Thursdays 9a-12p Zoom  https://linkprotect.cudasvc.com/url?a=https%3a%2f%2f%2f%2fbit.ly%2fvirtualfriam&c=E,1,Y3RbhQ0jMlZp2PXs8MK8UEtdWBHOy2--QC1JvBs5niB5yETPO04mxSVqjtx-GE3fgrLNHAbaA1T8fPmtZ7K-veZoGUWpIIPNxzE9uRL_uGa6fSJIBw_bn1xXR6XZ&typo=1
> un/subscribe https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fredfish.com%2fmailman%2flistinfo%2ffriam_redfish.com&c=E,1,nw6ordvO4WU0WNAd5J44vs00vTW49c-4sinykYywiUpzc2BYStLyqDsY19znuT7WeQyi1_iDrmqhPHdAgP1butgdI61BUEHHHB8TC8rsh9niOj-cTsyjoQ,,&typo=1
> FRIAM-COMIC https://linkprotect.cudasvc.com/url?a=http%3a%2f%2ffriam-comic.blogspot.com%2f&c=E,1,69LMnHPPEaXHKVFMb9FTDc75LoVQudnQP5uEjfjaFQ-U6FzdHysF0YSQC5PKbXSNPMqsfEQ1FKEti8iGaRw9BmY7Hzml6czvl9dq-t7u3ik4lH7lGqDzz9jiaXE,&typo=1
> archives:  5/2017 thru present https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fredfish.com%2fpipermail%2ffriam_redfish.com%2f&c=E,1,z9ZgM0NCMqMmq-Vjj0L06Y-b-M3YISPvFgc02ub0S34w3duB7UPkxSaq9qlIiCeSIIPbeJEjmz7ckBRx4yoyK_pa36NCY25Zsv91CmtPUgN0F98mofPDw9cy5M23&typo=1
>  1/2003 thru 6/2021  http://friam.383.s1.nabble.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20220524/d589ad6e/attachment.html>