<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>An interesting self-similar discussion to the topic of the
discussion?<br>
</p>
<p>So we have deduced the generative structure of Sabine's argument
to be<i> "just" a mish mash of semantic concepts arranged to fit
her conservative narrative</i>? We don't expect ( or need ) to
<i>find a better structure</i> to explain her behaviour? <br>
</p>
<p>Is her behaviour in some sense a part of an emergent,
qualitatively distinct paradigm? <br>
</p>
<p>And is my offering here just a mish-mash of dense semantic
concepts arranged to be disruptive or self-aggrandizing?<br>
</p>
<div class="moz-cite-prefix">glen wrote:<br>
</div>
<blockquote type="cite"
cite="mid:681b6549-0ac0-4daf-baaa-e1b61b05f12e@gmail.com">Yeah,
it's kinda sad. Sabine suggests someone's trying to *deduce* the
generators from the phenomena? Is that a straw man? And is she
making some kind of postmodernist argument that hinges on the
decoupling of scales? E.g. since the generator can't be deduced
[cough] from the phenomena, nothing means anything anymore?
<br>
<br>
What they're actually doing is induction, not deduction. And the
end products of the induction, the generative constraints, depend
fundamentally on the structure of the machine into which the data
is fed. That structure is generative, part of the forward map ...
deductive. But it's parameterized by the data. Even if we've
plateaued in parameterizing *this* structure, all it implies is
that we'll find a better structure. As Marcus and Jochen point
out, it's really the same thing we've been doing for decades, if
not centuries, in many disciplines.<br>
</blockquote>
<blockquote type="cite"
cite="mid:681b6549-0ac0-4daf-baaa-e1b61b05f12e@gmail.com">
<br>
So her rhetoric here is much like her rhetoric claiming that
"Science if Failing". It's just a mish-mash of dense semantic
concepts arranged to fit her conservative narrative.
<br>
</blockquote>
<br>
<blockquote type="cite"
cite="mid:681b6549-0ac0-4daf-baaa-e1b61b05f12e@gmail.com">
<br>
On 11/17/24 08:45, Roger Critchlow wrote:
<br>
<blockquote type="cite">Sabine is wondering about reported
failures of the new generations of LLM's to scale the way the
their developers expected.
<br>
<br>
<a class="moz-txt-link-freetext" href="https://backreaction.blogspot.com/2024/11/ai-scaling-hits-wall-rumours-say-how.html">https://backreaction.blogspot.com/2024/11/ai-scaling-hits-wall-rumours-say-how.html</a>
<a class="moz-txt-link-rfc2396E" href="https://backreaction.blogspot.com/2024/11/ai-scaling-hits-wall-rumours-say-how.html"><https://backreaction.blogspot.com/2024/11/ai-scaling-hits-wall-rumours-say-how.html></a>
<br>
<br>
On one slide she essentially draws the typical picture of an
emergent level of organization arising from an underlying
reality and asserts, as every physicist knows, that you cannot
deduce the underlying reality from the emergent level. Ergo, if
you try to deduce physical reality from language, pictures, and
videos you will inevitably hit a wall, because it can not be
done.
<br>
<br>
So she's actually grinding two axes at once: one is AI
enthusiasts who expect LLM's to discover physics, and the other
is AI enthusiasts who foresee no end to the improvement of
LLM's as they throw more data and compute effort at them.
<br>
<br>
But, of course, the usual failure of deduction runs in the
opposite direction, you can't predict the emergent level from
the rules of the underlying level. Do LLM's believe in particle
collliders? Or do they think we hallucinated them?
<br>
<br>
</blockquote>
<br>
<br>
</blockquote>
</body>
</html>