[FRIAM] Datasets as Experience

[Steve Smith]

DaveW -

I really don't know much of/if anything really about these modern AIs, 
beyond what pops up on the myriad popular science/tech feeds that are 
part of *my* training set/source.   I studied some AI in the 70s/80s and 
then "Learning Classifier Systems" and (other) Machine Learning 
techniques in the late 90s, and then worked with folks who did Neural 
Nets during the early 00s, including trying to help them find patterns 
*in* the NN structures to correlate with the function of their NNs and 
training sets, etc.

The one thing I would say about what I hear you saying here is that I 
don't think these modern learning models, by definition, have neither 
syntax *nor* semantics built into them..   they are what I colloquially 
(because I'm sure there is a very precise term of art by the same name) 
think of or call "model-less" models. At most I think the only models of 
language they have explicit in them might be the Alphabet and 
conventions about white-space and perhaps punctuation?   And very likely 
they span *many* languages, not just English or maybe even "Indo European".

I wonder what others know about these things or if there are known good 
references?

Perhaps we should just feed thesemaunderings into ChatGPT and it will 
sort us out forthwith?!

- SteveS


On 2/7/23 2:57 PM, Prof David West wrote:
> I am curious, but not enough to do some hard research to confirm or 
> deny, but ...
>
> Surface appearances suggest, to me, that the large language model AIs 
> seem to focus on syntax and statistical word usage derived from those 
> large datasets.
>
> I do not see any evidence in same of semantics (probably because I am 
> but a "bear of little brain.")
>
> In contrast, the Cyc project (Douglas Lenat, 1984 - and still out 
> there as an expensive AI) was all about semantics. The last time I 
> was, briefly, at MCC, they were just switching from teaching Cyc how 
> to read newspapers and engage in meaningful conversation about the 
> news of the day, to teaching it how to read the National Enquirer, 
> etc. and differentiate between syntactically and literally 'true' news 
> and the false semantics behind same.
>
> davew
>
>
> On Tue, Feb 7, 2023, at 11:35 AM, Jochen Fromm wrote:
>> I was just wondering if our prefrontal cortex areas in the brain 
>> contain a large language model too - but each of them trained on 
>> slightly different datasets. Similar enough to understand each other, 
>> but different enough so that everyone has a unique experience and 
>> point of view o_O
>>
>> -J.
>>
>>
>> -------- Original message --------
>> From: Marcus Daniels <marcus at snoutfarm.com>
>> Date: 2/6/23 9:39 PM (GMT+01:00)
>> To: The Friday Morning Applied Complexity Coffee Group 
>> <friam at redfish.com>
>> Subject: Re: [FRIAM] Datasets as Experience
>>
>> It depends if it is given boundaries between the datasets.   Is it 
>> learning one distribution or two?
>>
>>
>> *From:* Friam <friam-bounces at redfish.com> *On Behalf Of *Jochen Fromm
>> *Sent:* Sunday, February 5, 2023 4:38 AM
>> *To:* The Friday Morning Applied Complexity Coffee Group 
>> <friam at redfish.com>
>> *Subject:* [FRIAM] Datasets as Experience
>>
>>
>> Would a CV of a large language model contain all the datasets it has 
>> seen? As adaptive agents of our selfish genes we are all trained on 
>> slightly different datasets. A Spanish speaker is a person trained on 
>> a Spanish dataset. An Italian speaker is a trained on an Italian 
>> dataset, etc. Speakers of different languages are trained on 
>> different datasets, therefore the same sentence is easy for a native 
>> speaker but impossible to understand for those who do not know the 
>> language.
>>
>>
>> Do all large language models need to be trained on the same datasets? 
>> Or could many large language models be combined to a society of mind 
>> as Marvin Minsky describes it in his book "The society of mind"? Now 
>> that they are able to understand language it seems to be possible 
>> that one large language model replies to the questions from another. 
>> And we would even be able to understand the conversations.
>>
>>
>> -J.
>>
>>
>> -. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. .
>> FRIAM Applied Complexity Group listserv
>> Fridays 9a-12p Friday St. Johns Cafe   /   Thursdays 9a-12p Zoom 
>> https://bit.ly/virtualfriam
>> to (un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.com
>> FRIAM-COMIC http://friam-comic.blogspot.com/
>> archives:  5/2017 thru present 
>> https://redfish.com/pipermail/friam_redfish.com/
>>   1/2003 thru 6/2021 http://friam.383.s1.nabble.com/
>>
>
>
> -. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. .
> FRIAM Applied Complexity Group listserv
> Fridays 9a-12p Friday St. Johns Cafe   /   Thursdays 9a-12p Zoomhttps://bit.ly/virtualfriam
> to (un)subscribehttp://redfish.com/mailman/listinfo/friam_redfish.com
> FRIAM-COMIChttp://friam-comic.blogspot.com/
> archives:  5/2017 thru presenthttps://redfish.com/pipermail/friam_redfish.com/
>    1/2003 thru 6/2021http://friam.383.s1.nabble.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20230207/4153db5c/attachment.html>