<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"></head><body dir="auto"><div dir="auto">Oh Google has already created a "mixture of experts" architecture. Interesting.</div><div dir="auto">https://ai.googleblog.com/2021/12/more-efficient-in-context-learning-with.html</div><div dir="auto"><br></div><div dir="auto">The amount of data they use to train and implement large language models is mind-boggling. I am curious what Google and OpenAI will present this year. </div><div dir="auto"><br></div><div dir="auto">-J.</div><div dir="auto"><br></div><div><br></div><div align="left" dir="auto" style="font-size:100%;color:#000000"><div>-------- Original message --------</div><div>From: Jochen Fromm <jofr@cas-group.net> </div><div>Date: 2/5/23 1:38 PM (GMT+01:00) </div><div>To: The Friday Morning Applied Complexity Coffee Group <friam@redfish.com> </div><div>Subject: [FRIAM] Datasets as Experience </div><div><br></div></div><div dir="auto">Would a CV of a large language model contain all the datasets it has seen? As adaptive agents of our selfish genes we are all trained on slightly different datasets. A Spanish speaker is a person trained on a Spanish dataset. An Italian speaker is a trained on an Italian dataset, etc. Speakers of different languages are trained on different datasets, therefore the same sentence is easy for a native speaker but impossible to understand for those who do not know the language. </div><div dir="auto"><br></div><div dir="auto">Do all large language models need to be trained on the same datasets? Or could many large language models be combined to a society of mind as Marvin Minsky describes it in his book "The society of mind"? Now that they are able to understand language it seems to be possible that one large language model replies to the questions from another. And we would even be able to understand the conversations.</div><div dir="auto"><br></div><div dir="auto">-J.</div><div dir="auto"><br></div></body></html>