[FRIAM] Magic Harry Potter mirrors or more?

[glen]

Right. I mispoke, "training" may not be the right word. But my understanding is they have humans monitoring at least some of the ChatGPT usage and using it for RL. I have no idea what the frequency of the feedback is, though. I speculate that it was much faster early on, when they effectively took it offline for most people.

So, whether one's little online demonstration of the chat toolchain telling you what you want to hear impacts the RL is an open question. Shirley, the 2+2=5 lessons are ignored by the RL workflow.

On 3/1/23 09:08, Marcus Daniels wrote:
> It is pre-trained.   Just because there is a chat doesn’t mean it considers the correspondent as providing new evidence.
> 
> *From:* Friam <friam-bounces at redfish.com> *On Behalf Of *Gillian Densmore
> *Sent:* Wednesday, March 1, 2023 9:05 AM
> *To:* The Friday Morning Applied Complexity Coffee Group <friam at redfish.com>
> *Subject:* Re: [FRIAM] Magic Harry Potter mirrors or more?
> 
> Glen Funny you say that about chat gpt:
> 
> https://twitter.com/tasty_gigabyte7/status/1620571251344551938 <https://twitter.com/tasty_gigabyte7/status/1620571251344551938>
> 
> On Wed, Mar 1, 2023 at 10:02 AM Marcus Daniels <marcus at snoutfarm.com <mailto:marcus at snoutfarm.com>> wrote:
> 
>     On one hand, there needs to be ongoing debate (in training) to reflect actual uncertainty in responses.   One the other hand, humans spew a lot of nonsense, and a lot of it is just wrong.  That leads to the vulnerability to black hatters.  If there is bias in the (peer) review of the input data, there will be bias in the output distributions.
> 
>     -----Original Message-----
>     From: Friam <friam-bounces at redfish.com <mailto:friam-bounces at redfish.com>> On Behalf Of glen
>     Sent: Wednesday, March 1, 2023 8:51 AM
>     To: friam at redfish.com <mailto:friam at redfish.com>
>     Subject: Re: [FRIAM] Magic Harry Potter mirrors or more?
> 
>     Exactly. We recently started a rough eval of the newer "text-embedding-ada-002" model versus the older "text-similarity-curie-001" model. The newer model produces a lower dimensional embedding (1536) than the older (4096), which could imply the older model might provide a more fine-grained [dis]similarity. I don't think that's the case, though, because the encoding for the new model allows for 8192 tokens and the old one only 2046 tokens. So, the ability of the high dimensional embedding is limited by the granularity of the encoding. We're not done with the evaluation yet, though.
> 
>     One of the ideas I had when chatgpt took off, more along the lines of EricS' question, is to focus on red-teaming GPT. OpenAI's already doing this with their human-in-the-loop RL workflow. And the good faith skeptics in the world are publishing the edge cases they find (e.g. teaching GPT to say 2+2=5). But if a black hatter gets a backdoor into a *medically* focused app, she could really screw up particular domains (e.g. caregiver demographics, patient demographics, etc.). Or, if she were anti-corporate, she could screw up the interface between insurance companies and medical care.
> 
>     On 3/1/23 08:33, Marcus Daniels wrote:
>      > It seems to me the "mansplaining" is built into an algorithm that chooses the most likely response.  Choose all responses above probability 0.9 and present them all to give the user a sense of the uncertainty.
>      >
>      > -----Original Message-----
>      > From: Friam <friam-bounces at redfish.com <mailto:friam-bounces at redfish.com>> On Behalf Of glen
>      > Sent: Wednesday, March 1, 2023 8:31 AM
>      > To: friam at redfish.com <mailto:friam at redfish.com>
>      > Subject: Re: [FRIAM] Magic Harry Potter mirrors or more?
>      >
>      > Yep, that's the fundamental problem with the "chat" usage pattern. But it's much less of a problem with other usage patterns. For example, we have a project at UCSF where we're using GPT3.5 to help us with the embeddings for full text biomedical articles. This produces opportunities for several other usage patterns that preserve the inherent uncertainty, allowing the user to both gain some new insight without the "mansplaining" confidence of the chat mode. We're way upstream of the clinic so far, though. FDA approval for such a "device" might be sticky.
>      >
>      > On 3/1/23 08:19, Barry MacKichan wrote:
>      >> When I bought back my company about 25 years ago, the mantra for programmers was “Google the error message!” Now ChatGPT will write some of the code for you. The job of programming still requires a lot of knowledge and experience since using ChatGPT-generated code without quality checking is far from failsafe.
>      >>
>      >> —Barry
>      >>
>      >> On 1 Mar 2023, at 11:04, Marcus Daniels wrote:
>      >>
>      >>      I have seen doctors run internet searches in front of me. If a LLM is given all the medical journals, biology textbooks, and hospital records for training, that could be a useful resource for society.
>      >>
>      >>      -----Original Message-----
>      >>      From: Friam <friam-bounces at redfish.com <mailto:friam-bounces at redfish.com>> On Behalf Of Santafe
>      >>      Sent: Wednesday, March 1, 2023 4:45 AM
>      >>      To: The Friday Morning Applied Complexity Coffee Group <friam at redfish.com <mailto:friam at redfish.com>>
>      >>      Subject: Re: [FRIAM] Magic Harry Potter mirrors or more?
>      >>
>      >>      This is fun. Will have to watch it when I have time.
>      >>
>      >>      Is there a large active genre just now combining ChatGPT wiht deepfakes, to generate video of whomeever-saying-whatever?
>      >>
>      >>      I was thinking a couple of years ago about what direction in big-AI would be the most distructive, in requiring extra cognitive load to check what was coming in through every sense channel all the time. Certainly, as much as we must live by habit, because doing everything through the prefrontal cortex all the time is exhausting (go to a strange country, wake up in the middle of the night, where are the lightswitches in this country and how do they work?), there clearly are whole sensory modalities that we have just taken for granted as long as we could. I have assumed that the audiovisual channel of watching a person say something was near the top of that list.
>      >>
>      >>      Clearly a few years ago, deepfakes suddenly took laziness off the table for that channel. The one help was that human-generated nonsense still takes human time, on which there is some limit.
>      >>
>      >>      But if we have machine-generated nonsense, delivered through machine-generated rendering, we can put whole servers onto it full-time. Sort of like bitcoin mining. Burn a lot of irreplaceable carbon fuel to generate something of no value and some significant social cost.
>      >>
>      >>      So I assume there is some component of the society that is bored and already doing this (?)
>      >>
>      >>      Eric
>      >>
>      >>
>      >>          On Feb 28, 2023, at 9:10 PM, Gillian Densmore <gil.densmore at gmail.com <mailto:gil.densmore at gmail.com>> wrote:
>      >>
>      >>          This john oliver piece might either amus, and or mortify you.
>      >> https://www.youtube.com/watch?v=Sqa8Zo2XWc4&ab_channel=LastWeekTonight <https://www.youtube.com/watch?v=Sqa8Zo2XWc4&ab_channel=LastWeekTonight> <https://www.youtube.com/watch?v=Sqa8Zo2XWc4&ab_channel=LastWeekTonight <https://www.youtube.com/watch?v=Sqa8Zo2XWc4&ab_channel=LastWeekTonight>>
>      >>
>      >>          On Tue, Feb 28, 2023 at 4:00 PM Gillian Densmore <gil.densmore at gmail.com <mailto:gil.densmore at gmail.com>> wrote:
>      >>
>      >>
>      >>          On Tue, Feb 28, 2023 at 2:06 PM Jochen Fromm <jofr at cas-group.net <mailto:jofr at cas-group.net>> wrote:
>      >>          The "Transformer" movies are like the "Resident evil" movies based on a similar idea: we take a simple, almost primitive story such as "cars that can transform into alien robots" or "a bloody fight against a zombie apocalypse" and throw lots of money at it.
>      >>
>      >>          But maybe deep learning and large language models are the same: we take a simple idea (gradient descent learning for deep neural networks) and throw lots of money (and data) at it. In this sense transformer is a perfect name of the architecture, isn't it?
>      >>
>      >>          -J.
>      >> 😁😍🖖👍🤔
>      >>
>      >>          -------- Original message --------
>      >>          From: Gillian Densmore <gil.densmore at gmail.com <mailto:gil.densmore at gmail.com>>
>      >>          Date: 2/28/23 1:47 AM (GMT+01:00)
>      >>          To: The Friday Morning Applied Complexity Coffee Group
>      >>          <friam at redfish.com <mailto:friam at redfish.com>>
>      >>          Subject: Re: [FRIAM] Magic Harry Potter mirrors or more?
>      >>
>      >>          Transformer architecture works because it's cybertronian technology. And is so advanced as to be almost magic.
>      >>
>      >>          On Mon, Feb 27, 2023 at 3:51 PM Jochen Fromm <jofr at cas-group.net <mailto:jofr at cas-group.net>> wrote:
>      >>          Terrence Sejnowski argues that the new AI super chatbots are like a magic Harry Potter mirror that tells the user what he wants to hear: "When people discover the mirror, it seems to provide truth and understanding. But it does not. It shows the deep-seated desires of anyone who stares into it". ChatGPT, LaMDA, LLaMA and other large language models would "take in our words and reflect them back to us".
>      >> https://www.nytimes.com/2023/02/26/technology/ai-chatbot-information-t <https://www.nytimes.com/2023/02/26/technology/ai-chatbot-information-t> <https://www.nytimes.com/2023/02/26/technology/ai-chatbot-information-t <https://www.nytimes.com/2023/02/26/technology/ai-chatbot-information-t>>
>      >>          ruth.html
>      >>
>      >>          It is true that large language models have absorbed unimaginably huge amount of texts, but what if our prefrontal cortex in the brain works in the same way?
>      >> https://direct.mit.edu/neco/article/35/3/309/114731/Large-Language-Mod <https://direct.mit.edu/neco/article/35/3/309/114731/Large-Language-Mod> <https://direct.mit.edu/neco/article/35/3/309/114731/Large-Language-Mod <https://direct.mit.edu/neco/article/35/3/309/114731/Large-Language-Mod>>
>      >>          els-and-the-Reverse-Turing-Test
>      >>
>      >>          I think it is possible that the "transformer" architecture is so
>      >>          successful because it is - like the cortical columns in the neocortex
>      >>          - a modular solution for the problem what comes next in an
>      >>          unpredictable world https://en.wikipedia.org/wiki/Cortical_column <https://en.wikipedia.org/wiki/Cortical_column> <https://en.wikipedia.org/wiki/Cortical_column <https://en.wikipedia.org/wiki/Cortical_column>>
>      >>
>      >>          -J.
>      >>


-- 
ꙮ Mɥǝu ǝlǝdɥɐuʇs ɟᴉƃɥʇ' ʇɥǝ ƃɹɐss snɟɟǝɹs˙ ꙮ