ChatGPT with Rob Miles - Computerphile

I think it is still a problem that many people do not know how GPT really works. There is no part in GPT that can figure out something. So no intelligence and no learning of things. A good example is when you look at GPT writing poems. All poems that it writes are of the same structure and they are not very good. So it seems it does not know the difference between a sonnet and a limmerick. But what happens is, that there is a miss-alignment between writing a poem and making the user happy. GPT is trained that its highest goal is to make the user happy. GPT believes that users dont know what sonnets are anyway so i can get away with a rhyme that is not a sonnet. Here is a good video about that: ▶ this video

This video[1] cleared up a lot of questions for me as to the issues I've seen with ChatGPT. My current operating theory goes thusly: Think back to "This Person does not Exist" [2], a site which generates a simulated human face. This works by randomly picking a vector into the latent space of human faces from a trained model, and showing it to the user. When you're using ChatGPT, you're getting a simulated assistant, at random. The quality of the answer is highly dependent on how that portion of the latent space (and thus that particular simulated assistant) was trained. Thus, as you get a wide variety of faces from this-person-does-not-exist, you'll get a wide variety of simulated assistants from ChatGPT. For me, this explains the schizophrenic nature of ChatGPT. As for the Bullshit it seems to spew, it is working against a rating system (another AI) trained to act like a human rating text. If those humans didn't know something, they had no way to express it, they simply had "pick which is best", which removes all the other dimensions of consideration, and quashes it into a scalar. The optimum strategy for the rating system is to try to BS its way through things, which then teaches ChatGPT to BS its way through things as well. Much like rating systems on HN, and all social media, this destroys information and tends to dis-incentivize nuance. [1] ▶ this video [2] link

It might help quite a bit to watch this overview of how ChatGPT works, and especially how it was trained .[1] Human attention was too expensive for the training, so they made another AI to simulate the trainers. Unfortunately, the trainer simulator training didn't have an expressive enough set of options for the actual humans, which kept a lot of quite valuable feedback out of the whole system by reducing things to single ranking. If the person didn't know the correct answer, they'd likely just go based on gut. This in turn lead the training simulator to do the same. This lead ChatGPT to just try to bullshit an answer rather than stating it didn't know, and getting downvoted. [1] ▶ this video

It depends on your definition of cheating, but this is definitely not "using humans to answer all questions one could ask". Rather it's a way to tune language models to be more like assistants rather than "most likely continuation machines". For context I recommend watching the discussion on what chatgpt is/does[0], and what the open assistant project's aim is[1]. [0]: ▶ this video [1]: link

I’ve read that before, it is fascinating. GPT3 was just trained on source text, but ChatGPT has undergone an an additional very rigorous programme of intensive training bootstrapped by human feedback. This has evolved it in order to improve its performance as an information assistant, especially on specific tasks that have proved problematic in the past. This video explains the approach and some of its limitations. ▶ this video

Also it is important to note that GPT shows large miss-alignments. The problem comes from the fact, that it is hard or impossible to give an objective what GPT should be optimized to (nobody knows the truth), so proxys are used. One proxy is that it should make the user happy and the user should give many thumbs up. But this does not mean that it has to give the "correct" answer, which the user himself might not know in the first place. So it invents things because during the reinforcement learning users were happy with these answers. A funny example is the github co-pilot which writes buggy code, because it thinks this is what the user wants. Here is a video about that: ▶ this video

> In order to resolve this, ChatGPT's training corpus needs to contain a "correct answer" next to every unique permutation of every question. This is not quite the right understanding of how ChatGPT works. It's not necessary to show ChatGPT an example of every possible permutation of an animal crossing puzzle in order for it to solve one it has never seen before. That's because the neural network is not a database of recorded word probabilities. It can instead represent the underlying logic of the puzzle, the relationships between different animals and using this abstract, pared down information, extrapolate the correct answer to the puzzle. I see the failure in the example with the goat the lion and the cabbage as simply a matter of overfitting. Edit: I see a lot of people saying "it doesn't understand logic; it's just predicting the next word." I'm basing my understanding on this video: ▶ this video The claim is that it would be impossible to feed enough input into a system such that it could produce anything as useful as ChatGPT unless it was able to abstract the underlying logic from the information provided. If you consider the he number of permutations of the animal crossing puzzle this quickly becomes clear. In fact it would be impossible for ChatGPT to produce anything brand new without this capability.

▶ this video Computerphile did an interview with Rob Miles a few days ago talking about model training, model size, and bulllshittery which he sums up in the last few moments of the video. Numerous problems exist in training that enhance bad behaviors. For example it appears that the people giving input on the responses may have a (Yes|No) voting system, but not a (Yes | No | I actually have no idea on this question) which appears it can create some interesting alignment issues.

>We need someone with a warehouse of GPUs to train a model with the parameter count of GPT3 So I'm assuming that you don't follow Rob Miles. If you do this alone you're either going to create a psychopath or something completely useless. The GPT models have no means in themselves of understanding correctness or right/wrong answers. All of these models require training and alignment functions that are typically provided by human input judging the output of the model. And we still see where this goes wrong in ChatGPT where the bot turns into a 'Yes Man' because it's aligned with giving an answer rather than saying I don't know even when it's confidence in the answer is low. Computerphile did a video on this in the last few days on this subject. ▶ this video