Sparks of AGI: early experiments with GPT-4

It goes back to Sparks of AGI [0] unless I am mistaken. Can recommend the talk, one that has stayed in the back of my mind since I first saw it two years ago. Personally, still have major reservations about throwing claims of intelligence or understanding around, but I do agree that SVG code generation can be a very effective source to get a quick and easy to present understanding of a models ability to output code with a rather open ended prompt that needs a high degree of coherence and were a lot of layers depend/build on each other. Helps that these are eye catching (literally as the output is visual) and easy to grasp. Same reason a lot of hype is created around the web desktops. [0] ▶ this video

Highly highly highly recommend listen to this lecture by Sebastien Bubeck (the dude who wrote the paper), it's extremely interesting: ▶ this video

If you (or anyone else is) interested in the topic, I'd highly recommend giving some of these a look: Bubeck, Sébastien, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, et al. “Sparks of Artificial General Intelligence: Early Experiments with GPT-4.” arXiv, March 27, 2023. link . or watching Sebastien Bubeck's recent talk he gave describing what GPT-4 can do that previous LLMs couldn't: ▶ this video Geoffrey Hinton recently gave a very interesting interview and he specifically wanted to address the "auto-complete" topic: link Here's another way that Ilya Sutskever recently described it (comparing GPT 4 to 3): link I'd also recommend this recent Sam Bowman article that does a goood job reviewing some of the surprising recent developments/properties of the current crop of LLMs that's pretty fascinating: Bowman, Samuel R. “Eight Things to Know about Large Language Models.” arXiv, April 2, 2023. link .

I used to say this as well, I’ve written blog posts and made videos saying exactly this. But - I would strongly recommend trying GPT4. There’s a very good video that is worth watching as well that may not change your mind, but will certainly make you think: ▶ this video The thing to remember is that these models are unbelievably huge and deep. We don’t know what is really happening in the layers or what it has really learned. Thinking that it’s a simple language model that is just predicting the next most likely word is unwise.

You should take a look at this video. He is a researcher at Microsoft and had accès to private version of ChatGPT. He literally claims that ChatGPT 4 is not as good as before. His talk actually demonstrates the different evolutions. ▶ this video

Here are two. First one is Nathan Labenz who ended up Red Teaming for OpenAI during the six months after the base model was fully trained but before it was released (especially relevant parts are after 45 minutes): link Other one is Sebastien Bubeck who put Sydney into Bing before she was tamed down some notches; he explains the lobotomization process in a particularly poignant way, where they had the GPT keep trying to draw a unicorn even as it was progressively deadened: ▶ this video

You might find this video[1] on why "GPT is smarter than you think" interesting. They're older, but the "Sparks of AGI" paper[2] and video[3] are also well worth your time. [1] - link [2] - link [3] - ▶ this video

Smart people have been falling into this trap as long as LLMs have hit production. Supposedly early internal versions of GPT-4 had "sparks of AGI" but the public version was "dumbed down for safety" ▶ this video

Disagree. We know it _can_ learn out of distribution capabilities based on similarities to other distributions. Like the TikZ Unicorn[1] (which was not in training data anywhere) or my code (which has variable names and methods/ideas probably not seen 1:1 in training). IMO this out of distribution learning is all we need to scale to AGI. Sure there are still issues, it doesn't always know which distribution to pick from. Neither do we, hence car crashes. [1]: link or on YT ▶ this video

It's in this presentation ▶ this video The most significant part I took away is that when safety "alignment" was done the ability plummeted. So that really makes me wonder how much better these models would be if they weren't lobotomized to prevent them from saying bad words.

You might think that, but there is a great deal of compression and it seems that compression is understanding, giving rise to emergent properties ▶ this video

> I think the training to censoring of models for risky questions is also screwing up their ability to give answers to non-risky questions. I’ve heard this called the “alignment tax” or “safety tax”. See [1] for pre aligned GPT-4 examples. [1] ▶ this video