[1hr Talk] Intro to Large Language Models

All of Andrej Karpathy's stuff on YouTube is absolutely top tier. He's a rare example of a brilliant contributor in his field who is an equally brilliant educator. Start with his short intro to LLMs, only 1 hour long but is dense with valuable stuff: ▶ this video

Short answer is No. I highly recommended watching Andrej Karpathy's Intro to LLMs talk, particularly the section on System 1 vs System 2 thinking. Long story short, what you are describing, using more processing to prepare a better response, is something that is an area of interest, but is not currently part of ChatGPT (or any other LLM that I am aware of). See: ▶ this video

Andrej Karpathy's "Zero to Hero" series on YouTube is the ultimate guide to building LLMs. Extremely information-dense but as complete as it gets: link Also, an amazing high-level overview of LLMs, including extensive discussion about attack vectors, that he published a couple days ago: ▶ this video

The weights are just numbers. I don't what technical background you have in other areas of computing, but I think that this is a good, short introduction that doesn't assume too much: link To quote part of it, Training a model can be thought of as tuning the dials on a really big machine. The way that a language model behaves is entirely determined by these many different continuous values, usually called parameters or weights. Longer and slightly more technical, "Intro to Large Language Models" by Andrej Karpathy: ▶ this video

Look into the training process - ▶ this video The goal of the initial pretraining phase is to make it good at predicting the next word. The rest of the training process is aimed at making it (1) helpful and (2) as correct as possible. I think some people oversimplify things by calling LLMs "next token predictors" and they leave out the tuning towards helpfulness and correctness.

In addition to the other great suggestions, point him to Karpathy's YouTube channel[1]. Karpathy has an approachable communication style. Here's his "1 hour intro to LLMs" video: ▶ this video 1. link

I suspect these videos would be higher quality than most offerings from universities or business schools (given that it's a fast moving field, and most of the true experts are practitioners, and so if they're teaching, they're not teaching actual classes, at most they're doing youtube videos or interviews): - link (Jeremy Howard Hacker's Guide to LLMs) - ▶ this video (Karpathy Intro to LLMs)

Hey, imho best overall technical intro to LLMs (I guess that´s your main interest as you mentioned qlora + llama) is by Simon Willis [1]. Additionally or if you prefer videos, the recent 1h "busy persons intro" by Andrei Karpathy is great + dense as well [2]. [1] link [2] ▶ this video EDIT: Maybe I misunderstood as you asked about papers, not general intros. I don´t think that reading papers is the best way to "catch up" as the pace is rapid and knowledge very decentralized. I can confirm what Andrej recently wrote on X [3]: "Unknown to many people, a growing amount of alpha is now outside of Arxiv, sources include but are not limited to: - link - HN - that niche Discord server - anime profile picture anons on X - reddit" [3] link

My view is to focus on doing stuff. That's what I did. Pick up some task you want the model to do, try finetuning llama, playing with APIs from OpenAI, etc. Googling and asking GPT along the way. Foundational model training got so expensive that unless you can get hired by "owns nuclear power plant of GPUs" you are not going to get any "research" done. And as the area got white-hot those companies have more available talent than hardware nowadays. So just getting into the practitioner area is the best way to get productive with those models. And you improve as a practitioner by practicing, not by reading papers. If you're at the computer, your time is best spent writing code and interacting with those models in my opinion. If you cannot (e.g. commute) I listen to some stuff (e.g. ▶ this video - Anything from Karpathy on youtube, or link channel).

* "[1hr Talk] Intro to Large Language Models" by Andrej Karpathy: ▶ this video from link * Andrej Karpathy: link

I watched Karpathy's Intro to Large Language Models [0] not so long ago and must say that I'm a bit confused by this presentation, and it's a bit unclear to me what it adds. 1,5 years ago he saw all the tool uses in agent systems as the future of LLMs, which seemed reasonable to me. There was (and maybe still is) potential for a lot of business cases to be explored, but every system is defined by its boundaries nonetheless. We still don't know all the challenges we face at that boundaries, whether these could be modelled into a virtual space, handled by software, and therefor also potentially AI and businesses. Now it all just seems to be analogies and what role LLMs could play in our modern landscape. We should treat LLMs as encapsulated systems of their own ...but sometimes an LLM becomes the operating system, sometimes it's the CPU, sometimes it's the mainframe from the 60s with time-sharing, a big fab complex, or even outright electricity itself? He's showing an iOS app, which seems to be, sorry for the dismissive tone, an example for a better looking counter. This demo app was in a presentable state for a demo after a day, and it took him a week to implement Googles OAuth2 stuff. Is that somehow exciting? What was that? The only way I could interpret this is that it just shows a big divide we're currently in. LLMs are a final API product for some, but an unoptimized generative software-model with sophisticated-but-opaque algorithms for others. Both are utterly in need fo

It might be a preview or something because I have YT premium and doesn't show up that anywhere. Can you share a video that works for that? Like this one. ▶ this video