Khan Academy and the Effectiveness of Science Videos

Founder of StudyWand.com here, who received a 15k grant to develop an AI generating flashcard app in 2020 after an earlier prototype. We've found students more consistently study ready-made cards that are at desirable difficulty (they get about 80% correct) and which are segmented by topic (e.g. semantic grouping of flashcards to tackle "one lesson at a time" like Duolingo). Students would prefer to use pre-made flashcards by other students in their class, then AI flashcards, then create and use their own. There is limited evidence by Roediger and Karpicke who are the forefathers of retrieval practise that creating cards is also important. Frank Leeming (2002 study Exam-a-day) also showed that motivation when studying is peaked when you ask just a few questions a day, but every working day. Now one of the vital benefits of retrieval practise with AI over creating your own cards is foresight bias - not mentioned yet in this thread - the fact that particularly in some subjects like Physics, students don't know what they don't know (watch this amazing Veritasium video, it also explains why misconceptions are so handy for learning physics): ▶ this video - basically, if you use AI quizzes (or any prepared subject-specific right/wrong system), you learn quickly where your knowledge sits and what to focus on, and reduce your exam stress. If you just sit their making quizzes, firstly you make questions on things you already know, you overestimate how much you can learn, and you consolidat

Well if you want to pay great teachers to produce science videos, I would start by offering this guy a lot of money. Or at the very least, please watch the whole video because it explains one massive problem with video science lessons and how to partially fix the problem. ▶ this video

Some of the teachers I have worked with do this; and there are some great teachers(lecturers) like Martin who teach effective studying techniques: link Flashcards and Anki were incredibly effective for me. I am building a study tool which automatically generates them from lecture slides. Beyond that, it applies other psychology of learning techniques - It breaks up content into manageable chunks you can see your progress on (to boost your confidence of exam coverage) and has an exam-conditions like mode where it shows you relevant slides, then asks you questions and to freely recall by blurting onto a blank page everything you remember. It's then able to give you a realistic judgement of your learning and this way of learning by actively recalling freely is, though more intense, ultimately more effective than either reading, watching or quizzing in the long term. The part of learning that other flashcard apps miss, even Anki, is that students must be motivated. They must enjoy, visually engage, and be confident as they study. They must feel they are on a sustainable realistic path as they gather knowledge when on the way to their goals. And they don't have to study every day to do that - they just need to genuinely learn and be able to see the state of their progress... I am excited to be working on EdTech now[Revision.ai] which unveils and detects misconceptions to aid the student in eliminating them. Misconceptions are a massive opportunity - as much as spaced repetition

This makes some great points, and Martin Lombell has covered something very similar in a rather timeless one hour engaging lecture here: link Some other diamonds I've uncovered or found while working on learning psychology are: - that students who organise fun and exercise on their schedule before studying, get better grades than the students who start by deciding when to study; - that most of us can learn much better with confusing, misconception-revealing content, over simple content - we all assume too much, as Veritasium covered: ▶ this video - in principle, a desirable difficulty of varied content given in a medium that isn't lazy (read: you need to be able to get feedback, like a quiz, or rewriting, or tutoring) is the key way to establish viable long term learning; several other effects like Spaced Repetition only have benefits if you can stick to your motivation for learning, so you do need to prioritise motivation first. The article mentioned learning styles not being proven; in fact, one can argue the opposite is true; people who learn with mixed, varied content perform the best. But we also like the learning type which is comfortable for us; this boosts motivation. The problem is that studying students across a whole semester and multiple courses rarely happens, and studies focusing on how varying the learning content within a 10 minute window changed how much students learned, does not replicate the value of being familiar with a learning content style, nor the cost

"Plays into common misconceptions" is one of the worst properties that a video targeted at general audiences can have. This even happens if your video is technically correct but sounds sorta like the common misconception [1]. Someone who watched the Dr. Quantum video has learned a bunch of keywords and phrases associated with quantum mechanics. They've now heard of waves and particles and detectors and whatnot. But they have been actively misled with respect to how those things behave. I bet it's possible to put together a reasonable looking test where watching the Dr Quantum video caused scores to go down. 1: ▶ this video

Cognitive load theory is unfalsifiable and wrong about many things link Of course we have limits to how much we can do at once, but learning isn't like filling a bucket. Learning takes active work, effort. An analogy would be like saying 'weight lifting is easier with tiny weights' or 'don't lift too much weight.' Yes, but that's obvious, useless, and forgets the presumed goal of weight lifting, which requires putting in some effort. Similarly, you wouldn't force kids to listen to lectures about baseball or other sports and memorizing all the rules before letting them play. Some of the other things mentioned on the poster article are false or only apply to rote, trivial learning. Remember most of this type of traditional psych research “includes participants who have no specific interest in learning the domain involved and who are also given a very short study time.” Worked examples, like other passive learning situations, can cause an 'illusion of knowledge' - feeling like you know and understand, but not really. Look at pre-worked answers, build rote knowledge, etc, before trying to solve problems? We learn more doing the exact opposite - see research on 'productive failure'. We learn best in context, when we have a need to learn. Our intuitions about learning, teaching, etc, are often the exact opposite of reality. Here are just a couple of examples: Khan Academy and the effectiveness of science videos ▶ this video Measuring actual learning versus feeling of learning in respo

Wow, a lot of experts on education here. There are a lot of myths out there about (U.S. K12) education, such as all these "bad teachers" that we can't ever fire because of tenure, and how awful U.S. education is compared to other countries or compared to the U.S. in the 50s and 60s, and how charter schools or vouchers will solve all education's problems. There is actual data and research out there on all this, buried under all this opinion. Did you know for example that tenured teachers are twice as likely to lose their job as non-tenured teachers? link There is also a history behind all this. Before teacher tenure, teachers were regularly fired on the whim of principals or because of getting pregnant. See for example "Why Teacher Unions Are Good for Teachers—and the Public" link And see the book "50 Myths & Lies That Threaten America’s Public Schools" link As the authors state: "The mythical failure of public education has been created and perpetuated in large part by political and economic interests that stand to gain from the destruction of the traditional system." Once you get past the myths, I'd recommend doing a little reading on how people actually learn (it's not just dumping information on empty brains). See for example the book How People Learn (free, pdf): link Or for the more impatient, watch this beginning of this video Minds of Our Own: link Or this even shorter video on the counter-intuitive nature of how people learn from videos (criticizing the Khan

Sal, The only thing novel about Khan Academy is accessibility. Its the same old teacher-centered lectures we've endured for the past hundred years, with a somewhat more charismatic instructor. Most of the criticism I've seen of your work isn't "pseudo-intellectual babble," its merely pointing out what I've stated above, and that it isn't in line with what research tells us about how students learn. Your challenge for your critics to produce their own videos misses the point entirely. I happen to think your videos are fine overall, and while I find some specific science content oversimplified and focus too much on mechanics rather than conceptual understanding, it serves very well for student review. For learning new content, however, I happen to believe that there is a better way. Please view the video below: Effectiveness of Science Videos ▶ this video While you're there, consider his videos a part of your final challenge. link I am very interested in your thoughts about his approach to instructional videos.

> Effectiveness of Science Videos ▶ this video The biggest flaw in the "Effectiveness of Science" (EoS) video is that it criticizes khanacademy videos by showing that vastly inferior videos don't work. Khan videos are good because they make it easy to think: there are no distractions, just a blackboard and an intelligent voice "in your head" guiding you through the problem domain. In the EoS video, the failing videos try to teach physics by featuring the face of a talking girl with a guy juggling in the background. He might as well give his subjects a physics book and turn them loose in a sports bar to study while following NCAA basketball. Neither method is a good one for learning physics. What's sad is that the maker of the EoS video is not just some crank on the internet, but apparently a newly minted PhD in education who will likely end up teaching teachers or running a school system and it's obvious that despite his vast "education" this guy has no clue how to recognize good teaching.