There is no shortage of criticism of Khan Academy around these parts. In fact, Khan Academy criticism was among the first unifying themes of the math blogosphere. Since then, however, KA has made their platform more robust and useful. And those of us who swore it off might want to take another look.
I’m taking another look because my son is using it – not because his school is assigning it, but because he wants to learn the material. This in and of itself was perplexing to me, so I decided to figure out why he was so invested that he’d want to use KA as his limited screen time for the weekend.
Part of the hook is that it comes through a screen. For whatever reason, he’s more apt to do work when it’s on a computer. He’ll struggle to write an essay on physical paper, but struggle to keep his word count low in Google Docs. He doesn’t love to draw with a pencil, but will spend hours creating an interactive image in Scratch. Or even in the KA computer science section.
So that’s one hook. There are also probably lessons to take away around customization and visualization. For example, here is my son’s “bio.”
But there are two crucial things that make KA engaging and even pedagogically sound (or at least, can push us to be more pedagogically sound).
- It lowers the pain threshold for incorrect answers. By allowing instant retakes on problem sets, there’s less of an admonishment for incorrect answers. Students who are troubled by a “7/10” on the top of a paper may be more prone to give the assignment another shot if they know that score will essentially disappear.
- It offers immediate and actionable feedback. There’s no feedback more timely than immediate. Even the hardest working teachers return tests and quizzes the following day, if not several days later. And with the immediate feedback comes suggested instructional how-to videos.
Zaretta Hammond in Culturally Responsive Teaching and the Brain offers a few characteristics of quality feedback:
[Quality feedback] is specific and in the right dose… It is timely… It is delivered in a low stress, supportive environment.
Hammond (2015) p. 103
Hammond also cites research suggesting that students of color in particular often do not receive timely, quality feedback from teachers. I’d recommend checking out the book and especially this section about the types of feedback students receive and what type of feedback they should receive. While KA is not the solution to problems around timely feedback, it is instructive.
There remain countless valid criticisms of Khan Academy. The problems are rote and allow for little in the way of problem solving. It’s anti-social. Left in the hands of a less discerning teacher, it can reinforce negative attitudes about math and about one’s self as a mathematician. Even the timely feedback mechanism has pitfalls: a student can retake tests easily and quickly, which allows them to bypass reflection about the content and why the answers are incorrect. If KA is a cornerstone of a classroom, I’d have concerns.
But there are elements that we can learn from to make our own instruction better, particularly in the realm of supportive, timely, actionable and low-stress feedback. And for pet drawings
emergent math 
The interactive explorations, especially in Geometry are what has made me change my opinion. The transformations unit is excellent. I also like CK – 12 for the same reason.
Sheryl
Sent from my iPhone
>
I love your spirit of generosity towards a durable punching bag around here, Geoff. That said …
“There’s no feedback more timely than immediate.”
What about … delayed feedback? I’m really hesitant to conflate “timely” with “immediate.” Feedback a week later isn’t great. It doesn’t follow that immediate feedback is best.
And then I have questions about the content of the feedback. You’re told you’re right. You’re told you’re wrong. (Along with a hopeful growth-mindset-y … “yet!”)
But my reading of Butler is that this kind of feedback is much less likely to be effective than feedback that gives students information about their thinking about the task. The videos and step-by-step guides offer information about how someone else thinks about the task, but not about how students think about the task.
Really great reads, Dan. Thanks for sharing these. There are certainly benefits to delayed vs. immediate feedback. You’re right: timely =/= immediate. I suppose I’m just contrasting with lived classroom experience where feedback occurs days after the attempt. And the feedback on a typical assignment is often a teacher grade, not of much higher quality than the right/wrong that KA offers.
I found this taxonomy of Live/Immediate/Delayed feedback quite useful to think about:
View at Medium.com
I feel like I offer enough caveats throughout the post that I don’t need to repeat or rebut anything, but I would just again highlight the two elements that are instructive (lowering the pain threshold and timely feedback) and contrast that with what often actually happens in classrooms (high pain and several days passing to give feedback).
I don’t think there’s any value in discussing the quality of feedback without naming all the other contextual variables. This is the big mess of feedback research — so many of the experiments have been done in feedback-poor environments, whereas the classroom is a feedback-rich environment.
In the classroom, there are so many situations where immediate feedback from a computer would be counter-productive. (Like Benny, you can sometimes induct the wrong procedure by responding to the feedback in a limited way.) There are other moments when quick feedback is closer to an ideal.
But, at home? When you’re learning on your own? That’s an environment when students really have no feedback at all. And while I haven’t used Khan Academy for my classes, I have used DeltaMath. I’ve come away very impressed:
https://problemproblems.wordpress.com/2018/11/13/homework-software-is-ready/
I do have criticisms of Khan Academy as an at-home resource. First, I think the videos are non-ideal AS explanations. (I’d prefer a completely worked out example that you could either read or call up a narrator to explain with a click, rather than the sort of casual unrolling of the solution via video. Those videos are so long and boring!) Second, I find the entire interface pretty clunky and I always am confused about the sequence of videos and problems. I think if your’e designing an at-home resource it should err on the side of dead simple, and KA’s interface is very busy.
I should clarify that I’m using DeltaMath for homework, which is not the same as advocating for its use for self-learning. Context matters!
Yeah, I sort of sidestepped the issue of what IS the actual feedback students are receiving. But I’d maintain that the feedback students often receive on assignments isn’t that much more sophisticated than what you receive in a KA quiz.
That’s interesting you consider the interface clunky. I had no experience with the UI, nor did I encourage my son to walk through it or demonstrate anything and he is able to navigate it seamlessly. Granted, that’s a data set of one, but it’s my only data set.
Geoff, I’ve spent a considerable amount of time reading all I can that comes from educators much smarter than me. I’ve read all of Dan’s posts around KA at one point or another. With all that said, I still felt value in allowing my students to tackle low-level verbs like calculate, simplify, and solve. The fluency they developed using the system supports and the immediate feedback allowed me to spend a lot more time in projects that developed conceptual understanding, Desmos activities, and other learning opportunities that I rarely felt I had time to engage in before KA. I hope that makes sense. Bottom line, KA allowed me to design and spend more time in high-quality learning opportunities because my students developed proficiency and fluency so much faster… I felt the base was stronger with KA. I still remain open-minded about developing better ways to teach math.
Interesting. I too had issues with the timely != immediate but I think that’s been cleared up. Love that your son is so creative with KA programming stuff. I think it’s too restrictive of an environment to do deep learning for older kids, but the sandbox atmosphere is very useful anyway. They make many different size hammers for exactly this reason. A teacher and a more generic tool (python, processing or p5.js, etc) will be the best atmosphere for learning flexibly and deeply. Then again, I only really know education of HS age kids, and 7 year old and under kids, so I have some holes in my intuition. 🙂
Khan Academy is a free 24-hour gym for my students. Is it perfect? Nope. Is it a useful thing for students looking to develop some skills and confidence in their mathematical ability? Well, yes I believe it is. It takes next to no time for me as a teacher to suggest it to students or parents who can’t afford expensive tutors. Time is a rare commodity for teachers, as is access to extra tuition for disadvantaged families. Actually on second thoughts, for how my students and I used it, it is perfect.
Khan Academy is *anti social*? Really?