Students hear "AI is amazing" and they believe it. Or they hear "AI is garbage" and they believe that instead. Either way, they don't have a real mental model of what it's actually good at and where it falls apart.
You can fix this in one structured demo lesson that shows both sides at the same time.
The demo works because it's not you talking about AI capabilities. It's them seeing it happen, in real time, in two columns: here's what AI did well, here's where it failed.
The demo structure (45 minutes)
Setup (5 minutes):
- Tell students: "I'm going to use an AI tool three different ways. Each time, I'll ask it something, and we'll see what it does well and where it breaks down. You're looking for: when is the output useful and when is it misleading?"
Demo 1: Something AI is actually good at (10 minutes)
*Prompt:* "Write a simple Python function that takes a list of numbers and returns the average. Include comments."
*Result:* Beautiful, correct, well-commented code. Students see it immediately.
*The conversation:* "Notice what happened. I asked for a specific thing. The AI gave me something correct and useful. I didn't have to fact-check it. This is what AI is genuinely good at: following specific, technical instructions."
Demo 2: Something AI is mediocre at (10 minutes)
*Prompt:* "Explain why some students struggle with reading. What research supports this? Cite sources."
*Result:* (Watch it generate something that sounds authoritative but the sources are made up or mischaracterized.)
*The conversation:* "I asked for sources. It gave me source *names*, but they're hallucinated. They sound real. I would have believed them if I didn't check. This is the danger. When AI sounds confident but you can't verify it, you have to fact-check everything."
Demo 3: Something AI is bad at (10 minutes)
*Prompt:* "Here's my student's essay on [topic]. What grade should it get? Be honest about the weaknesses."
*Result:* (Generic praise, missed the actual problems you see, doesn't understand the context of your class standards.)
*The conversation:* "It gave me feedback, but it doesn't know my rubric, my class, or what level of writing I expect from this student. It's not *wrong*, it's just not useful. AI can't replace judgment that depends on context."
Debrief (10 minutes):
- "So AI is great at: clear technical tasks, generating ideas when you know how to filter them, explaining things if you have context to check them."
- "AI is mediocre at: synthesis, judgment calls, anything that requires knowing your specific situation."
- "AI is bad at: anything it needs to care about getting right, situations where hallucination is dangerous, anything where you need to trust the output without fact-checking."
- "Here's the move: use it for what it's good at, don't expect it to do your thinking, always assume it might be wrong."
Variation: do it with their questions
Instead of pre-planned demos, collect questions students actually have. "Can AI write a love letter?" "Can AI predict the stock market?" "Can AI write my college essay for me?"
Run each one live. Let them see what happens.
This is better because *their* curiosity drives it, and they remember it better.
The follow-up for next week
"Now you know what AI is good at. Next assignment, you can use it for [specific thing], but not for [other thing]. Here's why: it'll do the second thing, but it won't do it well, and you'll get a worse result."
Now the rules make sense instead of feeling arbitrary.
Where this gets deeper
Our [lesson plan generator](/lesson-plan-generator) helps you structure lessons that include opportunities to test AI capabilities, and [student onboarding](/student-onboarding) sets expectations for how students should actually use these tools in your class.
The test of a demo that worked
The next time students want to use AI for something, listen to how they talk about it. Do they ask themselves "is this something AI is good at?" Do they question the output automatically? Do they try to verify?
If yes, the demo rewired their thinking.
