A year ago Tracy Zager wrote a must-read post called “My Criteria for Fact-Based Apps.” In it, she lays out her three non-negotiables for mathematics-related edtech programs:
- No time pressure
- Conceptual basis for the operations
- Mistakes must be handled properly
Tracy goes on to share two apps she does recommend, one of which is DreamBox Learning. After Tracy’s enthusiastic review, I wanted to get my hands on it and try it out. I invited a sales rep to our district for a demo, but that was underwhelming as always. For whatever reason, edtech companies tend to reveal only the briefest of glimpses of the actual student experience of their products. This is so frustrating to me!
Back in the spring I was part of a request for proposal (RFP) process that looked at various computer-based math programs. One of my biggest questions when reviewing programs is always, “What is it like for a kid using this program?” The reps will show you a few screens, but generally not enough to get a sense of what kids are really experiencing. Rather, the bulk of the time is spent talking about things like adaptive pre-assessments, teacher dashboards, and the plethora of reports that can be generated and dissected. Since the companies aren’t marketing to children, they focus their time and energy on the features that the adults will use. However, students are the ones using these products the most to (hopefully!) learn more about math. Their user experience is the one I care most about understanding and evaluating.
I did get some sample DreamBox accounts to play with, but I really wanted to see how it works in the hands of a kid, especially considering the adaptive nature of the program.
Enter my daughter, @SplashSpeaks. We’ll call her Splash for short. Splash is going to be 5 years old in March. She’s on the young side to be using the program – it says it’s designed for grades K-8 – but we have been doing so much counting and talking about numbers in our day-to-day lives that I thought it would be worth giving it a shot. Since the program is adaptive, I figured it would ensure she was in appropriate content.
Over winter break, I decided to create a personal account and start a two-week free trial. This post is about how, at least for now, I’m not going to subscribe now that the trial is over.
When Splash and I sat down at my iPad Mini to play DreamBox for the first time, she was excited to try a new app. The first few activities were a piece of cake for her. All they asked her to do was determine either “Which has more?” or “Which has less?” from two images of dots. The only challenge was paying attention enough to know which was being asked for. Her default was to assume that it was going to ask her to find the one with more. All in all, she did well enough and new activities started opening up for her.
I will say I’m impressed with the variety of representations she encountered in DreamBox. These included ten frames, dot images, math racks, and number tracks. It was interesting to see which ones resonated more with her. The math rack is definitely her favorite!
Sometimes the interactivity to complete one screen was a bit cumbersome for her. In the above example, she had to count the beads in the static image, create a representation of the same number of beads in the interactive math rack, count the number of beads again to make sure she remembered the number, count along the number track until she found the number she was looking for, click it, and then click the green arrow to indicate she’s done.
Whew!
Thankfully not every screen was this involved, but when they were, she would often skip a step. For example, she would build the number on the math rack and then jump down to the green arrow, forgetting to also select the number on the number track.
The first red flag for me that this may not be a good choice for her was her reaction whenever her answers were checked by the system. If she got the answer right, she would turn to me and smile, but if she got it wrong, she had a physical reaction of frustration. Rather than knowing it for herself, she started putting her faith in the system to tell her whether she had counted correctly. I didn’t feel comfortable with that shift in authority. I want *her* to trust that she counted correctly or built the number correctly, not wait for a computer to tell her. And I didn’t like how that subtle shift so dramatically changed her reactions to being wrong.
I will recommend that if your children use DreamBox, young ones especially, you should sit with them. There are some activities that ask for things Splash didn’t understand at first. For example, after building some numbers with the math rack, it started asking her to do it in the fewest number of moves possible. She had no idea what that meant.
Perhaps I should have said nothing and let her fail at the task. Since the system is adaptive, it might have shifted her back to other activities. However, considering how quickly the system brought her to this point in the first place, my guess is that after another activity or two she would have been prompted with these same directions.
I opted to explain to her what the phrase meant and she was able to start doing it on her own with the math rack. It was definitely more confusing with the ten frame, but even then I started seeing her grab larger chunks of dots rather than just counting out one at a time.
Here’s where another red flag came up. If you make a mistake on a screen that asked for the fewest clicks possible and then correct your mistake, the system will chide you for not getting the answer in the fewest number of moves and make you do it again. For example, let’s say you were supposed to drag 7 beads on the math rack but you mis-click and drag 6. If you drag all the beads back and then click 7, which is what my daughter did, your answer is still wrong because it counted all the clicks you made on that turn. It doesn’t matter that your last click was the efficient one.
This caused my daughter a lot of distress because she felt pressure to make sure she was completing the task perfectly, but the mix of her 5 year old hand-eye coordination and my small iPad Mini screen meant this happened somewhat frequently. She had a similar issue with the number track where she’d be counting and pointing at the numbers on the track to find the number she wanted and accidentally click one of the numbers she was counting. In certain activities there is no green check mark. If you click the number track that’s it; the system thinks that’s your answer. It was frustrating to watch her getting discouraged at being told her answer was wrong even though it was a user interface issue.
Her frustration reached a breaking point when DreamBox started introducing Quick Images activities. If you’re not familiar, an image is flashed for about 2 seconds and then covered. The user has to select the number of beads/dots that were in the image. This just blew Splash’s mind! She can identify 1, 2, and 3 on sight, but if it’s 4, 5, or greater, she relies on counting one by one. This activity made her so annoyed the first time she did it. That is, until she had an idea. She hopped up and said, “I’ll be right back!” She came back with her personal math rack:
https://twitter.com/SplashSpeaks/status/814505457641521152
Suddenly the activity became much more do-able for her. By building the images herself, she started to notice that some images only had red beads and others had red or white. If the image had red or white then she learned she only had to count the white beads. Clever girl! She still hasn’t had the “a-ha” moment that all of the red beads are 5, but it’ll happen at some point down the road. I’m not worried.
Bringing in a math tool was a lifesaver for her. She had a renewed interest in the program and felt empowered using her tool to support her thinking. That is until she started getting Quick Images with dot images. This is where I’m curious how DreamBox gauges student ability with regards to numbers to 10. I already know my daughter is super comfortable with 1, 2, and 3. She clearly needs more work on 4 and 5. Numbers 6-10 I’m less concerned about though I know she can count them accurately.
The Quick Images activity is all over the map. It would show an image of 3 dots. Cool, no problem there. But then it would follow up with an image like this:
She took one look at this and was defeated. She had no idea how many dots there were. We haven’t played a lot of dice games yet, so she doesn’t know that arrangement of 5. And she doesn’t understand counting on yet so even though she can see two orange dots, that’s not useful for finding the total.
I let her take her best guess and get it wrong. I kept telling her it’s okay. If she gets it wrong the system knows she’s not ready for that problem and will give her a different one. This is where I ran into two big problems with DreamBox. First, the way it decides what numbers to give her seems random. After getting large quantities wrong, I figured it would adjust and only give her small quantities, but it kept ping ponging back and forth showing 3, then 9, then 2, then 8, then 10. In my head I was like, “Clearly she can’t figure out the big numbers, stop giving them to her!”
The other issue has to do with the length of the activities. Normally it seems like she answers 6-8 questions and the activity is over. There’s even a visual on the screen to help show progress. For example, a long dinosaur neck is inching along the bottom of the screen towards some leaves. In this same Quick Images activity, I saw that she was close to the point where the activity normally ends, so I encouraged her to do what felt like must be the last problem. And the one after that. And the one after that. And the one after that. It never ended! The dinosaur neck just kept inching and inching and inching toward that leaf. I felt like we were trapped in Zeno’s paradox. Each time, Splash got more and more upset and frustrated until she finally broke down in tears, and that’s when I ended it. If I had known the system was capable of extending an activity that long I would have backed out of it much, much sooner. As it is, I felt terrible! I love talking about and doing math with my daughter. The last thing I want is to bring her up to and well beyond the point of frustration.
We took a break from DreamBox for a day or two. When I asked her to try it again she said, “I don’t want to do it. I don’t like it.” That made me sad. I didn’t want to stop using DreamBox on the negative note of her last activity. I wanted to help remind her about all the amazing thinking she had been doing while using the program. I encouraged her to try again, but this time we would ignore those Quick Images activities. She was hesitant, but she agreed and we ended up having a good session. The next day we played again, but this time she chose an activity that I thought was something different but it turned out to be that dreaded Quick Images activity. Aargh!
Rather than give up, I took a quick look around the dining room and saw a tub of beads. Splash wanted to get right out of the activity, but I stopped her and said, “Why don’t you try using these to build the picture like you did with your math rack?” Building with beads sounded fun so she agreed. I also prompted her to look for small groups of dots in the pictures to help her. What a difference that made! She blew me away with her subitizing skills.
I was so proud of her! She managed to build every single image thrown at her. It wasn’t until the activity ended and said, “That’s okay, we’ll try Quick Images again another time,” that I realized the system was not as impressed with her performance. Apparently she was being timed. It took her a while to build and count each image. Even though she got every single answer correct, DreamBox considered it a failure and didn’t count the activity as complete.
A day or so later our 14-day trial ended and I was left with the decision about whether I should pay for a subscription. Splash clearly demonstrated some wonderful strategizing and thinking while using DreamBox, and I was tempted to see where it would take her, but I had a feeling in my gut that it wasn’t the right decision for her.
I couldn’t quite put it into words why until a week or so later when I read chapter 2 of Tracy Zager’s new book Becoming the Math Teacher You Wish You’d Had. The chapter is titled “What Do Mathematician’s Do?” In it, she shares the story of a primary classroom where students are asked what it means to do math. Initially their answers have to do with worksheets and giving answers. The teacher and Tracy work together to develop a mini-unit to open students’ eyes to what mathematicians really do. By the end of the unit, students are beginning to understand that math is about some wonderful verbs including noticing, wondering, asking, investigating, figuring, reasoning, connecting, and proving. They’re learning that math is all around them. Reading about the experiences of these students made me want to be in that classroom, experiencing that joy of discovery with them.
And then I thought of my daughter and all of the experiences we have daily with math. I realized that DreamBox might be better than nearly every other edtech program for practicing specific skills and working through a coherent progression of ideas, but it’s not the kind of math I want my 5 year old daughter to experience. I don’t want her worrying about whether a computer is telling her her answers are correct or whether she’s taking too long to come up with them or whether she’s finding them in the most efficient way possible.
In just two weeks I already saw that path leading to frustration and negative feelings toward mathematics. No thank you.
I want to continue down the joyful, meandering path we are already on where she investigates making shapes using her body and our tile floor:
Where she wonders about the biggest shape we can possibly make with plastic strips called Exploragons:
Where we figure out important things in our daily life, such as, “How many more days until the weekend?” and where we notice and play with math:
Down the road I might revisit DreamBox for my daughter, but not anytime soon. Lest you think I’m just being a harsh critic, I will still happily recommend it for parents and teachers who have older kids. When a child has more math under their belt and you want a system to be able to flexibly move backward and forward to meet their needs, then this is a great choice. It’s not perfect, but it’s far better than other programs I’ve seen. Kent Haines said it best:
But for a child just starting out and just beginning to develop her identity and relationship with mathematics, I’ll pass.
Teaching to the Beat of a Different Drummer 
Now you’ve got me trying to remember the blog post or web page where I read that for some students, it’s important to keep things symbolic the same for a whole lot longer than for neurotypicals.
My experience with Dreambox has also been, “ALMOST!!!”
Working with adults, my issues are slightly different: my folks need much more robust connections and practice with connecting those pictures with the language used to express them.
I love that @SlpashSpeaks figured out to use tools — and I think there’s room for making lessons where that’s part of it. I wish they’d recruit me 🙂 Except tomorrow I’ll have students coming for math help and I won’t care so much about designing stuff …
One thing I realized when playing with my son over break – I may have been overdoing it. I think the intention of the program is to be something you play with for 10-20 minutes a couple of times a week. If you play for 30 minutes three days in a row, the activities progress to the point that they’re too challenging. At least, that’s what happened to my son and your daughter (I think).
So maybe Dreambox is a side dish that shouldn’t be treated as a meal.
Agreed. I tried to limit her time to 2-3 activities, but some of the activities took her a while, especially when she brought in her math rack, so what I thought might be 10-15 minutes stretched longer. Also, with a 14-day trial, I wanted to see as much as I could. If I subscribed I’d definitely try your advice of a few days a week.
I think the issue is that they call the activities “lessons” but I’m not sure I’d agree with that term. The only lesson she’s learning from Quick Images is that it makes her cry and not want to use the program. The system offers no support in how to be successful with a Quick Images task. It’s like you get to it and suddenly you’re just supposed to be able to do it.
This makes me think of your side dish comment. I can envision a host of off-computer tasks that could help make up a meal that builds understanding that DreamBox could reinforce. Playing dice games will help her learn to recognize certain dot arrangements naturally. We can play our own version of Quick Images that’s less threatening. We have Tiny Polka Dot to play games with a variety of dot arrangements. And there are countless hands-on activities in Kathy Richardson’s Developing Number Concepts books that she can play. Not to mention activities that aren’t related to number concepts at all.
We recently bought licenses for Dreambox to be used with our intervention program. There is a lot I like about it. It’s by far better than any computer based program I’ve seen, but I agree with Brian’s concerns. Too often my students have been told they were wrong due to confusing vocabulary, dials or buttons, rather than their lack of understanding.
I love the side dish analogy. One of my biggest issues with computer programs is when people try to force them to be a steak when they should really be a salad. My students are annoyed when asked to work with the program instead of working with me more than once a week. They miss the discussion, challenges, and problem solving. With me they experience lessons. On Dreambox they practice and are assessed.
My family’s experience with DreamBox was similiar. My summary of the experience:
“Properly understood as a basic curriculum substitute or source of practice exercises, Dreambox is a solid application. Just don’t make the mistake of thinking it will either foster a love of math nor deeper mental habits.”
My full review is here: http://3jlearneng.blogspot.com/2016/07/beast-academy-and-dreambox-reviews.html
Part of the reason my criticism is less pointed is that the system has different strengths and weaknesses for different levels of learners and our three were at different stages. Also, the “adaptive” environment increased difficulty much too slowly for my older two, much too quickly for my youngest one.
FWIW, I think ST Math is probably a strong app (have a quick reference at the end of my dreambox review). However, we opted not to subscribe to ST Math when our subscription to DreamBox ended. Instead, we are focusing our time on non-electronic activities, especially board games, building materials, and baking.
Pingback: This Week: Refining, Global Math Project, and Math Interactions and Routines / Global Math Department
I just spammed you on twitter, sorry 🙂
I find this part about her reaction to right/wrong indicators fascinating:
>The first red flag for me that this may not be a good choice for her was her reaction whenever her answers were checked by the system. If she got the answer right, she would turn to me and smile, but if she got it wrong, she had a physical reaction of frustration. Rather than knowing it for herself, she started putting her faith in the system to tell her whether she had counted correctly. I didn’t feel comfortable with that shift in authority. I want *her* to trust that she counted correctly or built the number correctly, not wait for a computer to tell her. And I didn’t like how that subtle shift so dramatically changed her reactions to being wrong.
Like I said on twitter, I believe she is being reasonable when she wants to have the computer judge whether if she had counted correctly or not. She wants to know, and it will tell her… faster than she can figure out. But the system is not programmed to withhold the feedback like a parent or teacher might if they sense the child is beginning to “game” the system. On the other hand, she is upset by the wrong answers so she also feels there is not much information to act upon, only judgement.
She’s trying to play nicely as if it is a helpful tool (like the beads on the rod!) but the Dreambox only dings her and gives scant information back. “Wrong” is barely feedback.
Thank you for the detailed account of your trial!
I think the “diagnostics” are a place where so many of these programs could be improved. It isn’t *that* hard to have a “keep the numbers small” path and some “if this mistake, go this path to fix that misconception” — that’s the kind of thing Object-Oriented Programming is designed for. I could do it if I had the time…
I am a teacher/coach who uses Dreambox in 20 different K-5 classrooms. I only show a positive attitude toward Dreambox, though I too am frustrated by the immediate INCORRECT pop-up, which I heard was to help the teacher. [One who has 24 kids, perhaps 6 at a time on Dreambox, while simultaneously building numbers with a different group of 6 kids, and ensuring the other 12 kids are safe and maybe even on-task.]
I would LOVE to see Dreambox improve the math rack and ten frame features, because they are challenging physically, particularly on a computer track pad. I have shared my opinion with Dreambox, that the program’s greatest flaw, is that the lessons don’t start with smaller quantities and level up, as you noticed with the subitizing. Children should get comfortable and confident with 1-5 before 8 and 9 are introduced. And students should be comfortable with arrays of 6 X 13 before getting frustrated by 15 x 27. I am hopeful as the company has made quite a few improvements based on suggestions!
Dreambox has inspired many teachers to compose and decompose and subitize numbers in a greater variety of ways, which is good! Dreambox at home has inspired some parents to play more dice games, as you mentioned, which is GREAT!
In our school, we discuss OFTEN with our students that Dreambox is made up of code… algorithms… which are limiting. Only people can make up new ways of solving problems every day! Only people can see math in different ways at different times! Our students hear regularly that “the world is an imperfect place … and technology can be frustrating”… and Dreambox is coded to reward perfect efficiency. (I have had MANY little ones tell me when they get the dreaded RED response, that “Technology isn’t perfect!” 🙂
Of course dice and card games, and talking about numbers, and cooking and counting and comparing are THE WAY to learn about math! The best ways to learn about math are experienced at home, at the store, on a walk, or almost anywhere there aren’t textbooks. But the majority of students get few of those opportunities. And they still need to know how to use the distributive property of multiplication with visual arrays at 8 years old. Dreambox does that well. And fourth and fifth graders need to understand division by decomposing a dividend and finding partial quotients. Dreambox does that better than I ever did!
Dreambox is not without its issues, but there is still so much to celebrate. For now, I focus on the the misconceptions that are coming to light and growth my kids are making. Yesterday, a second grader practically screamed down the hall to me, “I did it! I leveled up on Dreambox! When can you come see what I can do?!”
WOOHOO!!
Hmmmm… Now that I think about it, I don’t recall any of the things I learned instantly as a kid. I only remember the struggles and eventual triumphs.
Wonderful comment! Thank you so much for sharing your experiences in a school setting since the program is designed specifically for school-age children. Thank you so much for taking the time to share such a detailed account of your own experiences. I appreciate it!
Thank you so much for starting this discussion. My son is in first grade and they use Dreambox a lot, and he struggles with other parts of the school day. The short story is, he was on half days, which meant he was not in math class (it was in the afternoon) for something like 3 months. His math skills are fine, based on what kinds of quizzes and homework he is completing, but here’s the thing: every month we get a nasty letter from, I guess, Dreambox, about how “behind” my son is with his dreambox activities. His report card says he is performing well, but there’s this other metric, that is seemingly completely divorced from the teacher and the rest of his class experience, that comes home every month. That’s creepy. I don’t want educational products exploiting this relationship and using school as a platform to demand more of my family’s time and attention. He’s in school to learn math, not to be a mark for app developers.
Pingback: dreambox 2017 – MAG 255 Abonnemang
Pingback: dreambox 2017 – IPTV Återförsäljare