A year ago Tracy Zager wrote a must-read post called “My Criteria for Fact-Based Apps.” In it, she lays out her three non-negotiables for mathematics-related edtech programs:
- No time pressure
- Conceptual basis for the operations
- Mistakes must be handled properly
Tracy goes on to share two apps she does recommend, one of which is DreamBox Learning. After Tracy’s enthusiastic review, I wanted to get my hands on it and try it out. I invited a sales rep to our district for a demo, but that was underwhelming as always. For whatever reason, edtech companies tend to reveal only the briefest of glimpses of the actual student experience of their products. This is so frustrating to me!
Back in the spring I was part of a request for proposal (RFP) process that looked at various computer-based math programs. One of my biggest questions when reviewing programs is always, “What is it like for a kid using this program?” The reps will show you a few screens, but generally not enough to get a sense of what kids are really experiencing. Rather, the bulk of the time is spent talking about things like adaptive pre-assessments, teacher dashboards, and the plethora of reports that can be generated and dissected. Since the companies aren’t marketing to children, they focus their time and energy on the features that the adults will use. However, students are the ones using these products the most to (hopefully!) learn more about math. Their user experience is the one I care most about understanding and evaluating.
I did get some sample DreamBox accounts to play with, but I really wanted to see how it works in the hands of a kid, especially considering the adaptive nature of the program.
Enter my daughter, @SplashSpeaks. We’ll call her Splash for short. Splash is going to be 5 years old in March. She’s on the young side to be using the program – it says it’s designed for grades K-8 – but we have been doing so much counting and talking about numbers in our day-to-day lives that I thought it would be worth giving it a shot. Since the program is adaptive, I figured it would ensure she was in appropriate content.
Over winter break, I decided to create a personal account and start a two-week free trial. This post is about how, at least for now, I’m not going to subscribe now that the trial is over.
When Splash and I sat down at my iPad Mini to play DreamBox for the first time, she was excited to try a new app. The first few activities were a piece of cake for her. All they asked her to do was determine either “Which has more?” or “Which has less?” from two images of dots. The only challenge was paying attention enough to know which was being asked for. Her default was to assume that it was going to ask her to find the one with more. All in all, she did well enough and new activities started opening up for her.
I will say I’m impressed with the variety of representations she encountered in DreamBox. These included ten frames, dot images, math racks, and number tracks. It was interesting to see which ones resonated more with her. The math rack is definitely her favorite!
Sometimes the interactivity to complete one screen was a bit cumbersome for her. In the above example, she had to count the beads in the static image, create a representation of the same number of beads in the interactive math rack, count the number of beads again to make sure she remembered the number, count along the number track until she found the number she was looking for, click it, and then click the green arrow to indicate she’s done.
Whew!
Thankfully not every screen was this involved, but when they were, she would often skip a step. For example, she would build the number on the math rack and then jump down to the green arrow, forgetting to also select the number on the number track.
The first red flag for me that this may not be a good choice for her was her reaction whenever her answers were checked by the system. If she got the answer right, she would turn to me and smile, but if she got it wrong, she had a physical reaction of frustration. Rather than knowing it for herself, she started putting her faith in the system to tell her whether she had counted correctly. I didn’t feel comfortable with that shift in authority. I want *her* to trust that she counted correctly or built the number correctly, not wait for a computer to tell her. And I didn’t like how that subtle shift so dramatically changed her reactions to being wrong.
I will recommend that if your children use DreamBox, young ones especially, you should sit with them. There are some activities that ask for things Splash didn’t understand at first. For example, after building some numbers with the math rack, it started asking her to do it in the fewest number of moves possible. She had no idea what that meant.
Perhaps I should have said nothing and let her fail at the task. Since the system is adaptive, it might have shifted her back to other activities. However, considering how quickly the system brought her to this point in the first place, my guess is that after another activity or two she would have been prompted with these same directions.
I opted to explain to her what the phrase meant and she was able to start doing it on her own with the math rack. It was definitely more confusing with the ten frame, but even then I started seeing her grab larger chunks of dots rather than just counting out one at a time.
Here’s where another red flag came up. If you make a mistake on a screen that asked for the fewest clicks possible and then correct your mistake, the system will chide you for not getting the answer in the fewest number of moves and make you do it again. For example, let’s say you were supposed to drag 7 beads on the math rack but you mis-click and drag 6. If you drag all the beads back and then click 7, which is what my daughter did, your answer is still wrong because it counted all the clicks you made on that turn. It doesn’t matter that your last click was the efficient one.
This caused my daughter a lot of distress because she felt pressure to make sure she was completing the task perfectly, but the mix of her 5 year old hand-eye coordination and my small iPad Mini screen meant this happened somewhat frequently. She had a similar issue with the number track where she’d be counting and pointing at the numbers on the track to find the number she wanted and accidentally click one of the numbers she was counting. In certain activities there is no green check mark. If you click the number track that’s it; the system thinks that’s your answer. It was frustrating to watch her getting discouraged at being told her answer was wrong even though it was a user interface issue.
Her frustration reached a breaking point when DreamBox started introducing Quick Images activities. If you’re not familiar, an image is flashed for about 2 seconds and then covered. The user has to select the number of beads/dots that were in the image. This just blew Splash’s mind! She can identify 1, 2, and 3 on sight, but if it’s 4, 5, or greater, she relies on counting one by one. This activity made her so annoyed the first time she did it. That is, until she had an idea. She hopped up and said, “I’ll be right back!” She came back with her personal math rack:
https://twitter.com/SplashSpeaks/status/814505457641521152
Suddenly the activity became much more do-able for her. By building the images herself, she started to notice that some images only had red beads and others had red or white. If the image had red or white then she learned she only had to count the white beads. Clever girl! She still hasn’t had the “a-ha” moment that all of the red beads are 5, but it’ll happen at some point down the road. I’m not worried.
Bringing in a math tool was a lifesaver for her. She had a renewed interest in the program and felt empowered using her tool to support her thinking. That is until she started getting Quick Images with dot images. This is where I’m curious how DreamBox gauges student ability with regards to numbers to 10. I already know my daughter is super comfortable with 1, 2, and 3. She clearly needs more work on 4 and 5. Numbers 6-10 I’m less concerned about though I know she can count them accurately.
The Quick Images activity is all over the map. It would show an image of 3 dots. Cool, no problem there. But then it would follow up with an image like this:
She took one look at this and was defeated. She had no idea how many dots there were. We haven’t played a lot of dice games yet, so she doesn’t know that arrangement of 5. And she doesn’t understand counting on yet so even though she can see two orange dots, that’s not useful for finding the total.
I let her take her best guess and get it wrong. I kept telling her it’s okay. If she gets it wrong the system knows she’s not ready for that problem and will give her a different one. This is where I ran into two big problems with DreamBox. First, the way it decides what numbers to give her seems random. After getting large quantities wrong, I figured it would adjust and only give her small quantities, but it kept ping ponging back and forth showing 3, then 9, then 2, then 8, then 10. In my head I was like, “Clearly she can’t figure out the big numbers, stop giving them to her!”
The other issue has to do with the length of the activities. Normally it seems like she answers 6-8 questions and the activity is over. There’s even a visual on the screen to help show progress. For example, a long dinosaur neck is inching along the bottom of the screen towards some leaves. In this same Quick Images activity, I saw that she was close to the point where the activity normally ends, so I encouraged her to do what felt like must be the last problem. And the one after that. And the one after that. And the one after that. It never ended! The dinosaur neck just kept inching and inching and inching toward that leaf. I felt like we were trapped in Zeno’s paradox. Each time, Splash got more and more upset and frustrated until she finally broke down in tears, and that’s when I ended it. If I had known the system was capable of extending an activity that long I would have backed out of it much, much sooner. As it is, I felt terrible! I love talking about and doing math with my daughter. The last thing I want is to bring her up to and well beyond the point of frustration.
We took a break from DreamBox for a day or two. When I asked her to try it again she said, “I don’t want to do it. I don’t like it.” That made me sad. I didn’t want to stop using DreamBox on the negative note of her last activity. I wanted to help remind her about all the amazing thinking she had been doing while using the program. I encouraged her to try again, but this time we would ignore those Quick Images activities. She was hesitant, but she agreed and we ended up having a good session. The next day we played again, but this time she chose an activity that I thought was something different but it turned out to be that dreaded Quick Images activity. Aargh!
Rather than give up, I took a quick look around the dining room and saw a tub of beads. Splash wanted to get right out of the activity, but I stopped her and said, “Why don’t you try using these to build the picture like you did with your math rack?” Building with beads sounded fun so she agreed. I also prompted her to look for small groups of dots in the pictures to help her. What a difference that made! She blew me away with her subitizing skills.
I was so proud of her! She managed to build every single image thrown at her. It wasn’t until the activity ended and said, “That’s okay, we’ll try Quick Images again another time,” that I realized the system was not as impressed with her performance. Apparently she was being timed. It took her a while to build and count each image. Even though she got every single answer correct, DreamBox considered it a failure and didn’t count the activity as complete.
A day or so later our 14-day trial ended and I was left with the decision about whether I should pay for a subscription. Splash clearly demonstrated some wonderful strategizing and thinking while using DreamBox, and I was tempted to see where it would take her, but I had a feeling in my gut that it wasn’t the right decision for her.
I couldn’t quite put it into words why until a week or so later when I read chapter 2 of Tracy Zager’s new book Becoming the Math Teacher You Wish You’d Had. The chapter is titled “What Do Mathematician’s Do?” In it, she shares the story of a primary classroom where students are asked what it means to do math. Initially their answers have to do with worksheets and giving answers. The teacher and Tracy work together to develop a mini-unit to open students’ eyes to what mathematicians really do. By the end of the unit, students are beginning to understand that math is about some wonderful verbs including noticing, wondering, asking, investigating, figuring, reasoning, connecting, and proving. They’re learning that math is all around them. Reading about the experiences of these students made me want to be in that classroom, experiencing that joy of discovery with them.
And then I thought of my daughter and all of the experiences we have daily with math. I realized that DreamBox might be better than nearly every other edtech program for practicing specific skills and working through a coherent progression of ideas, but it’s not the kind of math I want my 5 year old daughter to experience. I don’t want her worrying about whether a computer is telling her her answers are correct or whether she’s taking too long to come up with them or whether she’s finding them in the most efficient way possible.
In just two weeks I already saw that path leading to frustration and negative feelings toward mathematics. No thank you.
I want to continue down the joyful, meandering path we are already on where she investigates making shapes using her body and our tile floor:
Where she wonders about the biggest shape we can possibly make with plastic strips called Exploragons:
Where we figure out important things in our daily life, such as, “How many more days until the weekend?” and where we notice and play with math:
Down the road I might revisit DreamBox for my daughter, but not anytime soon. Lest you think I’m just being a harsh critic, I will still happily recommend it for parents and teachers who have older kids. When a child has more math under their belt and you want a system to be able to flexibly move backward and forward to meet their needs, then this is a great choice. It’s not perfect, but it’s far better than other programs I’ve seen. Kent Haines said it best:
But for a child just starting out and just beginning to develop her identity and relationship with mathematics, I’ll pass.
Teaching to the Beat of a Different Drummer 