Author Archives: bstockus

The Path Ahead

Last spring I wrote about how over the past few years I’ve continually revised and refined the scope and sequence of elementary mathematics in grades K-5 in my school district. You can read those posts here:

The tl;dr version is that I concluded the series in May 2018 with these parting thoughts:

…what started as a blog series where I was planning to reflect on the changes I might make for next year has instead reaffirmed that the work I’ve done with my teachers over the past three years has resulted in six scope and sequences that make sense and don’t actually require much tweaking at all. I’m proud of what we’ve accomplished. Are they perfect? Probably not. But they appear to be working for our teachers and students, and at the end of the day that’s what matters.

Source

Fast forward to this post I wrote reeling from my experiences at the Math Perspectives Leadership Institute in late June:

There is a HUGE disconnect between what [Kathy Richardson’s] experience says students are ready to learn in grades K-2 and what our state standards expect students to learn in those grades. I’ve been trying to reconcile this disconnect ever since, and I can tell it’s not going to be easy… I’m very conflicted right now. I’ve got two very different trajectories in front of me… Kathy Richardson is all about insight and understanding. Students are not ready to see…until they are. “We’re not in control of student learning. All we can do is stimulate learning.” Our standards on the other hand are all about getting answers and going at a pace that is likely too fast for many of our students. We end up with classrooms where many students are just imitating procedures or saying words they do not really understand. How long before these students find themselves in intervention? We blame the students (and they likely blame themselves) and put the burden on teachers down the road to try to build the foundation because we never gave it the time it deserved.

Source

What a difference a month makes.

In May I was feeling proud and confident of the work I’d accomplished developing and revising our elementary scope and sequence documents. A month later I’m calling everything into question and having a crisis of conscience about whether the scope and sequences I’ve planned are actually creating some of the struggles I was trying to prevent.

Back in July I closed my post with no answers:

But how to provide that time? That’s the question I need to explore going forward. If you were hoping for any answers in this post, I don’t have them. Rather, if you have any advice or insights, I’d love to hear them, and if I learn anything interesting along the way, I’ll be sure to share on my blog.

Source

This big question of how to reconcile the pace of learning for our youngest students with the pace of the state standards has been on my mind for months. Throughout the fall semester, I had countless conversations with colleagues in and out of my district. These conversations culminated in my taking a stab at revising our scope and sequences in grades K and 1 as well as proposing a new instructional model in grades K and 1. (Ultimately I made revisions to the scope and sequence documents for grades K-4, but I’m going to focus on K and 1 in this post.)

I’ve been sharing, talking about, and revising these document with teachers, instructional coaches, and curriculum specialists in my district for a couple of months now, and I feel like they’re finally in a shape that I want to share them here so you can see where all of this thinking has taken me since I last wrote about this in July.

As a point of reference, here are the Kindergarten and 1st grade units for the 2018-19 school year.

Kindergarten 2018-19

1st Grade 2018-19

Our curriculum is now open to the public, so if you’re interested in visiting any of these units to see unit rationales, standards, lessons, etc., you can do that here.

Contrast that with these proposed units for the 2019-20 school year:

Proposed Kindergarten 2019-20

  • Fall Semester
    • Unit 1 – I Am a Mathematician! (21 days)
    • Unit 2 – Beginning Number Concepts (30 days)
    • Unit 3 – Sorting and Classifying (30 days)
  • Spring Semester
    • Unit 4 – The Concepts of More, Less, and the Same (30 days)
    • Unit 5 – Joining and Separating Quantities (30 days)
    • Unit 6 – Building Number Concepts (30 days)

Proposed 1st Grade 2019-20

  • Fall Semester
    • Unit 1 – I Am a Mathematician! (15 days)
    • Unit 2 – Adding and Subtracting (30 days)
    • Unit 3 – Exploring Shapes and Fair Shares (27 days)
    • Unit 4 – Understanding Money (10 days)
  • Spring Semester
    • Unit 5 – More Adding and Subtracting (20 days)
    • Unit 6 – Collecting and Analyzing Data (10 days)
    • Unit 7 – Introducing Unitizing (15 days)
    • Unit 8 – Exploring the Place Value System (24 days)

Here are some of the changes and my rationale for them:

  • In Kindergarten we drastically reduced the number of units. Instead of 10 units, we’re down to 6. On top of that, the first unit has shifted from counting concepts to “I Am a Mathematician!” What does that mean? Here are the notes I took to describe this unit:
    • Exploring manipulatives
    • Exploring patterns
    • Reading books about counting, shapes, and patterns
    • Setting norms and expectations for engaging in a community of mathematicians
    • Establishing routines
    • Getting to know students’ strengths and areas of growth
  • I made the names of the units more vague. Rather than stress teachers out that their students should be counting to 5, then 10, then 20 in lockstep, I’m providing space for students to engage in number concepts in general. Teachers can differentiate as needed so students who need to work within 5 can continue to do that while other students are exploring 8 or 12 or 14.
  • I made the units in Kindergarten longer to give students time to “live” in the landscape of these concepts. This goes hand-in-hand with the new instructional model I’m proposing based on the work of Kathy Richardson. Now a typical day will include a short opening activity that’s done together as a whole class. The bulk of math time will be spent in an explore time where students self-select activities that are variations on the core concept of the unit. During this explore time, the teacher’s primary role is to confer with students and continually nudge them along in their understanding. Each day there is a short lesson close to help students reflect on their learning. Here’s a link to a sample suggested unit plan to help teachers envision what a unit might look like in grades K and 1. (Note: If you encounter a link you can’t access in the document it’s likely due to copyright that we don’t control.)
  • In 1st grade I reduced the number of units focusing on addition and subtraction. Similar to number concepts in 1st grade, I want to give students an extended amount of time to “live” in these concepts.
  • In 1st grade I moved place value to the very end of the year. According to Kathy Richardson, unitizing and place value topics are challenging for 1st graders. However, I have to include them because our state standards require it. In order to reconcile this, I want to give students as much of the year as possible for their brains to develop so they are working with the most up-to-date hardware when they start learning these critical concepts. Putting it at the end of the year also creates more proximity to when students will continue learning about place value in 2nd grade. I’ve even added a 2-digit place value unit to our 2nd grade scope and sequence to create a bridge and continue the learning.
  • In 1st grade, I created a unit just on unitizing and followed that up with a unit on place value. Using activities from Kathy Richardson’s Developing Number Concepts series, students will spend three weeks making, naming, and describing groups of 4, groups of 5, groups of 6, and eventually groups of 10. Then they’ll spend almost five weeks extending this as they learn how our place value system is built on groups of 10.

The units are just the tip of the iceberg. The math block in our district is 80 minutes and broken up across three components:

  • Focus Instruction (50 minutes)
  • Numeracy (10 minutes) – This used to be named Computational Fluency but I’m re-branding it because the names imply different goals.
  • Spiral Review (20 minutes)

So when I revised the scope and sequence documents, I also revised the learning across all three components.

Draft Kindergarten At-A-Glance 2019-20

Draft 1st Grade At-A-Glance 2019-20

Things to point out:

  • I’ve settled on a few anchor instructional routines across all grade levels – number talks, choral counting, and counting collections. That’s not to say that teachers can’t use other routines – I encourage them to – but my goal is to ensure that these three powerful, versatile routines are in everyone’s toolbox.
  • Kindergarten only has 60 minutes of math instruction in the fall semester so they don’t start spiral review until the spring semester.
  • In 1st grade the numeracy topics are fairly consistent across the year – skip counting, subitizing, making 10, and developing strategies for adding and subtracting within 20. My hope is that the consistency of topics across the year paired with the anchor instructional routines will allow the numeracy work to feel more like an ongoing conversation across the year.
  • In 1st grade creating, solving, and representing addition and subtraction problems is a spiral review topic over and over again. I want to ensure students have lots and lots of opportunities to engage with problems involving joining, separating, and comparing quantities.

Parting Thoughts

Now that I’ve started to get a plan in place, I have a lot of work ahead of me to create all the associated unit documents. I’m also going to be working on gathering teachers who want to pilot these new units. I’m wary of just dumping them on our teachers because they’ve already put so much work into learning the old units, and there are some heavy instructional shifts that might need to be made to make these units work. Thankfully I don’t think it will be too hard to find volunteers. Teachers who’ve looked at these plans and talked about them with me or their instructional coach have been really excited for the changes, so much so that I have an entire Kindergarten team trying out one of the new units right now!

While there are still a lot of unknowns and a lot of work ahead to support teachers, I do feel like all of the reflecting, conversations, and attempts at making a new plan over the past six months have brought me to a place where I feel like I’m moving in a good direction that I’m happy to follow for the time being.

Here’s to the path ahead.

The Annotated Numberless Word Problem

I recently modeled a numberless word problem in a 4th grade classroom. A few weeks later, I got an email about how the teachers were attempting to create and use some of their own, but they were encountering a problem…writing their own problems was harder than they thought!

They reached out to me for support, and I thought I’d share with you what I shared with them in case it’s helpful to anyone else creating their own numberless word problems.

1. Start with a problem

First things first, start with the problem you want to transform into a numberless word problem. Here’s the problem I started with for this example:

I type the problem on a slide, either in Powerpoint or Google Slides. You can create your problem on chart paper or on strips of paper if you’re working with a small group. I’m partial to digital slides because of some other features you’ll see later in the post.

2. Work backward

From here I create a copy of this slide and remove some of the information. Usually I start by removing the question.

Next I copy this new slide and again decide what information to remove. In this case I decided to remove the entire last sentence. That sentence dramatically changes our understanding of the situation. If you look at the slide below you’ll see that we know the total number of kids eating ice cream and the number of kids eating chocolate ice cream.

The situation is very open right now. The rest of the kids could be eating a variety of different flavors – vanilla, strawberry, chocolate chip. When I reveal the sentence that the rest of the students are eating vanilla ice cream, there’s a nice element of surprise because you aren’t necessarily expecting that the kids are only eating just two different flavors.

My next step is to remove one of the numbers. In this case I’ll take away the number of children eating chocolate ice cream.

Finally, I’ll remove the number in the first sentence to get me to the beginning of this problem. This is the first text students will read.

I structure my slides to minimize changes. I don’t want to overwhelm the students by revealing too much all at once. I will add new sentence, but I avoid changing language that’s already on the slide, if possible. More often than not I’m only changing a word like “some” into a specific quantity. There are rare instances where I’ll have to adjust a sentence as new information is added, but I try not to do that. I want the sentence structure to stay the same so that when the numbers are added that’s the only real change.

You might have noticed that I don’t include pictures on the slides with the text. This is intentional. I used to include pictures, but a colleague shared how distracting the pictures were for her students. Students were looking for meaning in them when they were only there essentially as decoration, with the intent that they would support visualizing. However, the pictures ended up confusing her students rather than helping because the students kept trying to make connections between the pictures and text. Since then I’ve avoided pictures on the text slides unless the picture is absolutely necessary.

3. Plan purposeful questions

The first step was to work backward to plan out each slide so that information is slowly revealed on each slide. Now it’s time to plan the questions I’m going to ask the students at each step along the way. I have two primary goals that I strive for in my questioning:

  1. I want students to visualize what the story is about as it unfolds. If they’re not “seeing” it, then they’re likely not making much sense of it.
  2. I want students to make guesses and estimates about quantities in the story using what they know about the situation and the relationships provided. I want them reasoning all along the way so that by the time they get to answering the question they are holding themselves accountable if their answer doesn’t make sense.

So now I go back through the slides in the order they will be presented and add the questions I plan to ask along the way.

Slide 1

Ask for a volunteer to read the story.

What are you picturing in your mind?
What do we know so far?
How many kids could be eating ice cream?
How many kids could be eating chocolate ice cream? Why do you say that?

Have students draw a quick sketch of the story so far.

Slide 2

Ask for a volunteer to read the slide.

What do we know now that we didn’t know before?
What does this tell us about the number of kids eating chocolate ice cream?

When a new slide is presented, I always ask a question to get students to state the new information. I’ve also worded this as, “What changed? What do we know now that we didn’t know before?”

Slide 3

Ask for a volunteer to read the slide.

What do we know now that we didn’t know before?
How does this number compare to our guesses? Does it make sense?
Are all of the kids eating chocolate ice cream?
What could the other kids be doing?

Slide 4

Ask for a volunteer to read the slide.

What do we know now that we didn’t know before?
What does this tell us about the number of kids eating vanilla ice cream? How do you know?

Have students draw another quick sketch of the story so far.

What question(s) could we ask about this math story?

Slide 5

What is the question asking?

Do you have all the information you need to answer that question?

Let students work on solving the problem. Confer with students as they work to look for strategies you want to bring up with the whole class.

4. The beginning and the end

Something I’ve been doing for the past year with numberless word problems is bookending them with visuals to add a little more texture to the experience.

The beginning

The first thing I do is find a high quality image or two to show the students and have them chat about before we dive into reading any text. My go-to website for images is Pixabay.

I type in a word or phrase related to the story problem, like ice cream, and more often than not I hit the jackpot:

I look for a photo that I think will capture kids’ attention and activate their prior knowledge of the context. It allows students who may be less familiar with a situation to hear the relevant language, such as ice cream, chocolate, vanilla, and cone, before we dive into reading the text.

Here’s the picture I ultimately chose to engage students at the start of this problem, along with some notes of how I’d facilitate the opening discussion with the students.

Image Source: https://pixabay.com/en/ice-ice-cream-milk-ice-cream-waffle-2367072/

What do you notice? What do you wonder? Give students 20-30 seconds of think/write time. Then let students share 1 noticing and 1 wondering with a partner. Finally let students share a few of their noticings and wonderings with the entire class. You may choose to record these in a t-chart, but it is not necessary for this problem.

Tell the students that today they are going to read a mathematical story about ice cream.

When I paste the picture on a slide, I always go into the Notes section of the slide and paste the source of the picture(s), usually the URL where I found it. On Pixabay, more often than not the photos have licenses allowing reuse.  You can find the license information to the right of each photo. I know in the privacy of your own classroom it feels easy to get away with grabbing whatever picture you can find on Google Images, but it’s good habit to pull legal photos to avoid unforseen issues down the road. And with amazing sites like Pixabay and Wikimedia Commons available, there’s no reason not to at least start by looking for freely available photos.

The ending

I’ve been making it a habit to close each numberless word problem with a short video. This serves two goals:

  1. It further builds students’ knowledge of the situation discussed. In the case of the problem I shared in this post, it was about kids eating ice cream so I found a short video of a kid making ice cream. Even if you can only find longer videos, you don’t have to show the whole thing. You could just watch the first minute (or whichever section is most relevant or interesting).
  2. It serves as a pay off for all of the hard work students just did to make sense of and solve the problem.
Here’s a link to the video I included in this problem

I’m sure you can guess where I go to find videos. YouTube has such an endless supply of videos, that I haven’t yet encountered a situation where I couldn’t find a video worth sharing. Sometimes it’s the first video and sometimes it’s the tenth, but it’s always there waiting to be discovered.

Final thoughts

Now that you’ve seen me put together this numberless word problem in pieces, here’s your chance to see the finished product. This link will take you to the slideshow for the finished product.

In the Notes section on some of the slides, you’ll see references to students sketching in boxes. I created a recording sheet to try out when I modeled a different problem recently. If you want to check out the recording sheet, here’s the link. I don’t have a lot of experience using it yet so I don’t want to say more about it right now, but I do want to share in case it’s helpful.

If you have any questions, don’t hesitate to reach out in the comments or tweet me @bstockus. And if you create your own problem, please share it with us on Twitter using the #numberlesswp hashtag.

Origin Story

This summer I was invited to give an Ignite Talk at my school district’s Teaching & Learning SummeRR Camp.

SummeRR Camp

It’s a talk I’m proud of because in five minutes I was able to share why it’s so urgent to me that we ensure sense making is the focus of our work with students. Students deserve to develop positive relationships with what they’re learning now. It’s a disservice to assume they’ll learn to like it later. We never know what doors are closed to students because they learned to hate a subject or grow up thinking they’re not smart enough.

It’s the Great (Big) Pumpkin, Charlie Brown!

At this point it’s become an annual tradition that I make a batch of numberless word problems based on the results of the Safeway World Championship Pumpkin Weigh-Off. I’ve collected together the problems I’ve written based on the results from 2016, 2017, and now 2018 in this folder. [Update: Thanks to a request from my daughter, I’ve added some primary level pumpkin problems as well.]As with all the other files I share, you are welcome to edit them if you want to tweak them for your students. All you have to do is make a copy if you have a Google account or download the file in an editable format like Power Point. You will have full editing rights of your copy.

If you’re looking for some more mathematical inspiration as Halloween approaches, check out these three blog posts I wrote which include lots of photos and ideas for how to use them to spark mathematical conversations with your students.

Enjoy!

Areas of Celebration and Exploration

After a brief interlude, it’s time to get back to the blog series I started recently about analyzing assessments.

  • In the first post, I shared the importance of digging into the questions, not just the standards they’re correlated to.
  • In the second post, I talked about how understanding how a test is designed can help us better understand the results we get.
  • In the third post, I shared how I learned to organize assessment data by item difficulty and the implications for supporting our students.
  • In this post, I’d like to talk about another way to look at assessment data to uncover areas of celebration and areas of exploration.

Let’s get started!


In my previous post I shared the order of questions based on item difficulty for the 2018 5th grade STAAR for the entire state of Texas. Here it is again:

2018-G5-Item-Difficulty-Sort

According to this ordering, question 9 was the most difficult item on the test, followed by question 18, question 8, and so on down to question 10 as the least difficult item (tied with questions 2 and 4).

Here’s my question: What is the likelihood that any given campus across the state would have the exact same order if they analyzed the item difficulty just for their students?

Hopefully you’re like me and you’re thinking, “Not very likely.” Let’s check to see. Here’s the item difficulty of the state of Texas compared to the item difficulty at just one campus with about 80 students. What do you notice? What do you wonder?

2018-G5-Texas-vs-Campus

Some of my noticings:

  • Questions 8, 9, 18, and 21 were some of the most difficult items for both the state and for this particular campus.
  • Question 5 was not particular difficulty for the state of Texas as a whole (it’s about midway down the list), but it was surprisingly difficult for this particular campus.
  • Question 22 was one of the most difficult items for the state of Texas as a whole, but it was not particularly difficult for this campus (it’s almost halfway down the list).
  • Questions 1, 2, 10, 25, and 36 were some of the least difficult items for both the state and for this particular campus.
  • Question 4 was tied with questions 2 and 10 for being the least difficult item for the state, but for this particular campus it didn’t crack the top 5 list of least difficult items.
  • There were more questions tied for being the most difficult items for the state and more questions tied for being the least difficult items for this particular campus.

My takeaway?

What is difficult for the state as a whole might not be difficult for the students at a particular school. Likewise, what is not very difficult for the state as a whole might have been more difficult than expected for the students at a particular school.

But is there an easier way to identify these differences than looking at an item on one list and then hunting it down on the second list? There is!

This image shows the item difficult rank for each question for Texas and for the campus. The final column shows the difference between these rankings.

2018-G5-Rank-Order

 

Just in case you’re having trouble making sense of it, let’s just look at question 9.

2018-G5-Rank-Order-Q9

As you can see, this was the number 1 most difficult item for the state of Texas, but it was number 3 on the same list for this campus. As a result, the rank difference is 2 because this question was 2 questions less difficult for the campus. However that’s a pretty small difference, which I interpret to mean that this question was generally about as difficult for this campus as it was for the state as a whole. What I’m curious about and interested in finding are the notable differences.

Let’s look at another example, question 5.

2018-G5-Rank-Order-Q5

This is interesting! This question was number 18 in the item difficulty for Texas, where 1 is the most difficult and 36 is the least difficult. However, this same question was number 5 in the list of questions for the campus. The rank difference is -13 because this questions was 13 questions more difficult for the campus. That’s a huge difference! I call questions like this areas of exploration. These questions are worth exploring because they buck the trend. If instruction at the campus were like the rest of Texas, this question should have been just as difficult for the campus than for the rest of the state…but it wasn’t. That’s a big red flag that I want to start digging to uncover why this question was so much more difficult. There are lots of reasons this could be the case, such as:

  • It includes a model the teachers never introduced their students to.
  • Teacher(s) at the campus didn’t know how to teach this particular concept well.
  • The question included terminology the students hadn’t been exposed to.
  • Teacher(s) at the campus skipped this content for one reason or another, or they quickly glossed over it.

In case you’re curious, here’s question 5 so you can see for yourself. Since you weren’t at the school that got this data, your guesses are even more hypothetical than there’s, but it is interesting to wonder.

2018-G5-Q5

Let me be clear. Exploring this question isn’t about placing blame. It’s about uncovering, learning what can be learned, and making a plan for future instruction so students at this campus hopefully don’t find questions like this so difficult in the future.

Let’s look at one more question from the rank order list, question 22.

2018-G5-Rank-Order-Q7

This is sort of the reverse of the previous question. Question 7 was much more difficult for the state as a whole than it was for this campus. So much so that it was 7 questions less difficult for this campus than it was for the state. Whereas question 5 is an area of exploration, I consider question 7 an area of celebration! Something going on at that campus made it so that this particular question was a lot less difficult for the students there.

  • Maybe the teachers taught that unit really well and student understanding was solid.
  • Maybe the students had encountered some problems very similar to question 7.
  • Maybe students were very familiar with the context of the problem.
  • Maybe the teachers were especially comfortable with the content from this question.

Again, in case you’re curious, here’s question 22 to get you wondering.

2018-G5-Q22

 

In Texas this is called a griddable question. Rather than being multiple choice, students have to grid their answer like this on their answer sheet:

2018-G5-Q22-Grid

Griddable items are usually some of the most difficult items on STAAR because of their demand for accuracy. That makes it even more interesting that this item was less difficult at this particular campus.

We can never know exactly why a question was significantly more or less difficult at a particular campus, but analyzing and comparing the rank orders of item difficulty does bring to the surface unexpected, and sometimes tantalizing, differences that are well worth exploring and celebrating.

Just this week I met with teams at a campus in my district to go over their own campus rank order data compared to our district data. They very quickly generated thoughtful hypotheses about why certain questions were more difficult and others were less so based on their memories of last year’s instruction. In meeting with their 5th grade team, for example, we were surprised to find that many of the questions that were much more difficult for their students involved incorrect answers that were most likely caused by calculation errors, especially if decimals were involved. That was very eye opening and got us brainstorming ideas of what we can work on together this year.


This post wraps up my series on analyzing assessment data. I might follow up with some posts specifically about the 2018 STAAR for grades 3-5 to share my analysis of questions from those assessments. At this point, however, I’ve shared the big lessons I’ve learned about how to look at assessments in new ways, particularly with regards to test design and item difficulty.

Before I go, I owe a big thank you to Dr. David Osman, Director of Research and Evaluation at Round Rock ISD, for his help and support with this work. And I also want to thank you for reading. I hope you’ve come away with some new ideas you can try in your own work!

Crystal Capture

This weekend I made something fun and wanted to share it in case it provides fun for anyone else.

My daughter has a board game called Unicorn Glitterluck.

It’s super cute, but not the most engrossing game. She and I especially like the purple cloud crystals, so this weekend I started brainstorming a math game I could make for us to play together. I know number combinations is an important idea she’ll be working on in 1st grade, so I thought about how to build a game around that while also incorporating the crystals.

Introducing…Crystal Capture!

Knowing that certain totals have greater probabilities of appearing than others, I created a game board that takes advantage of this. Totals like 6, 7, and 8 get rolled fairly frequently, so those spaces only get 1 crystal each. Totals like 2, 3, 11, and 12, on the other hand, have less chance of being rolled, so I only put 1 space above each of these numbers, but that space has 3 crystals.

I mocked up a game board and we did a little play testing. I quickly learned a few things:

Play-Test

I originally thought we would play until the board was cleared. Everything was going so well until all we had left was the one space above 12. We spent a good 15 minutes rolling and re-rolling. We just couldn’t roll a 12!! That was getting boring fast which led me to introduce a special move when you roll a double. That at least gave us something to do while we waited to finally roll a 12.

That evening I made a fancier game board in Powerpoint and we played the game again this morning:

Since clearing the board can potentially take a long time, which sucks the life out of the game, I changed the end condition. Now, if all nine of the spaces above 6, 7, and 8 are empty, the game ends. Since these numbers get rolled more frequently, the game has a much greater chance of ending without dragging on too long.

I did keep the special move when you roll doubles though. This adds a little strategic element. When you roll a double, you can replenish the crystals in any one space on the board. Will you refill a space above 6, 7, or 8 to keep the game going just a little bit longer? Or will you replenish one of the three-crystal spaces in hopes of rolling that number and claiming the crystals for yourself?

All in all, my daughter and I had a good time playing the game, and I learned a lot about where she’s at in her thinking about number combinations. Some observations:

  • She is very comfortable using her fingers to find totals.
  • Even though she knows each hand has 5 fingers, she’ll still count all 5 fingers one-at-a-time about 75% of the time.
  • She is pretty comfortable with most of her doubles. She knows double 5 is 10, for example. She gets confused whether double 3 or double 4 is 8. We rarely rolled double 6, so I have no idea what she knows about that one.
  • In the context of this game at least, she is not thinking about counting on from the larger number…yet. She doesn’t have a repertoire of strategies to help her even if she did stop and analyze the two dice. If she sees 1 and 5, she’ll put 1 finger up on one hand and 5 on the other, then she’ll count all.
  • I did see hints of some combinations slowly sinking in. That’s one benefit to dice games like this. As students continue to roll the same combinations over and over, they’ll start to internalize them.

Several folks on Twitter expressed interest in the game, so I wanted to write up this post and share the materials in case anyone out there wants to play it with their own children or students.

You’ll have to scrounge up your own crystals to put in the spaces, but even if you don’t have fancy purple ones like we do, small objects like buttons, along with a little imagination, work just as well. Oh, and if you can get your hands on sparkly dice, that helps, too. My daughter loves the sparkly dice I found in a bag of dice I had lying around.

Have fun!

Difficult

This post is the third in a series where I’m sharing how I’ve changed the ways that I look at assessments and assessment data.

  • In the first post, I shared the importance of digging into the questions, not just the standards they’re correlated to.
  • In the second post, I talked about how understanding how a test is designed can help us better understand the results we get.
  • In this post, I’d like to share one of the ways I’ve learned how to analyze assessment results.

Let’s get started!


Do you know what the most difficult item on an assessment is?

  • Is it the one with a pictograph with a scaled interval that involves combining the values from several categories?
  • Is it the multi-step story problem involving addition, subtraction, and multiplication?
  • Is it the one about matching a set of disorganized data with the correct dot plot out of four possible answer choices?

Here’s the thing I learned from Dr. Kevin Barlow, Executive Director of Research and Accountability in Arlington ISD, no matter how much time and effort someone spends designing an item, from crafting the wording to choosing just the right numbers, the only way to determine the difficulty of an item is to put it in front of students on an assessment. After students are finished, take a look at the results and find the question where the most students were incorrect.

You found it! That’s the most difficult item on the assessment.

Through their responses, our students will tell us every single time which question(s) were the most difficult for them. It’s our responsibility to analyze those questions to determine what made them so challenging.

Fortunately, the Texas Education Agency provides this information to us in Statewide Item Analysis Reports. Unfortunately, it starts out looking like this:

2018-G5-Item-Analysis-TEA

This is a great first step, but it’s not terribly useful in this format. You can’t glance at it and pick out anything meaningful. However, if I copy this data into a spreadsheet and sort it, it becomes so much more useful and meaningful:

2018-G5-Item-Difficulty-Sort

Now I’ve sorted the questions based on how students performed, from the item most students answered incorrectly (#9 was the most difficult item on this test) to the item the least number of students answered incorrectly (#2, #4, and #10 were tied for being the least difficult items on this test). It’s interesting to think that #9 and #10, back to back, turned out to be the least and most difficult for 5th graders across the state of Texas!

The items highlighted in red were the most difficult items for 5th graders. Remember, it doesn’t matter how the questions were designed. These items were the most difficult because the least number of students answered them correctly.

The items highlighted in blue, on the other hand, were the least difficult items for 5th graders in Texas. I’m intentional about calling them the least difficult items. We might be inclined to call them the easiest items, but that obscures the fact that these questions were still difficult enough that 14-17% of all Texas 5th graders answered them incorrectly. To put some real numbers with that, anywhere from 56,000 to 68,000 students answered these “easy” items incorrectly. These items were clearly difficult for these students, but they were the least difficult for the population of 5th graders as a whole.

Now what?

We might be inclined to go to the items in red and start analyzing those first. Great idea! But for whom?

Well, since they were the most difficult items, meaning the most students missed them, we should use these items to teach all of our students, right? Clearly everyone had issues with them!

I’m going to disagree with that.

These items were difficult even for some of our strongest students. If they struggled, then the last thing I want to do is bring this level of challenge to all of my students, especially those who struggled throughout the test. Rather, I’ll analyze the most difficult items to get ideas to provide challenge to my higher performing students. These kinds of questions are clearly structured in a way that gets them thinking, challenges them, and perhaps even confuses them. That’s good information to know!

(Please don’t misinterpret this as me saying that I don’t want to challenge all students. Rather, I want to ensure all students are appropriately challenged, and that’s what I’m trying to identify through this kind of analysis. Read on to see what I mean.)

But what about students who struggled throughout the test? For those students, I’m going to analyze the least difficult items. In this case, 14-17% of students in Texas answered even these items incorrectly. These items posed a challenge for quite a number of students, and I want to analyze the items to figure out what made them challenging for these students.

Let’s pretend that this is school data instead of Texas data, and let’s pretend we’re a team of 6th grade teachers analyzing 5th grade data for our 200 6th graders. That would mean at least 28-34 students in our 6th grade did not do well on these least difficult items when they took 5th grade STAAR last spring. That’s a pretty significant number of kids! They could for sure benefit from some form of intervention based on what we learn from analyzing these items.

And that’s where I’m going to leave this in your hands! Here is a document where I’ve collected the most difficult and least difficult items from the 2018 5th grade STAAR. These are the actual test questions along with the percentage of students who selected each answer choice. Spend a little time analyzing them. Here are some questions to guide you:

  • What are the features of each question? (How is the question constructed? What are its components and how are they put together in the question?)
  • Why do you suppose the features of a given question made it more/less difficult for students?
  • What mathematical knowledge and skills are required to be successful with each question?
  • What non-mathematical knowledge and skills are required to be successful with each question?
  • What can you learn from analyzing the distractors? What do they tell you about the kinds of mistakes students made or the misunderstandings they might have had?
  • What lessons can we learn from these questions to guide us in how we support our students? (We don’t want to teach our students these exact questions. That’s not terribly useful since they won’t be taking this exact test again. Rather, seek out general themes or trends that you observe in the questions that can guide your classroom instruction and/or intervention.)

I’ve opened up the document so that anyone can comment. If you’d like to share your thoughts on any of the questions, please do! I look forward to reading your thoughts about the least and most difficult items on the 2018 5th grade STAAR.

I’m giving you a very small set of questions to analyze right now. You may or may not be able to generalize much from them depending on your own experiences analyzing assessment items. However, it’s worth doing regardless of your experience, because now the repertoire of items you’ve analyzed will be that much larger.

As for myself, I’ve been analyzing assessment items like this for several years. What I’d like to do in my next post is share some of the lessons I’ve learned from this analysis across multiple years. I do feel like there are consistent trends (and a few surprises) that can inform our work in ways that simultaneously align with high-quality math instruction (because ultimately this is what I care much more about than testing) while also ensuring students are given the supports they need to succeed on mandatory high stakes tests (because they are a fact of life and it’s our responsibility to ensure students, especially those who are relying on school for this support, are prepared for them).