Misplaced Priorities

Every spring thousands upon thousands of Texas students take the State of Texas Assessments of Academic Readiness (STAAR for short). It’s a one-day snapshot meant to evaluate a year of learning within a subject area. Even though many disagree with one-time events as assessments of learning, the fact of the matter is that they are a reality for us and our students. Because these assessments carry so much weight, we pore over the data they generate, often looking for standards where our students performed poorly so we can identify what to focus on in our instruction and intervention.

But what if I told you this well-intentioned practice may be sending us in unproductive directions? Rather than focusing on what our students really need, we may be spending time on topics and/or skills that are not the priority.

Let me illustrate what I mean with a story. I was working with a 4th grade team after a district benchmark we call STAAR Ready. Every spring in my district we give our students a released STAAR to gauge readiness for the actual STAAR coming up in May. Afterward, teams analyze the data to determine which topics to revisit and which students to put into intervention groups.

As I met with this 4th grade team, they showed me a list of the low-performing TEKS (Side note: this is what we call our standards in Texas – the Texas Essential Knowledge and Skills, TEKS for short) they had identified after analyzing the STAAR Ready data. One of the TEKS jumped out at me immediately because I was familiar with the test:

TEKS 4.4A add and subtract whole numbers and decimals to the hundredths place using the standard algorithm;

I asked them to tell me more, and the team told me they had identified students who performed poorly on the questions correlated to this standard. They created an intervention group with these students to work on adding and subtracting whole numbers and decimals to make sure they could do these computations accurately.

I followed up with a question, “Have you looked at the actual questions correlated to these TEKS?” Because they were looking at so much data and so many standards, they hadn’t gotten back into the test. Instead they’d just been identifying high-priority TEKS based on student performance on the questions.

I pulled up the test and showed them this question that had immediately come to mind when they told me they were making a group focused on TEKS 4.4A:

Source: Texas Education Agency, STAAR Math, Grade 4, Item 34

Take a moment and analyze the question.

• Can you see how it involves adding and/or subtracting with whole numbers and/or decimals?
• But what other skills are involved in answering this question correctly?
• What features of the problem might have made it more difficult for the students to answer correctly?

As it turns out, this was an incredibly difficult problem for students! When it was given to students on the actual STAAR in spring 2016, only 43% of students across the state of Texas were able to answer correctly. That means 57% of Texas 4th graders, or roughly 209,390 students, couldn’t find the total cost of three items in a shopping basket. That’s…concerning.

In my own school district, we used the 2016 released STAAR as our STAAR Ready in spring 2017. This allowed me to collect data Texas doesn’t make available to everyone. When we gave the test in spring 2017, the problem was nearly as difficult for our students. About 48% of students in my district answered it correctly. I was also able to determine this was the 6th most difficult item on the entire test of 48 questions!

What’s going on? A lot actually, for such a short question. For starters, key information is spread across two sentences. The first sentence of the problem indicates the quantities of items purchased – 1 hat and 2 skirts. The second sentence indicates their prices. This is subtle, but separating that information across two sentences upped the level of difficulty significantly for 9 and 10 year olds. Students who are not reading closely can quickly jump to the conclusion that they only need to add the two prices shown without realizing that one of those prices needs to be used twice.

The second feature of this problem that ups the difficulty is the fact that it is an open response question, not multiple choice. On this kind of question, a student’s answer has to be absolutely 100% accurate. If they’re off by even 1 penny, the answer is marked wrong. No pressure, kids!

I was curious which feature made the problem more difficult for the students in my district, so I dove into the data. One thing I had available that Texas doesn’t release is the actual answers every student submitted for this problem. I was able to analyze roughly 3,600 answers to see what students were doing. Here’s what I found out.

While only 48% of students got this question correct, there was a chunk of students whose answers were in the ballpark. These are kids who likely made a small calculation error. Unfortunately, if I calculate the percent of students who got it right or reasonably close, that only brings it up to 51% of our 4th graders. That’s not terribly impressive.

So what was everyone else doing? Here’s where it gets interesting. I predicted that these students only found the cost of 1 hat and 1 skirt, and it turns out that’s exactly what 33% of students in my district did. Nearly 1,200 students failed to comprehend that the total cost is composed of a hat, a skirt, and another skirt.

Going back to the team I was working with, I asked, “So now that we’ve analyzed this question, do you think the issue is that your students are struggling with adding and subtracting whole numbers and decimals?” We talked about it and they agreed that the bigger issue is how their students read and comprehend word problems.

Looking just at the standards is a very limiting view of analyzing data. There are often many different ways to assess a standard, and if we don’t take the time to look at the exact questions our students interact with, we might be missing critical information. Had this team done an intervention on pure addition and subtraction of whole numbers and decimals their kids would have gotten better at those skills for sure. But is that really what they needed?

Over the past year, I’ve been analyzing assessment data differently than in the past. In follow up posts I’d like to share some of that with you. In the meantime, please dive into your assessments and analyze those questions, not just the standards. You’ll hopefully come away with a truer picture of what’s challenging your students so that you can more accurately target with what and how to support them.

 

Interpretation Frustration [UPDATED]

[UPDATE] I wrote an email to TEA and heard back from them within three days. I’m very impressed! The person who wrote me went over each of my concerns one by one:

1. The example for 3(3)(E) in the side-by-side document discusses the partitioning of objects and the fraction as a concept of division, not a numerical representation. The goal would be that a student realizes that each student would receive five half-pieces of cookie. With this basis, students can the develop improper fractions and mixed numbers in grade 4.

Saying that the answer should not be a numerical representation sounds like splitting hairs. The text of the standard says: solve problems involving partitioning an object or a set of objects among two or more recipients using pictorial representations of fractions with denominators of 2, 3, 4, 6, and 8;

If I look at the phrase “solve problems” I’m led to believe I’ll get a numerical answer like I get when I solve other math problems. Yet I think he’s focusing on the phrase “pictorial representations” to say that their answer is meant to be less formal. All in all, I feel students can solve problems like sharing five cookies among 2 people, but the avoidance of 5/2 as an answer leaves me scratching my head. What purpose does it serve?

2. Much like with 3(3)(E), the focus of [3(7)(A)] is the division of the line segment. In the given example, the mark is ¼ of the distance between the numbers 16 and 17 on the number line.

This is some shady logic. The standards don’t ever mention mixed numbers in any elementary grades, but apparently they are implying that because a mixed number is composed of a whole number (3rd graders should be comfortable with those by now) and a fraction less than one (introduced in grade 3), they are fair game. I’m not against this interpretation. What I don’t like is the vague language that leaves it open to interpretation in the first place. I feel like I need to hire a lawyer to help me make sure I’m interpreting the language of these standards accurately!

3. You are correct; the last stand-alone measurement standard is 2(9)(D). However, students can be asked to measure the side lengths of a polygon in 3(7)(B).With the process code of 3(1)(A), a ruler could stand in for a number line in 3(7)(A).

And yet another example of relying on implication rather than writing standards that were clear and easy to follow in the first place. And he ends with my favorite line that I’ve heard over the years:

4. Please remember that the Texas Essential Knowledge and Skills are minimum standards and are not intended to limit what is taught.

It’s the “Get Out of Jail Free” card. I don’t think the issue is that teachers are scared of teaching beyond the standards. The problem is teachers trying to get a good grasp of what the bare minimum is in the first place. After reading through the TEKS, which are technically the standards, teachers can walk away thinking they know where the bar is. However, based on supplemental documents and email clarifications, the bar seems to be in a state of flux, leaving teachers unsure of how high their students need to jump. This doesn’t seem like a fair position to put teachers (or their students!) in.

Original post follows.

***************************************************************************

I blame the TEKS formy headache today. Specifically the grade 3 TEKS. They are not on my good side right now.

To give you some background, for the past few years I designed curriculum based on the Common Core Standards. I’ve also designed materials for Texas, but lately it was kind of secondary to the Common Core stuff. I’ve grown to love the Common Core standards. There is a lot of thought and care into the progression of topics from grade to grade. They aren’t perfect, but I value how much they do make sense, especially if you read the accompanying progressions documents.

Several years ago, Texas decided to write some new math standards. They didn’t want to adopt Common Core…because Texas…but it was clear the writing team appreciated those standards, too. The first draft of the new Texas standards had so much Common Core language in them, they may as well have been the Common Core. But then the Texas standards went through a round of revisions and what came back looked like someone had hacked off pieces of the Common Core standards, shuffled them around a bit, and called the final product new Texas standards. Needless to say, I’ve been unimpressed.

However, in my new job, I am working squarely in a Texas district in the state of Texas so the Texas standards (TEKS for short) are my focus from here on out. Lord help me.

Today, while putting together assessment materials for a grade 3 unit on fractions, I started to come across some inconsistencies in the language of the TEKS. It started with 3.3A and 3.3B:

3.3A represent fractions greater than zero and less than or equal to one with denominators of 2, 3, 4, 6, and 8 using concrete objects and pictorial models, including strip diagrams and number lines;

3.3B determine the corresponding fraction greater than zero and less than or equal to one with denominators of 2, 3, 4, 6, and 8 given a specified point on a number line;

Remember, I come from a Common Core background. Their standards say this:

Understand a fraction 1/b as the quantity formed by 1 part when a whole is partitioned into b equal parts; understand a fraction a/b as the quantity formed by a parts of size 1/b.

And this:

Represent a fraction a/b on a number line diagram by marking off a lengths 1/b from 0. Recognize that the resulting interval has size a/b and that its endpoint locates the number a/b on the number line.

Can you spot the main difference? In Common Core there is no specification that the numerator a has to create a fraction that is less than or equal to 1. You could just as easily make 5/4 as you could 3/4. In the new TEKS, however, there is a clear specification that third graders are working with fractions greater than 0 but less than or equal to 1. (By the way, what’s with the fractions having to be greater than 0? Anything wrong with discussing 0/4?)

Ok. I can handle that. But what’s this grade 2 standard over here say?

2.3C use concrete models to count fractional parts beyond one whole using words and recognize how many parts it takes to equal one whole;

Oh, so in second grade it’s okay to count fractional parts above one whole, but we need to stop in grade 3? Apparently that’s the case because improper fractions aren’t brought up again until this grade 4 standard:

4.3A represent a fraction a/b as a sum of fractions 1/b, where a and b are whole numbers and b > 0, including when a > b;

Weird. Let’s introduce an idea in grade 2, completely skip it for a year in grade 3, and come back to it in grade 4. Well, at least that’s settled…I think.

Let’s look at another grade 3 standard:

3.3E solve problems involving partitioning an object or a set of objects among two or more recipients using pictorial representations of fractions with denominators of 2, 3, 4, 6, and 8;

In the Side-by-Side comparison documents provided by the Texas Education Agency, we see the following example provided to help clarify 3.3E:

Examples of problems include situations such as 2 children sharing 5 cookies.

I can buy students solving this problem. That’s fine, but how do you rationalize the answer? You are either going to end up with 5/2 which contradicts the rigidity of 3.3A and 3.3B, or you’re going to end up with 2 ½ which is a mixed number. By the way, did I mention the term mixed number doesn’t appear in the TEKS at all across grades K-5? At all. Can you see why this might make my head hurt a bit?

My guess is that they are cheating a bit in their interpretation of 3.3A and 3.3B. By having students use mixed numbers, they are really only writing a whole number combined with a fraction less than one. Do you get it? The number 2 ½ doesn’t break their rule because the fractional part is less than 1.

So students are likely going to be held accountable for understanding mixed numbers in grade 3 even though they aren’t mentioned in the standards and several of the grade 3 standards explicitly state students work with fraction less than or equal to 1. (Good luck third grade teachers!)

I’m pretty sure this is how they are interpreting it because of how they interpret another standard. In the old TEKS we had this standard:

Old 3.10 The student is expected to locate and name points on a number line using whole numbers and fractions, including halves and fourths.

On this year’s high stake test (STAAR), the students had to locate the mixed number 16 1/4 on a number line. Do you think they would ask the same thing based on the wording of the new TEK? I sure can!

3.7A represent fractions of halves, fourths, and eighths as distances from zero on a number line;

And that’s not all! Looking at the TEKS related to fractions on a number line got me thinking about measuring to fractions of a unit. Guess what! That’s a whole new can of worms. Here is the linear measurement standard from grade 2:

2.9D determine the length of an object to the nearest marked unit using rulers, yardsticks, meter sticks, or measuring tapes;

In which grade level do you think they specify measuring to the nearest half, fourth, or eighth of an inch? If you guessed “they never specify it”, you’re right! The standard 2.9D is the FINAL linear measurement standard in the TEKS. The only mention I could find about measuring to fractions of a unit comes from the grade 5 Side-by-Side document put out by TEA. Here’s the standard:

5.4H represent and solve problems related to perimeter and/or area and related to volume.

And here’s how the Side-By-Side “clarifies” it:

Because fluency with the addition and subtraction of positive rational numbers is expected within the Revised TEKS (2012), lengths may reflect fractional measures with perimeter.

So the wording of the standards themselves never brings up fractional measures in grades K-5. The only way you would even know this grade 5 standard uses fractional measures is if you happen to cross reference it in the Side-By-Side document which is available on a completely different website from the standards themselves. I’m not even sure they’re available on the Texas Education Agency website.

Can you see why I had a headache today?

I did email someone at TEA today to request clarification. I can’t imagine I’m the only person who finds these particular standards unclear and confusing. If I hear back, I’ll be sure to share! I don’t want anyone, teachers or students, to suffer as I did today.